Aircraft identification method, device, medium and electronic equipment based on panoramic video

Abstract

The embodiments of the present application disclose an aircraft recognition method, device, medium and electronic equipment based on panoramic video, which relate to the field of airport monitoring technology. By obtaining continuous frame images in a panoramic video for recognition, a feature point detection algorithm is used to identify the feature points of the aircraft on the original image. If the number of feature points detected on a single frame of the original image is different, it means that there are missing feature points. Since some feature points do not appear in the picture due to the movement of the aircraft, the posture change, etc., it is necessary to determine the missing type. It is confirmed that the missing feature points exist when the detection position exists. Then, feature points are obtained on the original images of other frames as a reference. The positions of the missing feature points are inferred based on the changes in these reference feature points, and the missing feature points are supplemented on the missing original image so that the number of feature points on each frame of the original image is sufficient to ensure the recognition effect of the aircraft.

Description

Technical Field

[0001] The present application relates to the field of airport monitoring technology, and in particular to a method, device, medium and electronic equipment for aircraft identification based on panoramic video. Background Art

[0002] Airport surveillance is a crucial tool for ensuring safe and efficient airport operations. Panoramic video stitching technology utilizes cameras throughout the airport to capture real-world images and assemble them into a panoramic space. Multiple images can be stitched together into a single large-scale image or 360-degree panorama, enabling real-time monitoring from the airport's monitoring center. Aircraft identification in surveillance video is a crucial aspect of airport surveillance. This not only assists with airport operations and optimizes management, but also allows for the timely detection of potential safety hazards, such as aircraft malfunctions and unauthorized intrusions.

[0003] Feature point detection is usually used to identify aircraft in video images. However, when an aircraft is in motion, motion blur will occur, affecting the detection of feature points. For example, a feature point identified in the previous frame may not be identified in the next frame. This situation is further exacerbated by the image quality of the camera feedback. The lack of feature points will obviously reduce the recognition and tracking effect of the aircraft. Summary of the Invention

[0004] The main purpose of this application is to provide an aircraft recognition method, device, medium and electronic equipment based on panoramic video, aiming to solve the problem of poor aircraft recognition effect caused by the lack of feature points in the existing technology.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for aircraft recognition based on panoramic video, comprising the following steps:

[0007] Obtaining original images of consecutive frames based on panoramic video;

[0008] Based on the feature point detection algorithm, the feature points of the target aircraft on the original image are obtained;

[0009] When the number of feature points on each frame of the original image is different, obtaining a target original image and determining the missing type of the missing feature points on the target original image;

[0010] In response to the missing type being the target missing type, obtaining reference feature points on the remaining original images; wherein the reference feature points include a first reference feature point and a second reference feature point, the first reference feature point being a feature point adjacent to the missing feature point on the target original image and corresponding to the feature point on the remaining original images, and the second reference feature point being a feature point corresponding to the missing feature point on the target original image on the remaining original images;

[0011] According to the changes of reference feature points in the original images of consecutive frames, the missing feature points are supplemented in the target original image to identify the target aircraft.

[0012] In a possible implementation of the first aspect, completing missing feature points on a target original image based on changes in reference feature points on original images of consecutive frames includes:

[0013] Obtaining a displacement of a first reference feature point on the original image of the continuous frame according to a change of the reference feature point on the original image of the continuous frame;

[0014] According to the displacement and the second reference feature point, the missing feature points are completed on the target original image.

[0015] In a possible implementation of the first aspect, completing missing feature points on the target original image according to the displacement and the second reference feature point includes:

[0016] Obtaining a first mapping relationship, based on the displacement, between the first reference feature point and the corresponding feature point on the target original image;

[0017] Adjust the first mapping relationship according to the linear change of the first reference feature point to obtain a second mapping relationship;

[0018] The second reference feature points are mapped to the target original image according to the second mapping relationship to complete the missing feature points on the target original image.

[0019] In a possible implementation of the first aspect, supplementing missing feature points in a target original image based on changes in reference feature points in original images of consecutive frames to identify a target aircraft includes:

[0020] According to the changes of the reference feature points on the original images of consecutive frames, the missing feature points are completed on the target original image;

[0021] Obtain aircraft features based on feature points on the original images of consecutive frames;

[0022] Match aircraft features in the model library to identify the target aircraft.

[0023] In a possible implementation of the first aspect, acquiring feature points of the target aircraft in the original image based on a feature point detection algorithm includes:

[0024] Based on the feature point detection algorithm, the change in the pixel value of the original image in the moving window is obtained;

[0025] According to the variation, the covariance matrix is constructed;

[0026] Calculate the corner point response value according to the eigenvalue of the covariance matrix;

[0027] The corner point response values greater than the threshold are determined to obtain the feature points of the target aircraft on the original image.

[0028] In a possible implementation manner of the first aspect, in response to the missing type being a target missing type, before acquiring reference feature points on the remaining original images, the method further includes:

[0029] The existence of the detection position but the missing feature points is defined as the target missing type.

[0030] In a possible implementation of the first aspect, acquiring original images of consecutive frames based on a panoramic video includes:

[0031] Continuous frame images are obtained based on the panoramic video, and the images are preprocessed to obtain original images of the continuous frames.

[0032] In a second aspect, an embodiment of the present application provides an aircraft identification device based on panoramic video, comprising:

[0033] An image acquisition module is used to acquire original images of consecutive frames based on a panoramic video;

[0034] A feature point detection module is used to obtain feature points of the target aircraft on the original image based on a feature point detection algorithm;

[0035] The missing judgment module is used to obtain the target original image and judge the missing type of the missing feature points on the target original image when the number of feature points on each frame of the original image is different;

[0036] A reference acquisition module, the reference acquisition module is used to obtain reference feature points on the remaining original images in response to the missing type being the target missing type; wherein the reference feature points include a first reference feature point and a second reference feature point, the first reference feature point being a feature point adjacent to the missing feature point on the target original image and corresponding to the feature point on the remaining original images, and the second reference feature point being a feature point corresponding to the missing feature point on the target original image on the remaining original images;

[0037] The recognition module is used to fill in the missing feature points on the target original image according to the changes of the reference feature points on the original images of consecutive frames to identify the target aircraft.

[0038] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, the panoramic video-based aircraft recognition method provided in any one of the first aspects above is implemented.

[0039] In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, wherein:

[0040] Memory is used to store computer programs;

[0041] The processor is used to load and execute a computer program so that the electronic device executes the aircraft recognition method based on panoramic video as provided in any one of the first aspects above.

[0042] Compared with the prior art, the present invention has the following advantages:

[0043] The embodiments of the present application propose a method, device, medium and electronic device for aircraft identification based on panoramic video, which include: obtaining original images of continuous frames based on panoramic video; obtaining feature points of the target aircraft on the original image based on a feature point detection algorithm; obtaining the target original image when the number of feature points on each frame of the original image is different, and determining the missing type of the missing feature points on the target original image; in response to the missing type being the target missing type, obtaining reference feature points on the remaining original images; wherein the reference feature points include a first reference feature point and a second reference feature point, the first reference feature point being a feature point adjacent to the missing feature point on the target original image and corresponding to the feature point on the remaining original images, and the second reference feature point being a feature point corresponding to the missing feature point on the target original image on the remaining original images; based on the changes of the reference feature points on the original images of continuous frames, the missing feature points are completed on the target original image to identify the target aircraft. This application obtains images of continuous frames in a panoramic video as original images for recognition, and first uses a feature point detection algorithm to identify the feature points of the aircraft on the original image. If the number of feature points detected on a single frame of the original image is different, it means that there are missing feature points. Since some feature points may not be identified due to picture reasons, but due to the movement of the aircraft, posture changes, etc., the detection position does not appear in the picture, so it is necessary to determine the missing type to confirm that the missing feature points exist when the detection position exists. Then, feature points are obtained on the original images of other frames as a reference, and the positions of the missing feature points are inferred through the changes in these reference feature points. The missing feature points are supplemented on the missing original image to ensure that the number of feature points on each frame of the original image is sufficient to ensure the recognition effect of the aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present application;

[0045] Figure 2 A schematic diagram of a flow chart of a method for aircraft identification based on panoramic video provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of feature points on a target original image in the panoramic video-based aircraft recognition method provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of feature points on an original image in the panoramic video-based aircraft recognition method provided in an embodiment of the present application;

[0048] Figure 5 A schematic diagram of a module of an aircraft identification device based on panoramic video provided in an embodiment of the present application;

[0049] Markings in the figure: 101 - processor, 102 - communication bus, 103 - network interface, 104 - user interface, 105 - memory. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0051] Refer to the attached Figure 1 , attached Figure 1 This is a schematic diagram of the structure of an electronic device of the hardware operating environment involved in the embodiment of the present application. The electronic device may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and optionally the user interface 104 may also include a standard wired interface and a wireless interface. The network interface 103 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 105 may optionally be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed random access memory (RAM) memory, or a stable non-volatile memory (NVM), such as at least one disk storage. The processor 101 may be a general-purpose processor, including a central processing unit, a network processor, etc., or may be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0052] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0053] As attached Figure 1 As shown, the memory 105 as a storage medium may include an operating system, a network communication module, a user interface module, and an aircraft recognition device based on panoramic video.

[0054] In the attached Figure 1In the electronic device shown, the network interface 103 is mainly used for data communication with the network server; the user interface 104 is mainly used for data interaction with the user; the processor 101 and the memory 105 in this application can be set in the electronic device, and the electronic device calls the panoramic video-based aircraft identification device stored in the memory 105 through the processor 101, and executes the panoramic video-based aircraft identification method provided in the embodiment of the present application.

[0055] Refer to the attached Figure 2 Based on the hardware device of the aforementioned embodiment, an embodiment of the present application provides an aircraft recognition method based on panoramic video, comprising the following steps:

[0056] S10: Acquire original images of consecutive frames based on the panoramic video.

[0057] In the specific implementation process, in the application scenario of airport monitoring, panoramic video, that is, the real-time monitoring screen of the airport monitoring center, is usually displayed in real time by stitching the images sent back by each camera through panoramic stitching technology. To ensure the quality of the original image of the continuous frame, that is, the frame by frame of the panoramic video, it can be pre-processed. That is, the original image of the continuous frame is obtained based on the panoramic video, including:

[0058] Continuous frame images are obtained based on the panoramic video, and the images are preprocessed to obtain original images of the continuous frames.

[0059] Image preprocessing can include denoising, contrast enhancement, color and grayscale adjustment, etc. Denoising methods include mean filtering, Gaussian filtering, bilateral filtering, etc. By preprocessing continuous frame images, the image quality can be improved and the accuracy of subsequent recognition can be improved.

[0060] S20: Based on a feature point detection algorithm, obtain feature points of the target aircraft on the original image.

[0061] In practice, feature point detection aims to automatically identify and accurately locate pixel regions with significant properties in an image. These regions are typically relatively insensitive to changes in lighting, viewing angle, or scale. These feature points can be corners, edges, or other local structures with specific patterns, providing stable identification markers for different areas of the image. Specifically, based on the feature point detection algorithm, the feature points of the target aircraft in the original image are obtained, including:

[0062] Based on the feature point detection algorithm, the change in the pixel value of the original image in the moving window is obtained;

[0063] According to the variation, the covariance matrix is constructed;

[0064] Calculate the corner point response value according to the eigenvalue of the covariance matrix;

[0065] The corner point response values greater than the threshold are determined to obtain the feature points of the target aircraft on the original image.

[0066] In the specific implementation process, a feature point detection method is provided to identify corner points as feature points. First, a moving window is set. When the window moves on the image, the change in the pixel value in the window is calculated. Then, the gradient information of the pixels in the window is used to construct a covariance matrix. This matrix describes the degree of change of the pixel values in the window in different directions. Finally, the corner point response value is calculated based on the eigenvalue of the covariance matrix. The corner point response value reflects the degree of change of the pixel value in the window in different directions. The larger the response value, the more likely there is a corner point in the window. When the response value is greater than the set threshold, the point is considered to be a corner point and is obtained as a feature point.

[0067] S30: When the number of feature points on each frame of the original image is different, a target original image is acquired, and the missing type of the missing feature points on the target original image is determined.

[0068] In the specific implementation process, the number of feature points identified on each frame of the original image is compared to determine whether there are any missing feature points. However, considering that the aircraft may be in motion, the displacement and posture changes may also affect the number of feature points identified. For example, the aircraft is partially blocked by other objects and cannot be identified, or the aircraft changes its posture and loses the position of the originally identified feature point. However, the aforementioned situations are not due to the reason of the picture that cannot be identified, that is, there is a detection position in the picture but the feature point is missing. This situation is defined as a target missing type, that is: in response to the missing type being a target missing type, before obtaining the reference feature points on the remaining original images, the method also includes: defining the existence of a detection position but missing feature points as a target missing type.

[0069] For example, if one image frame recognizes a feature point while the other image frame does not recognize the feature point at the same position in two frames, this situation is the target missing type. Similarly, if one image frame recognizes a feature point while the other image frame does not recognize the feature point, but the reason is that the aircraft posture changes or there is an object blocking the detection position, this situation is defined as the non-target missing type.

[0070] S40: In response to the missing type being the target missing type, obtaining reference feature points on the remaining original images; wherein the reference feature points include a first reference feature point and a second reference feature point, the first reference feature point being a feature point adjacent to the missing feature point on the target original image and corresponding to the feature point on the remaining original images, and the second reference feature point being a feature point corresponding to the missing feature point on the target original image on the remaining original images.

[0071] In the specific implementation process, it should be noted that the target original image in the above steps is an image that is confirmed to have missing feature points. However, after judgment, the target original image in this step should be an original image with a missing feature point type of target missing type. After confirming that the missing feature point type is the target missing type, the feature points on the target original image are supplemented by the information of the feature points on the remaining original images. As shown in the attached figure Figure 3 As shown in the figure, it represents two feature points identified on the original target image. The dotted circle indicates that there is a missing feature point. Figure 4 The original image of another frame is shown, and three feature points are identified, among which Figure 3 The missing feature points on the image correspond to the attached Figure 4 The feature point at the middle position of the image is determined as the second reference feature point, and the other two feature points adjacent to the feature point are determined as the first reference feature points.

[0072] S50: According to the changes of the reference feature points in the original images of the consecutive frames, the missing feature points are supplemented in the target original image to identify the target aircraft.

[0073] In the specific implementation process, the identification of reference feature points is not limited to a single frame image. If there are corresponding reference feature points on the original images of other frames, then these original images can be used together as the basis for completing the target original image, so that more information can be used to reflect the changes in the trajectory of the aircraft and achieve more accurate feature point completion. The completion of missing feature points is based on the changes in the reference feature points in consecutive frame images. Simply put, the same feature points at the same detection position must have a linear correlation with the frame-by-frame changes of the aircraft. In this way, the overall situation can be grasped through this linear change, so that a missing position can be completed.

[0074] After completion, the characteristics of the aircraft, such as contour features, marking features, etc., can be obtained through more complete feature point information. Using this information, not only can the aircraft be tracked in real time, but these identified features can also be matched in the established model library to identify the information of the target aircraft, such as what type, model, brand, etc. If it can be matched, it means that it is a self-owned aircraft that has been reported and filed for management. If it cannot be matched, it may be an illegally intruding aircraft and needs to be reported to the relevant personnel immediately for processing.

[0075] In one embodiment, the missing feature points are supplemented on the target original image according to the change of the reference feature points on the original image of consecutive frames, including:

[0076] Obtaining a displacement of a first reference feature point on the original image of the continuous frame according to a change of the reference feature point on the original image of the continuous frame;

[0077] According to the displacement and the second reference feature point, the missing feature points are completed on the target original image.

[0078] In the specific implementation process, the changes are first obtained through the existing and complete feature points. Specifically, the first reference feature points are used. These are the feature points identified at the corresponding detection positions in all original images. The displacement of these reference feature points in the consecutive frames is obtained. Based on the changes in these displacements, the changes that the second reference feature points should have are inferred. This leads to the locations of the missing feature points in the target original image and completes the feature points. As for the representation of the displacement, since the size of the monitoring screen does not change, it can be divided into a grid and regarded as a plane coordinate system. The displacement can be expressed as a coordinate change or a vector between feature points.

[0079] In one embodiment, completing missing feature points on the target original image according to the displacement amount and the second reference feature point includes:

[0080] Obtaining a first mapping relationship, based on the displacement, between the first reference feature point and the corresponding feature point on the target original image;

[0081] Adjust the first mapping relationship according to the linear change of the first reference feature point to obtain a second mapping relationship;

[0082] The second reference feature points are mapped to the target original image according to the second mapping relationship to complete the missing feature points on the target original image.

[0083] During the specific implementation process, the change of the feature point is converted into a mapping relationship between the feature points based on the displacement information. If it is a simple translation change, the first mapping relationship should be the same as the second mapping relationship. However, considering that the monitoring perspective cannot be completely in the same direction, and the movement of the aircraft cannot be completely translated relative to the monitoring perspective, the aircraft will inevitably have a certain degree of rotational offset. Therefore, it is not possible to completely follow the mapping relationship of the existing feature points. The linear change between the first reference feature points on the same original image caused by the rotational offset must also be considered. In other words, the same first reference feature point will show different distances on different original images according to the rotational offset. The first mapping relationship is adjusted to obtain the second mapping relationship, and then the second reference feature point is mapped back to the target original image using the mapping relationship, so that the missing feature points can be completed.

[0084] In this embodiment, continuous frames of images are obtained from a panoramic video as original images for recognition. First, a feature point detection algorithm is used to identify the feature points of the aircraft on the original image. If the number of feature points detected on a single frame of the original image is different, it means that there are missing feature points. Since some feature points may not be identified due to picture reasons, but due to the movement of the aircraft, posture change, etc., so that the detection position does not appear in the picture, it is necessary to determine the missing type to confirm that the missing feature point exists when the detection position exists. Then, feature points are obtained from the original images of other frames as a reference. The positions of the missing feature points are inferred through the changes in these reference feature points, and the missing feature points are supplemented on the missing original image to ensure that the number of feature points on each frame of the original image is sufficient to ensure the recognition effect of the aircraft.

[0085] Refer to the attached Figure 5 Based on the same inventive concept as in the aforementioned embodiment, the present embodiment further provides an aircraft identification device based on panoramic video, comprising:

[0086] An image acquisition module is used to acquire original images of consecutive frames based on a panoramic video;

[0087] A feature point detection module is used to obtain feature points of the target aircraft on the original image based on a feature point detection algorithm;

[0088] The missing judgment module is used to obtain the target original image and judge the missing type of the missing feature points on the target original image when the number of feature points on each frame of the original image is different;

[0089] A reference acquisition module, the reference acquisition module is used to obtain reference feature points on the remaining original images in response to the missing type being the target missing type; wherein the reference feature points include a first reference feature point and a second reference feature point, the first reference feature point being a feature point adjacent to the missing feature point on the target original image and corresponding to the feature point on the remaining original images, and the second reference feature point being a feature point corresponding to the missing feature point on the target original image on the remaining original images;

[0090] The recognition module is used to fill in the missing feature points on the target original image according to the changes of the reference feature points on the original images of consecutive frames to identify the target aircraft.

[0091] Those skilled in the art should understand that the division of the various modules in the embodiment is merely a division of logical functions, and in actual application, they can be fully or partially integrated into one or more actual carriers, and these modules can all be implemented in the form of software called through a processing unit, or all be implemented in the form of hardware, or in the form of a combination of software and hardware. It should be noted that the modules in the panoramic video-based aircraft identification device in this embodiment correspond one-to-one to the steps in the panoramic video-based aircraft identification method in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned panoramic video-based aircraft identification method, and will not be repeated here.

[0092] Based on the same inventive concept as in the aforementioned embodiment, an embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, the panoramic video-based aircraft identification method provided in the embodiment of the present application is implemented.

[0093] Based on the same inventive concept as in the above embodiment, an embodiment of the present application further provides an electronic device, including a processor and a memory, wherein:

[0094] Memory is used to store computer programs;

[0095] The processor is used to load and execute a computer program so that the electronic device executes the aircraft recognition method based on panoramic video as provided in the embodiment of the present application.

[0096] In some embodiments, the computer-readable storage medium may be a memory device such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface mount memory, optical disk, or CD-ROM; or various devices including any one or any combination of the above memories. The computer may be various computing devices including smart terminals and servers.

[0097] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0098] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0099] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0100] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0101] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0102] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0103] In summary, the present application provides an aircraft identification method, device, medium and electronic device based on panoramic video, which method includes: obtaining original images of continuous frames based on panoramic video; obtaining feature points of the target aircraft on the original image based on a feature point detection algorithm; when the number of feature points on each frame of the original image is different, obtaining the target original image, and judging the missing type of the missing feature points on the target original image; in response to the missing type being the target missing type, obtaining reference feature points on the remaining original images; wherein the reference feature points include a first reference feature point and a second reference feature point, the first reference feature point being a feature point adjacent to the missing feature point on the target original image, and corresponding to the feature point on the remaining original images, and the second reference feature point being a feature point corresponding to the missing feature point on the target original image on the remaining original images; according to the changes of the reference feature points on the original images of continuous frames, the missing feature points are completed on the target original image to identify the target aircraft. This application obtains images of continuous frames in a panoramic video as original images for recognition, and first uses a feature point detection algorithm to identify the feature points of the aircraft on the original image. If the number of feature points detected on a single frame of the original image is different, it means that there are missing feature points. Since some feature points may not be identified due to picture reasons, but due to the movement of the aircraft, posture changes, etc., the detection position does not appear in the picture, so it is necessary to determine the missing type to confirm that the missing feature points exist when the detection position exists. Then, feature points are obtained on the original images of other frames as a reference, and the positions of the missing feature points are inferred through the changes in these reference feature points. The missing feature points are supplemented on the missing original image to ensure that the number of feature points on each frame of the original image is sufficient to ensure the recognition effect of the aircraft.

[0104] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for aircraft recognition based on panoramic video, characterized in that: The following steps are involved: Obtaining original images of consecutive frames based on panoramic video; Acquire feature points of the target aircraft on the original image based on a feature point detection algorithm; When the number of the feature points on each frame of the original image is different, obtaining a target original image, and determining the missing type of the missing feature points on the target original image; In response to the missing type being a target missing type, obtaining reference feature points on the remaining original images; wherein the reference feature points include a first reference feature point and a second reference feature point, the first reference feature point being the feature point adjacent to the missing feature point on the target original image and corresponding to the feature point on the remaining original images, the second reference feature point being the feature point corresponding to the missing feature point on the target original image on the remaining original images, and the target missing type being a missing feature point at a detection position; The missing feature points are supplemented on the target original image according to changes in the reference feature points on the original image of consecutive frames to identify the target aircraft.

2. The method for aircraft recognition based on panoramic video according to claim 1, characterized in that: The method of completing the missing feature points on the target original image according to the change of the reference feature points on the original image of consecutive frames includes: Obtaining a displacement of the first reference feature point on the original image of consecutive frames according to a change of the reference feature point on the original image of consecutive frames; The missing feature points are completed on the target original image according to the displacement amount and the second reference feature points.

3. The method for aircraft recognition based on panoramic video according to claim 2, characterized in that: The step of filling in the missing feature points on the target original image according to the displacement and the second reference feature points includes: Obtaining, according to the displacement, a first mapping relationship in which the first reference feature point is mapped to the corresponding feature point on the target original image; Adjust the first mapping relationship according to the linear change of the first reference feature point to obtain a second mapping relationship; The second reference feature points are mapped to the target original image according to the second mapping relationship to complete the missing feature points on the target original image.

4. The method for aircraft recognition based on panoramic video according to claim 1, characterized in that: The method of supplementing the missing feature points on the target original image according to the change of the reference feature points on the original images of consecutive frames to identify the target aircraft includes: Completing the missing feature points on the target original image according to changes in the reference feature points on the original image of consecutive frames; Obtaining aircraft features based on the feature points on the original images of consecutive frames; The aircraft features are matched in a model library to identify the target aircraft.

5. The method for aircraft recognition based on panoramic video according to claim 1, characterized in that: The step of obtaining the feature points of the target aircraft on the original image based on the feature point detection algorithm includes: Based on a feature point detection algorithm, obtaining a change in pixel values of the original image within a moving window; constructing a covariance matrix based on the variation; Calculating corner point response values according to the eigenvalues of the covariance matrix; The corner point response value greater than a threshold is determined to obtain feature points of the target aircraft on the original image.

6. The method for aircraft recognition based on panoramic video according to claim 1, characterized in that: In response to the missing type being a target missing type, before acquiring reference feature points on the remaining original images, the method further includes: The target missing type is defined as the existence of the detection position but the missing feature point.

7. The method for aircraft recognition based on panoramic video according to claim 1, characterized in that: The method of obtaining original images of consecutive frames based on the panoramic video includes: Continuous frame images are acquired based on the panoramic video, and the images are preprocessed to acquire original images of the continuous frames.

8. An aircraft identification device based on panoramic video, characterized in that: include: An image acquisition module, configured to acquire original images of consecutive frames based on a panoramic video; A feature point detection module, configured to obtain feature points of the target aircraft on the original image based on a feature point detection algorithm; a missing feature point determination module, configured to obtain a target original image and determine a missing feature point type on the target original image when the number of feature points on each frame of the original image is different; a reference acquisition module, the reference acquisition module being configured to acquire reference feature points on the remaining original images in response to the missing type being a target missing type; wherein the reference feature points include a first reference feature point and a second reference feature point, the first reference feature point being the feature point adjacent to the missing feature point on the target original image and corresponding to the feature point on the remaining original images, the second reference feature point being the feature point corresponding to the missing feature point on the target original image on the remaining original images, and the target missing type being a missing feature point at a detection position; The recognition module is used to complete the missing feature points on the target original image according to the changes of the reference feature points on the original images of consecutive frames to identify the target aircraft.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is loaded and executed by a processor, the method for aircraft recognition based on panoramic video according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: comprising a processor and a memory, wherein: The memory is used to store computer programs; The processor is configured to load and execute the computer program so as to enable the electronic device to perform the method for aircraft recognition based on panoramic video according to any one of claims 1 to 7.

Images