Method and device for seeker to identify and track maneuvering target, electronic equipment and medium

Abstract

The invention provides a method and a device for a seeker to identify and track a maneuvering target, electronic equipment and a medium, the method is applied to the seeker, and the method comprises the following steps: preprocessing each first initial image of the maneuvering target, and carrying out first target identification detection based on each preprocessed first initial image and an improved YOLO neural network model; if the maneuvering target is detected for the first time, determining a matching result of the moving state information of the maneuvering target and the predicted position of the next frame; if the matching is successful, controlling the seeker to track the maneuvering target in the first scene until entering a second scene; performing second target identification detection on the maneuvering target in a second scene by adopting an improved YOLO neural network model; if the maneuvering target is detected again, the seeker is controlled to be aligned with the maneuvering target until the seeker makes contact with the maneuvering target; and if the matching is unsuccessful, the improved YOLO neural network model is triggered to perform target identification detection again, so that the complexity of target identification tracking is reduced, and the identification tracking accuracy is improved.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to a method, device, electronic device and storage medium for a seeker to identify and track a maneuvering target. Background Art

[0002] At present, with the increasing mobility of various targets, it is becoming increasingly difficult for the seeker to track the maneuvering targets in real time and effectively.

[0003] Due to the rapid development of military, aerospace and civil fields, higher requirements are placed on high-precision and high-robust mobile target tracking technology. Traditional target recognition and tracking methods have problems such as high computational complexity, poor real-time performance, and susceptibility to environmental interference. In addition, they cannot be recognized in time after the mobile target enters another scene, which also reduces the recognition and tracking accuracy of the seeker.

[0004] The prior art seeker identification and tracking method for maneuvering targets has the problems of high complexity, poor real-time performance and low identification and tracking accuracy. Summary of the invention

[0005] The embodiments of the present application provide a method, device, electronic device and medium for a seeker to identify and track a maneuvering target, which can solve the problems of high complexity, poor real-time performance and low identification and tracking accuracy in the identification and tracking method of maneuvering targets.

[0006] In a first aspect, an embodiment of the present application provides a method for a seeker to identify and track a maneuvering target, which is applied to the seeker and includes:

[0007] Acquire a plurality of first initial images of the maneuvering target in a first scene;

[0008] Preprocessing the received first initial images to obtain preprocessed first initial images;

[0009] Performing first target recognition detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model;

[0010] If the maneuvering target is detected for the first time in each of the first initial images, determining a matching result between the movement state information of the maneuvering target and a predicted position of the maneuvering target in a next frame based on the movement state information of the maneuvering target and a tracker;

[0011] If the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, control the seeker to track the maneuvering target in the first scene until the maneuvering target enters a second scene, where the second scene is any scene where the maneuvering target enters other than the first scene;

[0012] Performing second target recognition and detection on the maneuvering target based on each second initial image of the maneuvering target in the second scene and the trained second YOLO neural network model;

[0013] If the maneuvering target is detected again in each of the second initial images, controlling the seeker to aim at the maneuvering target until it contacts the maneuvering target;

[0014] If the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames, triggering the trained first YOLO neural network model or the trained second YOLO neural network model to re-perform target recognition detection on the maneuvering target;

[0015] Among them, N≧3, the first YOLO neural network model and the second YOLO neural network model are both improved YOLO neural network models.

[0016] In one embodiment, preprocessing each received first initial image to obtain each preprocessed first initial image includes:

[0017] Performing noise reduction processing based on the noise type of each of the first initial images to obtain a corresponding first noise-reduced initial image;

[0018] Performing histogram processing on each of the first denoised initial images to obtain a corresponding first histogram initial image;

[0019] Randomly splicing a preset number of the first histogram initial images into a large image to obtain a plurality of first spliced ​​initial images;

[0020] Each of the first spliced ​​initial images is divided into a plurality of grids of preset pixel areas to obtain each of the preprocessed first initial images.

[0021] In one embodiment, the trained first YOLO neural network model includes a backbone network, a neck network, and a head improvement prediction end;

[0022] Performing first target recognition and detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model includes:

[0023] Based on the training set and the test set of the maneuvering target in the first scene, obtaining the trained first YOLO neural network model;

[0024] The backbone network of the trained first YOLO neural network model extracts features corresponding to the maneuvering target from each of the preprocessed first initial images;

[0025] The neck network of the trained first YOLO neural network model aggregates and refines features corresponding to the maneuvering target;

[0026] The head improvement prediction end of the trained first YOLO neural network model outputs the predicted bounding box of the maneuvering target in each of the preprocessed first initial images to perform first target recognition detection on the maneuvering target.

[0027] In one of the embodiments, the predicted bounding box includes the existence recognition probability of the maneuvering target, the target category probability and the bounding box position information, and the head improvement prediction end includes a confidence loss function, a category loss function and an improved position loss function;

[0028] The head improvement prediction end of the trained first YOLO neural network model outputs the predicted bounding box of the maneuvering target in each of the preprocessed first initial images, including:

[0029] The head improvement prediction end outputs the existence recognition probability of the maneuvering target in each of the preprocessed first initial images through a confidence loss function;

[0030] The head improvement prediction end outputs the target category probability of the maneuvering target in each of the preprocessed first initial images through a category loss function;

[0031] The head improvement prediction end outputs the bounding box position information of the maneuvering target in each of the preprocessed first initial images by improving the position loss function.

[0032] In one embodiment, the improved position loss function outputs the bounding box position information of the maneuvering target in each of the preprocessed first initial images, including:

[0033] Determine the improved position loss function value based on the intersection-over-union ratio, the center Euclidean distance, the minimum diagonal length, the maximum diagonal length, the balance parameter and the difference parameter;

[0034] Outputting the bounding box position information of the maneuvering target based on the improved position loss function value and the movement state information of the maneuvering target in each of the preprocessed first initial images;

[0035] Among them, the central Euclidean distance is the Euclidean distance between the center points of the predicted box and the real box, the minimum diagonal length is the minimum diagonal length in the minimum closed box of the predicted box and the real box, the maximum diagonal length is the maximum diagonal length in the minimum closed box of the predicted box and the real box, and the difference parameter is the difference in the aspect ratio between the predicted box and the real box.

[0036] In one embodiment, the movement state information includes pixel position, speed and acceleration, the pixel position represents the pixel coordinates of the centroid of the maneuvering target in the image frame;

[0037] If the maneuvering target is detected for the first time in each of the first initial images, determining a matching result between the movement state information of the maneuvering target and a predicted position of the maneuvering target in a next frame based on the movement state information of the maneuvering target and a tracker, including:

[0038] If the maneuvering target is detected for the first time in each of the first initial images, the pixel position, velocity and acceleration of the maneuvering target in the current frame and the pixel position, velocity and acceleration in the next frame are respectively obtained;

[0039] The tracker obtains a predicted position of the maneuvering target in a next frame based on the pixel position, velocity and acceleration of the maneuvering target in a current frame;

[0040] The pixel position, speed and acceleration of the maneuvering target in the next frame are matched with the predicted position of the maneuvering target in the next frame, and a matching result is determined.

[0041] In one embodiment, the movement state information includes pixel position, speed and acceleration;

[0042] If the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, controlling the seeker to track the maneuvering target in the first scene until the maneuvering target enters the second scene, comprising:

[0043] If the pixel position, velocity and acceleration of the maneuvering target in the next frame are respectively matched successfully with the predicted position of the maneuvering target in the next frame, the pixel position, velocity and acceleration of the maneuvering target after the next frame are continuously acquired;

[0044] If the pixel position, velocity and acceleration of the maneuvering target in the next frame are respectively matched successfully with the predicted position of the maneuvering target in the next frame, the attitude and velocity of the seeker corresponding to the maneuvering target are obtained;

[0045] Based on the pixel position, speed and acceleration of the maneuvering target after the next frame, the attitude and speed of the seeker corresponding to the maneuvering target in the first scene are controlled until the maneuvering target enters the second scene.

[0046] In a second aspect, the present application provides a device for a seeker to identify and track a maneuvering target, which is applied to a seeker and includes:

[0047] An acquisition module, used for acquiring a plurality of first initial images of a maneuvering target in a first scene;

[0048] An acquisition module, used for preprocessing the received first initial images to obtain the preprocessed first initial images;

[0049] A first determination module is used to perform a first target recognition detection on the mobile target based on the preprocessed first initial images and the trained first YOLO neural network model;

[0050] a second determination module, configured to determine a matching result between the movement state information of the maneuvering target and a predicted position of the maneuvering target in a next frame based on the movement state information of the maneuvering target and a tracker if the maneuvering target is detected for the first time in each of the first initial images;

[0051] a first control module, configured to control the seeker to track the maneuvering target in the first scene if the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame until the maneuvering target enters a second scene, wherein the second scene is any scene where the maneuvering target enters other than the first scene;

[0052] a detection module, configured to perform second target recognition detection on the maneuvering target based on each second initial image of the maneuvering target in the second scene and a trained second YOLO neural network model;

[0053] A second control module, configured to control the seeker to aim at the maneuvering target until the seeker contacts the maneuvering target if the maneuvering target is detected again in each of the second initial images;

[0054] A triggering module, configured to trigger the trained first YOLO neural network model or the trained second YOLO neural network model to re-perform target recognition detection on the maneuvering target if the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames;

[0055] Among them, N≧3, the first YOLO neural network model and the second YOLO neural network model are both improved YOLO neural network models.

[0056] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in any one of the contents of the first aspect is implemented.

[0057] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method as described in any one of the contents of the first aspect is implemented.

[0058] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes any one of the methods described in the first aspect above.

[0059] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0060] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0061] The method for identifying and tracking a maneuvering target by a seeker of the present application is applied to a seeker, and the first initial images of the received maneuvering target are preprocessed, and the first target identification and detection of the maneuvering target is performed based on the preprocessed first initial images and the improved YOLO neural network model; if the maneuvering target is detected for the first time, the matching result of the mobile state information of the maneuvering target and the predicted position of the maneuvering target in the next frame is determined; if the mobile state information of the maneuvering target matches the predicted position of the maneuvering target in the next frame successfully, the seeker is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene; the improved YOLO neural network model is used to perform second target identification and detection on the maneuvering target in the second scene; if the maneuvering target is detected again, the seeker is controlled to align with the maneuvering target until it contacts the maneuvering target; if the matching is unsuccessful, the improved YOLO neural network model is triggered to re-identify and detect the maneuvering target. Compared with the prior art, the improved YOLO neural network model is used to perform targeted identification and tracking of the maneuvering target in the first scene and the second scene respectively, and the maneuvering target is re-identified and tracked when the matching is unsuccessful, thereby reducing the complexity of the target identification and tracking method, improving the real-time performance of identification and tracking, and improving the identification and tracking accuracy of the maneuvering target. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0063] Figure 1 It is a flowchart of a method for a seeker to identify and track a maneuvering target provided by an embodiment of the present application;

[0064] Figure 2 This is a flow chart of preprocessing each received first initial image to obtain each preprocessed first initial image in step S200 provided in an embodiment of the present application;

[0065] Figure 3 It is a flowchart of performing first target recognition and detection on a mobile target based on each preprocessed first initial image and a trained first YOLO neural network model in step S300 provided in an embodiment of the present application;

[0066] Figure 4 It is a flow chart of outputting the predicted bounding boxes of the maneuvering targets in each first initial image after preprocessing by the head improvement prediction end of the trained first YOLO neural network model in step S340 provided by an embodiment of the present application;

[0067] Figure 5 It is a flowchart of determining a matching result between the movement state information of the maneuvering target and the predicted position of the maneuvering target in the next frame based on the movement state information of the maneuvering target and the tracker if the maneuvering target is detected for the first time in each first initial image in step S400 provided by an embodiment of the present application;

[0068] Figure 6 It is a flowchart of controlling the seeker to track the maneuvering target in the first scene until the maneuvering target enters the second scene if the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame in step S500 provided by an embodiment of the present application;

[0069] Figure 7 It is a structural schematic diagram of a mobile target identification and tracking device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0071] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0072] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0073] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0074] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0075] Maneuvering target tracking is a hot topic and difficulty in the research of seeker tracking. At present, with the increasing mobility of various targets, it is becoming more and more difficult for the seeker to track maneuvering targets in real time and effectively.

[0076] Due to the rapid development of military, aerospace and civil fields, higher requirements are placed on high-precision and high-robust mobile target tracking technology. Traditional target recognition and tracking methods have problems such as high computational complexity, poor real-time performance, and susceptibility to environmental interference. In addition, they cannot be recognized in time after the mobile target enters another scene, which also reduces the recognition and tracking accuracy of the seeker.

[0077] In order to overcome the above technical problems, the present application provides a method for a seeker to identify and track a maneuvering target, which is applied to a seeker, and includes: acquiring multiple frames of first initial images of the maneuvering target in a first scene; preprocessing each received first initial image to obtain each preprocessed first initial image; performing first target recognition and detection on the maneuvering target based on each preprocessed first initial image and a trained first YOLO neural network model; if the maneuvering target is detected for the first time in each first initial image, based on the mobile state information of the maneuvering target and a tracker, determining a matching result between the mobile state information of the maneuvering target and the predicted position of the maneuvering target in the next frame; if the mobile state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, controlling the seeker to track the maneuvering target in the first scene. target, until the maneuvering target enters the second scene, the second scene is any scene other than the first scene where the maneuvering target enters; based on each second initial image of the maneuvering target in the second scene and the trained second YOLO neural network model, the maneuvering target is recognized and detected as the second target; if the maneuvering target is detected again in each second initial image, the seeker is controlled to aim at the maneuvering target until it contacts the maneuvering target; if the mobile state information of the maneuvering target does not match the predicted position of the maneuvering target in consecutive N frames, the trained first YOLO neural network model or the trained second YOLO neural network model is triggered to re-recognize and detect the maneuvering target; wherein, N≧3, the first YOLO neural network model and the second YOLO neural network model are both improved YOLO neural network models. Compared with the prior art, the improved YOLO neural network model is used to perform targeted recognition and tracking of the maneuvering target in the first scene and the second scene respectively, and the maneuvering target is re-recognized and tracked when the matching fails, thereby reducing the complexity of the target recognition and tracking method, improving the real-time performance of recognition and tracking, and improving the recognition and tracking accuracy of the maneuvering target.

[0078] The technical solution of the present application is described below through specific embodiments.

[0079] First, as Figure 1 As shown, the present application provides a method for a seeker to identify and track a maneuvering target, which is applied to the seeker and includes:

[0080] S100, acquiring a plurality of frames of first initial images of a maneuvering target in a first scene.

[0081] In one embodiment, the scene in which the mobile target that the seeker of the present application needs to identify and track will change. When the scene changes, the background of the mobile target will change accordingly. Therefore, it is necessary to obtain multiple frames of initial images of the mobile target in multiple scenes, so that the seeker can quickly identify the mobile target moving in different scenes. For example, the mobile target is set as a vehicle, the first scene is set as an urban scene, and the second scene is set as a wild scene. The mobile target enters the wild scene from the urban scene, or enters the urban scene from the wild scene. Due to the exterior paint of the vehicle itself, it has different significant features in the two scenes, thereby increasing the difficulty of identification and tracking by the seeker. This embodiment does not impose specific restrictions on the scene and mobile target, and is set according to the requirements of the seeker identification and tracking. For example, the mobile target can also be a person and a ship, and the scene can also be a water scene.

[0082] S200, preprocessing each received first initial image to obtain each preprocessed first initial image.

[0083] In this embodiment, each received first initial image is preprocessed to obtain each preprocessed first initial image. Since the first initial image is preprocessed into an image that meets the requirements, the complexity of subsequent image processing is reduced and the recognition efficiency is improved.

[0084] In one embodiment, Figure 2 As shown, step S200, preprocessing each received first initial image to obtain each preprocessed first initial image, includes:

[0085] S210: Perform noise reduction processing based on the noise type of each first initial image to obtain a corresponding first noise-reduced initial image.

[0086] In this embodiment, noise reduction processing is performed based on the noise type of each first initial image to obtain the corresponding first noise-reduced initial image, specifically: the noise type of each first initial image is determined; based on the noise type of each first initial image, noise reduction processing corresponding to the noise type is performed to obtain the corresponding first noise-reduced initial image. For example, the noise types of the first initial image include additive noise, multiplicative noise and salt and pepper noise (also known as impulse noise), and additive noise includes Gaussian noise, uniform noise and Poisson noise; for the noise type of the first initial image, a noise reduction processing method corresponding to the noise type is selected, such as if the first initial image is Gaussian noise, a Gaussian filter is used for denoising, and if the first initial image is impulse noise, a median filter is used for denoising, thereby achieving a better denoising effect, improving the noise reduction efficiency of the first initial image, and also improving the detection accuracy of high-speed mobile targets in the first initial image.

[0087] S220, performing histogram processing on each first denoised initial image to obtain a corresponding first histogram initial image.

[0088] In this embodiment, each first denoised initial image is subjected to histogram processing to obtain a corresponding first rectangular initial image. Specifically, each first denoised initial image is subjected to histogram equalization, that is, the grayscale histogram of each first denoised initial image is changed from a relatively concentrated grayscale interval to a distribution within a larger grayscale range. By adjusting the pixel value distribution of each first denoised initial image, the brightness distribution of each first denoised initial image is made more uniform, the contrast of each first denoised initial image is improved, the detail visibility of each first denoised initial image is enhanced, and the detection accuracy of the maneuvering target in the first initial image is improved.

[0089] S230: randomly splice a preset number of first rectangular initial images into a large image to obtain a plurality of first spliced ​​initial images.

[0090] In this embodiment, a preset number of first rectangular initial images are randomly spliced ​​into a large image to obtain multiple first spliced ​​initial images, so that the maneuvering target can be detected in a larger range, and the probability of detecting the maneuvering target in the first spliced ​​initial image is improved. Among them, the preset number is 2, 4, or 6, which is conducive to forming first spliced ​​initial images of various sizes, thereby improving the detection efficiency of the maneuvering target.

[0091] In another embodiment, if the pixel size of the first rectangular initial image is greater than or equal to the preset pixel size, the first rectangular initial image is determined to be the first spliced ​​initial image, thereby improving the processing efficiency of the initial image. The preset pixel size is greater than or equal to 598*598, thereby reducing the processing steps of splicing.

[0092] S240, dividing each first stitched initial image into a plurality of grids of preset pixel areas to obtain each preprocessed first initial image.

[0093] In this embodiment, each first spliced ​​initial image is divided into a plurality of grids of preset pixel areas to obtain each preprocessed first initial image, thereby reducing the step of dividing the region of interest for detecting the maneuvering target, and directly treating the grids divided in the first spliced ​​initial image as the region of interest, thereby improving the processing efficiency of the initial image. The preset pixel area is less than or equal to 13*13, thereby improving the image processing speed of a single grid.

[0094] S300: Perform first target recognition and detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model.

[0095] In this embodiment, since each first initial image after preprocessing is an image of the maneuvering target in the first scene, the first YOLO neural network model corresponding to the first scene is used to perform first target recognition and detection on the maneuvering target. Specifically, the first YOLO neural network model simultaneously predicts the maneuvering target for each grid in each first initial image after preprocessing, and the first YOLO neural network model is an improved YOLO neural network model, that is, the position loss function is improved on the basis of the YOLO neural network model, thereby improving the detection speed of the maneuvering target in the first initial image, and at the same time, also improving the detection accuracy of the maneuvering target.

[0096] In one embodiment, the trained first YOLO neural network model includes a backbone network, a neck network and a head improvement prediction end; the backbone network and the neck network both adopt the structure of the YOLOv5 version, which is a prior art and will not be elaborated on in detail; since the position loss function of the head prediction end of the prior art changes the background after the maneuvering target enters the second scene from the first scene, the distance error between the predicted bounding box output by the head prediction end and the real box is large, which reduces the detection accuracy of the maneuvering target, thereby reducing the tracking accuracy. The head improvement prediction end of this embodiment reduces the distance error between the predicted bounding box and the real box, improves the detection accuracy of the maneuvering target, thereby improving the tracking accuracy.

[0097] In one embodiment, Figure 3 As shown, step S300, based on the preprocessed first initial images and the trained first YOLO neural network model, performs first target recognition detection on the mobile target, including:

[0098] S310, obtaining a trained first YOLO neural network model based on a training set and a test set of the maneuvering target in the first scene.

[0099] S320, the backbone network of the trained first YOLO neural network model extracts features corresponding to the maneuvering target from each preprocessed first initial image.

[0100] S330, the neck network of the trained first YOLO neural network model aggregates and refines features corresponding to the maneuvering target.

[0101] S340, the head improvement prediction end of the trained first YOLO neural network model outputs the predicted bounding box of the maneuvering target in each preprocessed first initial image to perform first target recognition detection on the maneuvering target.

[0102] In this embodiment, since the trained first YOLO neural network model uses the head improvement prediction end to output the predicted bounding box of the maneuvering target in each first initial image after preprocessing, the accuracy of the first target recognition and detection of the maneuvering target is improved, and the processing speed of the first initial image is also improved.

[0103] In one embodiment, the predicted bounding box includes the existence recognition probability, target category probability and bounding box position information of the maneuvering target, the existence recognition probability and the target category probability are 0 to 1, and the bounding box position information includes the center coordinate x value, the center coordinate y value, the pixel width, the pixel height and the confidence of the bounding box; the head improvement prediction end includes a confidence loss function, a category loss function and an improved position loss function; the confidence loss function represents whether the maneuvering target is correctly predicted and penalizes the confidence of the background box, the category loss function represents the prediction accuracy of the maneuvering target category, and the improved position loss function represents the position difference between the predicted bounding box of the maneuvering target and the real box based on the square difference of the merged box diagonal; this embodiment also sets a confidence threshold, and filters invalid predicted bounding boxes by judging whether the confidence of the bounding box position information is greater than or equal to the confidence threshold, and the confidence of the invalid predicted bounding box is less than the confidence threshold, thereby improving the processing speed of the initial image, setting the confidence threshold according to the difference between the first scene and the second scene and the accuracy of the maneuvering target in different scene training, and improving the prediction accuracy of the maneuvering target. For example, when the first scene is an urban scene, the maneuvering target is a vehicle, and the confidence threshold is set to 0.9; when the second scene is a wild scene, the maneuvering target is still a vehicle, and the confidence threshold is set to 0.8.

[0104] In one embodiment, Figure 4 As shown, in step S340, the head improvement prediction end of the trained first YOLO neural network model outputs the predicted bounding box of the maneuvering target in each first initial image after preprocessing, including:

[0105] S341, the head improvement prediction end outputs the existence recognition probability of the maneuvering target in each preprocessed first initial image through a confidence loss function.

[0106] S342, the head improvement prediction end outputs the target category probability of the maneuvering target in each preprocessed first initial image through the category loss function.

[0107] S343, the head improvement prediction end outputs the bounding box position information of the maneuvering target in each preprocessed first initial image by improving the position loss function.

[0108] In this embodiment, since the boundary position information output by the improved position loss function has been confidence screened and the position difference between the real box and the predicted box of the maneuvering target has been reduced, the existence recognition probability and target category probability of the maneuvering target output by the head improved prediction end will increase with the output of the boundary box position information of the improved position loss function, thereby reducing the processing speed of the initial image while also improving the prediction accuracy of the maneuvering high target.

[0109] In one embodiment, the improved position loss function outputs the bounding box position information of the maneuvering target in each first initial image after preprocessing, including: determining the improved position loss function value based on the intersection ratio, the central Euclidean distance, the minimum diagonal length, the maximum diagonal length, the balance parameter and the difference parameter; outputting the bounding box position information of the maneuvering target based on the improved position loss function value and the moving state information of the maneuvering target in each first initial image after preprocessing; wherein the central Euclidean distance is the Euclidean distance between the center points of the predicted box and the real box, the minimum diagonal length is the minimum diagonal length in the minimum closed box of the predicted box and the real box, the maximum diagonal length is the maximum diagonal length in the minimum closed box of the predicted box and the real box, and the difference parameter is the difference in aspect ratio between the predicted box and the real box.

[0110] In this embodiment, since the position loss function of the prior art outputs the bounding box position information through the length of any diagonal line in the minimum closed enclosing frame of the predicted box and the real box, due to the change of the scene, there is a large difference in the ratio of pixel width to pixel height between the predicted box and the real box, the value of the position loss function of the prior art will decrease, thereby reducing the prediction accuracy of the maneuvering target. This embodiment uses the square difference between the maximum diagonal length and the minimum diagonal length in the minimum closed enclosing frame of the predicted box and the real box, reduces the position difference between the predicted box and the real box, thereby increasing the overlap between the predicted box and the real box, and thereby improving the prediction accuracy of the maneuvering target.

[0111] In one embodiment, the position loss function is calculated as:

[0112]

[0113] Among them, L iou To improve the position loss function value;

[0114] IoU is the intersection over union ratio;

[0115] ρ is the prediction box c gt The central Euclidean distance between the center points of the real box c;

[0116] a is the maximum diagonal length of the minimum closed box of the predicted box and the true box;

[0117] b is the minimum diagonal length of the minimum closed box of the predicted box and the real box;

[0118] α is the equilibrium parameter;

[0119] v is the difference parameter between the aspect ratio of the predicted box and the real box.

[0120] The calculation formula of the difference parameter is:

[0121]

[0122] Where w p is the pixel width of the prediction box; h p is the pixel height of the predicted box; w is the pixel width of the real box; h is the pixel height of the real box.

[0123] S400: If a maneuvering target is detected for the first time in each first initial image, a matching result between the maneuvering target's movement state information and a predicted position of the maneuvering target in a next frame is determined based on the movement state information of the maneuvering target and a tracker.

[0124] In one embodiment, if a maneuvering target is detected for the first time in each first initial image, a Kalman filter is used to predict the movement state information of the maneuvering target in the tracker, and a matching result between the movement state information of the maneuvering target and the predicted position of the maneuvering target in the next frame is determined, which is used to continuously track the maneuvering target and improve the accuracy of tracking the maneuvering target.

[0125] In one embodiment, the movement state information includes pixel position, speed and acceleration, wherein the pixel position represents the pixel coordinates of the centroid of the maneuvering target in the image frame; the speed and acceleration are used to determine the predicted position of the maneuvering target in the next frame.

[0126] In one embodiment, Figure 5 As shown, in step S400, if a maneuvering target is detected for the first time in each first initial image, based on the movement state information of the maneuvering target and the tracker, a matching result between the movement state information of the maneuvering target and the predicted position of the maneuvering target in the next frame is determined, including:

[0127] S410: If a maneuvering target is detected for the first time in each first initial image, the pixel position, velocity and acceleration of the maneuvering target in the current frame and the pixel position, velocity and acceleration of the maneuvering target in the next frame are respectively obtained.

[0128] S420, based on the pixel position, velocity and acceleration of the maneuvering target in the current frame, the tracker obtains the predicted position of the maneuvering target in the next frame.

[0129] S430, matching the pixel position, speed and acceleration of the maneuvering target in the next frame with the predicted position of the maneuvering target in the next frame respectively, and determining the matching result.

[0130] In one embodiment, the Hungarian algorithm is used to match the pixel position, speed and acceleration of the maneuvering target in the next frame with the predicted position of the maneuvering target in the next frame, and the matching result is determined by determining the overlap degree between the pixel position of the maneuvering target in the next frame and the predicted position and the similarity of the appearance features. The matching result is then normalized and a matching threshold is set. When the matching result is greater than or equal to the matching threshold, the matching result is determined to be successful; wherein, the weight of the overlap degree between the pixel position of the maneuvering target in the next frame and the predicted position and the similarity of the appearance features are set according to the training recognition accuracy of the maneuvering target in the first scene and the second scene respectively. For example, if the matching threshold is greater than or equal to 0.9, the weight of the overlap degree in the matching result is 0.55, and the weight of the appearance feature similarity in the matching result is 0.45.

[0131] S500, if the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, the seeker is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene, and the second scene is any scene where the maneuvering target enters other than the first scene.

[0132] In this embodiment, if the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, it means that the maneuvering target is still the maneuvering target of the previous frame, and the tracking state of the maneuvering target is maintained. The seeker is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene. The second scene is any scene where the maneuvering target enters other than the first scene, thereby improving the tracking efficiency and accuracy of the maneuvering target.

[0133] In one embodiment, Figure 6 As shown, step S500, if the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, controlling the seeker to track the maneuvering target in the first scene until the maneuvering target enters the second scene, includes:

[0134] S510: If the pixel position, velocity and acceleration of the maneuvering target in the next frame are successfully matched with the predicted position of the maneuvering target in the next frame, the pixel position, velocity and acceleration of the maneuvering target after the next frame are continuously obtained.

[0135] S520: If the pixel position, velocity and acceleration of the maneuvering target in the next frame are successfully matched with the predicted position of the maneuvering target in the next frame, the attitude and velocity of the seeker corresponding to the maneuvering target are obtained.

[0136] S530, based on the pixel position, velocity and acceleration of the maneuvering target after the next frame, controlling the attitude and velocity of the seeker corresponding to the maneuvering target in the first scene until the maneuvering target enters the second scene.

[0137] In this embodiment, after the matching is successful, the precision and accuracy of continuously tracking the maneuvering target are improved by controlling the attitude and speed of the seeker corresponding to the pixel position, speed and acceleration of the maneuvering target. The attitude of the seeker includes azimuth, pitch and heading, and the speed includes travel speed and tracking angular speed.

[0138] It should be noted that when the seeker is tracking a maneuvering target, it will also evaluate the tracking confidence and tracking stability based on the pixel position, velocity and acceleration of the maneuvering target and the corresponding attitude and velocity of the seeker, thereby improving the precision and accuracy of the seeker's continuous tracking of the maneuvering target.

[0139] S600: Perform second target recognition and detection on the maneuvering target based on each second initial image of the maneuvering target in the second scene and the trained second YOLO neural network model.

[0140] In this embodiment, after the maneuvering target enters the second scene, each second initial image of the maneuvering target is obtained, and the second target recognition and detection of the maneuvering target is performed using the trained second YOLO neural network model. Since the second YOLO neural network model uses a training set corresponding to the second scene, the recognition accuracy of the maneuvering target is improved after the maneuvering target changes the scene.

[0141] S700: If the maneuvering target is detected again in each second initial image, control the seeker to aim at the maneuvering target until it contacts the maneuvering target.

[0142] In this embodiment, if a maneuvering target is detected again in each second initial image, the seeker is controlled to aim at the maneuvering target until it contacts the maneuvering target, thereby achieving the purpose of close-range tracking or attacking the maneuvering target.

[0143] S800, if the mobile state information of the maneuvering target does not match the predicted position of the maneuvering target in N consecutive frames, trigger the trained first YOLO neural network model or the trained second YOLO neural network model to re-perform target recognition detection on the maneuvering target; wherein N≧3.

[0144] In this embodiment, if the mobile state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames, it means that the maneuvering target is blocked in the first scene or the second scene or temporarily disappears from the field of view of the seeker. At this time, the trained first YOLO neural network model or the trained second YOLO neural network model is triggered to re-perform target recognition and detection on the maneuvering target to retrieve the lost maneuvering target, so that the maneuvering target can still be tracked when the maneuvering target is not successfully matched in N consecutive frames, thereby improving the stability of the seeker's continuous tracking of the maneuvering target and the accuracy of recognition and detection.

[0145] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0146] Second, as Figure 7 As shown, the present application provides a device 100 for a seeker to identify and track a maneuvering target, which is applied to a seeker and includes:

[0147] The acquisition module 110 is used to acquire a plurality of first initial images of the maneuvering target in a first scene.

[0148] The acquisition module 120 is used to preprocess the received first initial images to obtain the preprocessed first initial images.

[0149] The first determination module 130 is used to perform first target recognition and detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model.

[0150] The second determination module 140 is used to determine the matching result between the movement state information of the maneuvering target and the predicted position of the maneuvering target in the next frame based on the movement state information of the maneuvering target and the tracker if the maneuvering target is detected for the first time in each first initial image.

[0151] The first control module 150 is used to control the seeker to track the maneuvering target in the first scene if the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame until the maneuvering target enters the second scene, and the second scene is any scene where the maneuvering target enters other than the first scene.

[0152] The detection module 160 is used to perform second target recognition detection on the maneuvering target based on each second initial image of the maneuvering target in the second scene and the trained second YOLO neural network model.

[0153] The second control module 170 is configured to control the seeker to aim at the maneuvering target until the seeker contacts the maneuvering target if the maneuvering target is detected again in each second initial image.

[0154] The trigger module 180 is used to trigger the trained first YOLO neural network model or the trained second YOLO neural network model to re-perform target recognition detection on the maneuvering target if the mobile state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames; wherein N≧3, the first YOLO neural network model and the second YOLO neural network model are both improved YOLO neural network models.

[0155] It can be understood that in the process of dynamically tracking a maneuvering target, the result of identifying and detecting the maneuvering target or the position information of the maneuvering target is tracked, and the encrypted data is then transmitted to the control module of the identification and tracking device and the control center of the seeker.

[0156] It should be noted that the information interaction, execution process and other contents between the above-mentioned devices are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0157] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0158] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in any one of the contents of the first aspect is implemented.

[0159] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method as described in any one of the contents of the first aspect is implemented.

[0160] In a fifth aspect, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes any one of the methods described in the first aspect.

[0161] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0162] The method for identifying and detecting a mobile target provided in an embodiment of the present application can be applied to various terminal devices, and the embodiment of the present application does not impose any restrictions on the specific type of the terminal device.

[0163] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiments when executed by a processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc.

[0164] The computer-readable medium may include at least: any entity or device capable of carrying the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0165] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0166] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0167] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0168] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0169] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for a seeker to identify and track a maneuvering target, characterized in that: Applied to seeker, including: Acquire a plurality of first initial images of the maneuvering target in a first scene; Preprocessing the received first initial images to obtain preprocessed first initial images; Performing first target recognition detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model; If the maneuvering target is detected for the first time in each of the first initial images, determining a matching result between the movement state information of the maneuvering target and a predicted position of the maneuvering target in a next frame based on the movement state information of the maneuvering target and a tracker; If the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, control the seeker to track the maneuvering target in the first scene until the maneuvering target enters a second scene, where the second scene is any scene where the maneuvering target enters other than the first scene; Performing second target recognition and detection on the maneuvering target based on each second initial image of the maneuvering target in the second scene and the trained second YOLO neural network model; If the maneuvering target is detected again in each of the second initial images, controlling the seeker to aim at the maneuvering target until it contacts the maneuvering target; If the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames, triggering the trained first YOLO neural network model or the trained second YOLO neural network model to re-perform target recognition detection on the maneuvering target; Among them, N≧3, the first YOLO neural network model and the second YOLO neural network model are both improved YOLO neural network models.

2. The method according to claim 1, characterized in that Preprocessing each of the received first initial images to obtain each of the preprocessed first initial images includes: Performing noise reduction processing based on the noise type of each of the first initial images to obtain a corresponding first noise-reduced initial image; Performing histogram processing on each of the first denoised initial images to obtain a corresponding first histogram initial image; Randomly splicing a preset number of the first histogram initial images into a large image to obtain a plurality of first spliced ​​initial images; Each of the first spliced ​​initial images is divided into a plurality of grids of preset pixel areas to obtain each of the preprocessed first initial images.

3. The method according to claim 1, characterized in that The trained first YOLO neural network model includes the backbone network, the neck network, and the head improved prediction end; Performing first target recognition and detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model includes: Based on the training set and the test set of the maneuvering target in the first scene, obtaining the trained first YOLO neural network model; The backbone network of the trained first YOLO neural network model extracts features corresponding to the maneuvering target from each of the preprocessed first initial images; The neck network of the trained first YOLO neural network model aggregates and refines features corresponding to the maneuvering target; The head improvement prediction end of the trained first YOLO neural network model outputs the predicted bounding box of the maneuvering target in each of the preprocessed first initial images to perform first target recognition detection on the maneuvering target.

4. The method according to claim 3, characterized in that The predicted bounding box includes the existence recognition probability, target category probability and bounding box position information of the maneuvering target, and the head improvement prediction end includes a confidence loss function, a category loss function and an improved position loss function; The head improvement prediction end of the trained first YOLO neural network model outputs the predicted bounding box of the maneuvering target in each of the preprocessed first initial images, including: The head improvement prediction end outputs the existence recognition probability of the maneuvering target in each of the preprocessed first initial images through a confidence loss function; The head improvement prediction end outputs the target category probability of the maneuvering target in each of the preprocessed first initial images through a category loss function; The head improvement prediction end outputs the bounding box position information of the maneuvering target in each of the preprocessed first initial images by improving the position loss function.

5. The method according to claim 4, characterized in that The improved position loss function outputs the bounding box position information of the maneuvering target in each of the preprocessed first initial images, including: Determine the improved position loss function value based on the intersection-over-union ratio, the center Euclidean distance, the minimum diagonal length, the maximum diagonal length, the balance parameter and the difference parameter; Outputting the bounding box position information of the maneuvering target based on the improved position loss function value and the movement state information of the maneuvering target in each of the preprocessed first initial images; Among them, the central Euclidean distance is the Euclidean distance between the center points of the predicted box and the real box, the minimum diagonal length is the minimum diagonal length in the minimum closed box of the predicted box and the real box, the maximum diagonal length is the maximum diagonal length in the minimum closed box of the predicted box and the real box, and the difference parameter is the difference in the aspect ratio between the predicted box and the real box.

6. The method according to claim 1, characterized in that The movement state information includes pixel position, speed and acceleration, wherein the pixel position represents the pixel coordinates of the centroid of the maneuvering target in the image frame; If the maneuvering target is detected for the first time in each of the first initial images, determining a matching result between the movement state information of the maneuvering target and a predicted position of the maneuvering target in a next frame based on the movement state information of the maneuvering target and a tracker, including: If the maneuvering target is detected for the first time in each of the first initial images, the pixel position, velocity and acceleration of the maneuvering target in the current frame and the pixel position, velocity and acceleration in the next frame are respectively obtained; The tracker obtains a predicted position of the maneuvering target in a next frame based on the pixel position, velocity and acceleration of the maneuvering target in a current frame; The pixel position, speed and acceleration of the maneuvering target in the next frame are matched with the predicted position of the maneuvering target in the next frame, and a matching result is determined.

7. The method according to claim 1, characterized in that The movement state information includes pixel position, speed and acceleration; If the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, controlling the seeker to track the maneuvering target in the first scene until the maneuvering target enters the second scene, comprising: If the pixel position, velocity and acceleration of the maneuvering target in the next frame are respectively matched successfully with the predicted position of the maneuvering target in the next frame, the pixel position, velocity and acceleration of the maneuvering target after the next frame are continuously acquired; If the pixel position, velocity and acceleration of the maneuvering target in the next frame are respectively matched successfully with the predicted position of the maneuvering target in the next frame, the attitude and velocity of the seeker corresponding to the maneuvering target are obtained; Based on the pixel position, speed and acceleration of the maneuvering target after the next frame, the attitude and speed of the seeker corresponding to the maneuvering target in the first scene are controlled until the maneuvering target enters the second scene.

8. A device for a seeker to identify and track a maneuvering target, characterized in that: Applied to seeker, including: An acquisition module, used for acquiring a plurality of first initial images of a maneuvering target in a first scene; An acquisition module, used for preprocessing the received first initial images to obtain the preprocessed first initial images; A first determination module is used to perform a first target recognition detection on the mobile target based on the preprocessed first initial images and the trained first YOLO neural network model; a second determination module, configured to determine a matching result between the movement state information of the maneuvering target and a predicted position of the maneuvering target in a next frame based on the movement state information of the maneuvering target and a tracker if the maneuvering target is detected for the first time in each of the first initial images; a first control module, configured to control the seeker to track the maneuvering target in the first scene if the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame until the maneuvering target enters a second scene, wherein the second scene is any scene where the maneuvering target enters other than the first scene; a detection module, configured to perform second target recognition detection on the maneuvering target based on each second initial image of the maneuvering target in the second scene and a trained second YOLO neural network model; A second control module, configured to control the seeker to aim at the maneuvering target until the seeker contacts the maneuvering target if the maneuvering target is detected again in each of the second initial images; A triggering module, configured to trigger the trained first YOLO neural network model or the trained second YOLO neural network model to re-perform target recognition detection on the maneuvering target if the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames; Among them, N≧3, the first YOLO neural network model and the second YOLO neural network model are both improved YOLO neural network models.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images