Methods, devices, electronic equipment, and media for seeking and tracking maneuvering targets

Abstract

This application provides a method, apparatus, electronic device, and medium for a seeker to identify and track maneuvering targets. The method includes: preprocessing each initial first image of the maneuvering target; performing first target identification and detection based on the preprocessed initial first images and an improved YOLO neural network model; if a maneuvering target is detected for the first time, determining the matching result between the maneuvering target's movement state information and the predicted position in the next frame; if the match is successful, controlling the seeker to track the maneuvering target in a first scene until entering a second scene; using the improved YOLO neural network model to perform second target identification and detection in the second scene; if a maneuvering target is detected again, controlling the seeker to align with the maneuvering target until contact is made; if the match is unsuccessful, triggering the improved YOLO neural network model to re-perform target identification and detection, thereby reducing the complexity of target identification and tracking and improving the accuracy of identification and tracking.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a method, apparatus, electronic device and storage medium for a seeker head to identify and track moving targets. Background Technology

[0002] Currently, with the increasing mobility of various targets, it is becoming increasingly difficult for seekers to track maneuvering targets in real time and effectively.

[0003] The rapid development of military, aerospace, and civilian fields has placed higher demands on high-precision and robust maneuvering target tracking technology. Traditional target recognition and tracking methods suffer from high computational complexity, poor real-time performance, and susceptibility to environmental interference. Furthermore, they cannot be identified in a timely manner after a maneuvering target enters another scene, which also reduces the accuracy of the seeker's recognition and tracking.

[0004] Existing methods for seeking and tracking maneuvering targets suffer from high complexity, poor real-time performance, and low accuracy. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and medium for a seeker head to identify and track maneuvering targets, which can solve the problems of high complexity, poor real-time performance, and low accuracy in the identification and tracking of maneuvering targets.

[0006] In a first aspect, embodiments of this application provide a method for a seeker to identify and track a maneuvering target, applied to a seeker, comprising:

[0007] Acquire multiple frames of the first initial image of the maneuvering target in the first scene;

[0008] The received first initial images are preprocessed to obtain preprocessed first initial images;

[0009] Based on the preprocessed first initial images and the trained first YOLO neural network model, the maneuvering target is identified and detected as a first target.

[0010] If the moving target is detected for the first time in each of the first initial images, based on the movement state information of the moving target and the tracker, the matching result between the movement state information of the moving target and the predicted position of the moving target in the next frame is determined;

[0011] If the movement state information of the maneuvering target matches the predicted position of the maneuvering target in the next frame, the seeker head is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene, which is any scene other than the first scene.

[0012] Based on the second initial images of the maneuvering target in the second scenario and the trained second YOLO neural network model, the maneuvering target is identified and detected as a second target.

[0013] If the maneuvering target is detected again in each of the second initial images, the seeker head is controlled to align with the maneuvering target until it makes contact with the maneuvering target;

[0014] If the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames, the trained first YOLO neural network model or the trained second YOLO neural network model is triggered to re-identify and detect the maneuvering target.

[0015] Where N≧3, both the first YOLO neural network model and the second YOLO neural network model are improved YOLO neural network models.

[0016] In one embodiment, preprocessing is performed on each of the received first initial images to obtain preprocessed first initial images, including:

[0017] Denoising is performed on each of the first initial images based on its noise type to obtain the corresponding first denoised initial image;

[0018] Each of the first noise-reduced initial images is subjected to histogram processing to obtain the corresponding first histogram initial image.

[0019] A preset number of the first histogram initial images are randomly stitched together to form a large image, resulting in multiple first stitched initial images.

[0020] Each of the first initial images is divided into multiple grids with a preset pixel area to obtain the preprocessed first initial images.

[0021] In one embodiment, the trained first YOLO neural network model includes a backbone network, a neck network, and a head-improving prediction end;

[0022] Based on the preprocessed first initial images and the trained first YOLO neural network model, the first target recognition and detection of the maneuvering target is performed, including:

[0023] Based on the training and testing sets of the maneuvering target in the first scenario, the trained first YOLO neural network model is obtained;

[0024] The backbone network of the trained first YOLO neural network model extracts the features corresponding to the maneuvering target from each of the preprocessed first initial images;

[0025] The neck network of the trained first YOLO neural network model aggregates and refines the features corresponding to the maneuvering target;

[0026] The head-improved prediction end of the trained first YOLO neural network model outputs the predicted bounding boxes of the maneuvering targets in each of the preprocessed first initial images, so as to perform first target recognition and detection on the maneuvering targets.

[0027] In one embodiment, the predicted bounding box includes the presence and identification probability of the maneuvering target, the target category probability, and the bounding box location information, and the improved head prediction end includes a confidence loss function, a category loss function, and an improved location loss function;

[0028] The head-level improved prediction terminal of the trained first YOLO neural network model outputs the predicted bounding boxes of the maneuvering targets in each of the preprocessed first initial images, including:

[0029] The improved head prediction terminal outputs the probability of the presence and recognition of the maneuvering target in each of the preprocessed first initial images through a confidence loss function.

[0030] The improved head prediction terminal outputs the target category probability of the maneuvering target in each of the preprocessed first initial images through a category loss function;

[0031] The improved head prediction terminal outputs the bounding box position information of the maneuvering target in each of the preprocessed first initial images through an improved position loss function.

[0032] In one embodiment, the improved position loss function outputs the bounding box position information of the maneuvering target in each of the preprocessed first initial images, including:

[0033] Based on the intersection-union ratio, center Euclidean distance, minimum diagonal length, maximum diagonal length, balance parameter, and difference parameter, the improved location loss function value is determined;

[0034] Based on the improved position loss function value and the movement state information of the maneuvering target in each of the preprocessed first initial images, the bounding box position information of the maneuvering target is output.

[0035] Wherein, the center Euclidean distance is the Euclidean distance between the center points of the predicted box and the ground truth box, the minimum diagonal length is the minimum diagonal length within the minimum closed envelope of the predicted box and the ground truth box, the maximum diagonal length is the maximum diagonal length within the minimum closed envelope of the predicted box and the ground truth box, and the difference parameter is the difference in aspect ratio between the predicted box and the ground truth box.

[0036] In one embodiment, the motion state information includes pixel position, velocity, and acceleration, wherein the pixel position represents the pixel coordinates of the centroid of the maneuvering target in the image frame;

[0037] If the moving target is detected for the first time in each of the first initial images, based on the movement state information of the moving target and the tracker, the matching result between the movement state information of the moving target and the predicted position of the moving target in the next frame is determined, including:

[0038] If the maneuvering target is detected for the first time in each of the first initial images, the pixel position, velocity and acceleration of the maneuvering target in the current frame and the pixel position, velocity and acceleration in the next frame are obtained respectively.

[0039] Based on the pixel position, velocity, and acceleration of the maneuvering target in the current frame, the tracker obtains the predicted position of the maneuvering target in the next frame;

[0040] The pixel position, velocity, and acceleration of the maneuvering target in the next frame are matched with the predicted position of the maneuvering target in the next frame, and the matching result is determined.

[0041] In one embodiment, the motion state information includes pixel position, velocity, and acceleration;

[0042] If the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, the seeker is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene, including:

[0043] If the pixel position, velocity, and acceleration of the maneuvering target in the next frame successfully match the predicted position of the maneuvering target in the next frame, the pixel position, velocity, and acceleration of the maneuvering target after the next frame are continuously acquired.

[0044] If the pixel position, velocity, and acceleration of the maneuvering target in the next frame successfully match the predicted position of the maneuvering target in the next frame, the attitude and velocity of the seeker corresponding to the maneuvering target are obtained.

[0045] Based on the pixel position, velocity, and acceleration of the maneuvering target in the next frame, the attitude and velocity of the seeker head corresponding to the maneuvering target in the first scene are controlled until the maneuvering target enters the second scene.

[0046] Secondly, this application provides a device for a seeker head to identify and track maneuvering targets, applied to a seeker head, comprising:

[0047] The acquisition module is used to acquire multiple frames of the first initial image of the maneuvering target in the first scene;

[0048] The acquisition module is used to preprocess each of the received first initial images to obtain preprocessed first initial images;

[0049] The first determining module is used to perform first target recognition and detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model.

[0050] The second determining module is used to determine, if the moving target is detected for the first time in each of the first initial images, the matching result between the moving target's movement state information and the tracker's predicted position in the next frame;

[0051] The first control module is configured to control the seeker to track the maneuvering target in the first scene if the movement state information of the maneuvering target matches the predicted position of the maneuvering target in the next frame, until the maneuvering target enters the second scene, wherein the second scene is any scene other than the first scene into which the maneuvering target enters.

[0052] The detection module is used to perform second target recognition and detection on the maneuvering target based on each second initial image of the maneuvering target in the second scene and the trained second YOLO neural network model;

[0053] The second control module is used to control the seeker to align with the maneuvering target until it contacts the maneuvering target if the maneuvering target is detected again in each of the second initial images.

[0054] The triggering module is used to trigger the trained first YOLO neural network model or the trained second YOLO neural network model to re-identify and detect the maneuvering target if the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames.

[0055] Where N≧3, both the first YOLO neural network model and the second YOLO neural network model are improved YOLO neural network models.

[0056] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of the first aspects above.

[0057] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any one of the first aspects above.

[0058] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in any one of the first aspects above.

[0059] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect above, and will not be repeated here.

[0060] The beneficial effects of the embodiments in this application compared with the prior art are:

[0061] The method for identifying and tracking maneuvering targets using a seeker in this application is applied to a seeker. It involves preprocessing each received initial image of a maneuvering target, and then performing a first target identification and detection based on the preprocessed initial images and an improved YOLO neural network model. If a maneuvering target is detected for the first time, the method determines the matching result between the target's movement state information and its predicted position in the next frame. If the match between the target's movement state information and its predicted position in the next frame is successful, the seeker is controlled to track the target in a first scene until it enters a second scene. In the second scene, the improved YOLO neural network model is used to perform a second target identification and detection. If a maneuvering target is detected again, the seeker is controlled to align with the target until it makes contact. If the match fails, the improved YOLO neural network model is triggered to re-identify and detect the target. Compared with existing technologies, this method reduces the complexity of the target identification and tracking method, improves the real-time performance of identification and tracking, and increases the accuracy of target identification and tracking by using an improved YOLO neural network model to specifically identify and track the target in both the first and second scenes, and re-identifying and tracking the target when a match fails. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating a method for a seeker head to identify and track a maneuvering target according to an embodiment of this application;

[0064] Figure 2 This is a flowchart illustrating step S200 of an embodiment of the present application, which involves preprocessing each received first initial image to obtain preprocessed first initial images.

[0065] Figure 3 This is a schematic diagram of the process of step S300, provided in an embodiment of this application, for identifying and detecting a first target based on each preprocessed first initial image and a trained first YOLO neural network model;

[0066] Figure 4 This is a flowchart illustrating the process of the head improvement prediction end of the trained first YOLO neural network model outputting the preprocessed predicted bounding boxes of the maneuvering targets in each first initial image in step S340 of an embodiment of this application.

[0067] Figure 5 This is a schematic diagram of the process of step S400 provided in an embodiment of this application, which is to determine the matching result of the movement state information of the moving target and the predicted position of the moving target in the next frame based on the movement state information of the moving target and the tracker if a moving target is detected for the first time in each first initial image.

[0068] Figure 6 This is a schematic diagram of the process of step S500 provided in an embodiment of this application, in which if the movement state information of the maneuvering target is successfully matched with the predicted position of the maneuvering target in the next frame, the seeker head is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene.

[0069] Figure 7 This is a schematic diagram of the structure of the mobile target identification and tracking device provided in the embodiments of this application. Detailed Implementation

[0070] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0071] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0072] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0073] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0074] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0075] Tracking maneuvering targets is a hot topic and a difficult area of ​​research in the field of seeker tracking. Currently, with the increasing maneuverability of various targets, it is becoming increasingly difficult for seekers to track maneuvering targets in real time and effectively.

[0076] The rapid development of military, aerospace, and civilian fields has placed higher demands on high-precision and robust maneuvering target tracking technology. Traditional target recognition and tracking methods suffer from high computational complexity, poor real-time performance, and susceptibility to environmental interference. Furthermore, they cannot be identified in a timely manner after a maneuvering target enters another scene, which also reduces the accuracy of the seeker's recognition and tracking.

[0077] To overcome the aforementioned technical problems, this application provides a method for a seeker to identify and track maneuvering targets, applied to a seeker, comprising: acquiring multiple first initial images of a maneuvering target in a first scene; preprocessing each received first initial image to obtain preprocessed first initial images; performing first target recognition and detection on the maneuvering target based on the preprocessed first initial images and a trained first YOLO neural network model; if a maneuvering target is detected for the first time in each of the first initial images, determining the matching result between the maneuvering target's motion state information and the predicted position of the maneuvering target in the next frame based on the maneuvering target's motion state information and the tracker; if the maneuvering target's motion state information successfully matches the predicted position of the maneuvering target in the next frame, controlling the seeker to track the maneuvering target in the first scene. The system tracks and identifies the target until it enters a second scenario, which is any scenario other than the first scenario. Based on each second initial image of the target in the second scenario and a trained second YOLO neural network model, the system performs second target recognition and detection. If the target is detected again in any of the second initial images, the system controls the seeker to align with the target until contact is made. If the target's movement state information fails to match the predicted position of the target in N consecutive frames, either the trained first or second YOLO neural network model is triggered to re-identify and detect the target. Here, N ≥ 3, and both the first and second YOLO neural network models are improved YOLO neural network models. Compared to existing technologies, this method reduces the complexity of the target recognition and tracking method, improves the real-time performance of recognition and tracking, and increases the accuracy of target recognition and tracking by using improved YOLO neural network models to specifically identify and track the target in both the first and second scenarios, and re-identifying and tracking the target when a match fails.

[0078] The technical solution of this application will be described below through specific embodiments.

[0079] Firstly, such as Figure 1 As shown, this application provides a method for a seeker to identify and track maneuvering targets, applied to a seeker, including:

[0080] S100, acquire multiple frames of the first initial image of the maneuvering target in the first scene.

[0081] In one embodiment, the scene in which the seeker needs to identify and track a moving target may change. As the scene changes, the background of the moving target also changes. Therefore, it is necessary to acquire multiple initial images of the moving target in multiple scenes to facilitate the seeker's rapid identification of the moving target in different scenes. For example, the moving target may be a vehicle, the first scene may be an urban scene, and the second scene may be a wilderness scene. The moving target may move from an urban scene to a wilderness scene, or vice versa. Due to the vehicle's exterior paint, it has different salient features in the two scenes, thus increasing the difficulty of the seeker's identification and tracking. This embodiment does not impose specific limitations on the scene and the moving target; the settings are based on the seeker's identification and tracking needs. For example, the moving target can also be a person or a ship, and the scene can also be a water scene.

[0082] S200, preprocess each received first initial image to obtain each preprocessed first initial image.

[0083] In this embodiment, each of the received first initial images is preprocessed to obtain preprocessed first initial images. Since the first initial images are preprocessed into images that meet the requirements, the complexity of subsequent image processing is reduced and the recognition efficiency is improved.

[0084] In one embodiment, such as Figure 2 As shown, step S200 involves preprocessing each received first initial image to obtain preprocessed first initial images, including:

[0085] S210, noise reduction processing is performed based on the noise type of each first initial image to obtain the corresponding first denoised initial image.

[0086] In this embodiment, denoising processing is performed based on the noise type of each first initial image to obtain a corresponding first denoised initial image. Specifically, the noise type of each first initial image is determined; denoising processing corresponding to the noise type is performed based on the noise type of each first initial image to obtain a corresponding first denoised initial image. For example, the noise types of the first initial images include additive noise, multiplicative noise, and salt-and-pepper noise (also known as impulse noise). Additive noise includes Gaussian noise, uniform noise, and Poisson noise. For each noise type of the first initial image, a denoising processing method corresponding to the noise type is selected. For example, if the first initial image has Gaussian noise, a Gaussian filter is used for denoising; if the first initial image has impulse noise, a median filter is used for denoising. This achieves better denoising results, improves the denoising efficiency of the first initial image, and also improves the detection accuracy of mobile high targets in the first initial image.

[0087] S220, perform histogram processing on each of the first initial noise reduction images to obtain the corresponding first histogram initial image.

[0088] In this embodiment, histogram processing is performed on each first initial denoised image to obtain a corresponding first histogram initial image. Specifically, histogram equalization is performed on each first initial denoised image, which changes the gray-level histogram of each first initial denoised image from a relatively concentrated gray-level range to a distribution over a wider gray-level range. By adjusting the pixel value distribution of each first initial denoised image, the brightness distribution of each first initial denoised image becomes more uniform, improving the contrast of each first initial denoised image, enhancing the detail visibility of each first initial denoised image, and also improving the detection accuracy of moving targets in the first initial image.

[0089] S230, randomly stitch a preset number of first square initial images into a large image to obtain multiple first stitched initial images.

[0090] In this embodiment, a preset number of first square initial images are randomly stitched together to form a large image, resulting in multiple first stitched initial images. This allows for the detection of moving targets over a wider area, increasing the probability of detecting moving targets in the first stitched initial images. The preset number of images is 2, 4, or 6, which facilitates the formation of first stitched initial images of various sizes, thereby improving the detection efficiency of moving targets.

[0091] In another embodiment, if the pixel size of the first square initial image is greater than or equal to a preset pixel size, the first square initial image is determined as the first stitching initial image, thus improving the processing efficiency of the initial image. The preset pixel size is greater than or equal to 598*598, thereby reducing the number of stitching processing steps.

[0092] S240, each of the first initial images is divided into multiple grids with preset pixel areas to obtain the preprocessed first initial images.

[0093] In this embodiment, each first stitched initial image is divided into multiple grids with preset pixel areas to obtain preprocessed first initial images. This reduces the step of dividing the region of interest (ROI) for detecting moving targets, and directly treats the grids in the first stitched initial images as ROIs, thus improving the processing efficiency of the initial images. The preset pixel area is less than or equal to 13*13, which increases the image processing speed of a single grid.

[0094] S300 performs first target recognition and detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model.

[0095] In this embodiment, since each of the preprocessed first initial images is an image of a maneuvering target in the first scene, a first YOLO neural network model corresponding to the first scene is used to perform first target recognition and detection on the maneuvering target. Specifically, the first YOLO neural network model simultaneously predicts the maneuvering target in each grid of each of the preprocessed first initial images. The first YOLO neural network model is an improved YOLO neural network model, that is, the position loss function is improved on the basis of the YOLO neural network model, thereby improving the detection speed of maneuvering targets in the first initial images, and also improving the detection accuracy of maneuvering targets.

[0096] In one embodiment, the trained first YOLO neural network model includes a backbone network, a neck network, and a head-improving prediction endpoint. Both the backbone network and the neck network adopt the YOLOv5 architecture, which is prior art and will not be elaborated upon further. Because the position loss function of the head prediction endpoint in the prior art changes the background after the maneuvering target enters the second scene from the first scene, the distance error between the predicted bounding box output by the head prediction endpoint and the ground truth box is large, which reduces the detection accuracy of the maneuvering target and thus reduces the tracking accuracy. The head-improving prediction endpoint in this embodiment reduces the distance error between the predicted bounding box and the ground truth box, improves the detection accuracy of the maneuvering target, and thus improves the tracking accuracy.

[0097] In one embodiment, such as Figure 3 As shown, step S300 involves performing first target recognition and detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model, including:

[0098] S310 obtains a trained first YOLO neural network model based on the training and test sets of the maneuvering target in the first scenario.

[0099] S320, the backbone network of the trained first YOLO neural network model extracts the features corresponding to the maneuvering target from each of the preprocessed first initial images.

[0100] S330, the neck network of the trained first YOLO neural network model aggregates and refines the features corresponding to the maneuvering target.

[0101] S340, the head of the trained first YOLO neural network model outputs the predicted bounding boxes of the maneuvering targets in each of the first initial images after preprocessing, so as to perform first target recognition and detection on the maneuvering targets.

[0102] In this embodiment, since the trained first YOLO neural network model uses the preprocessed predicted bounding boxes of the maneuvering targets in each of the first initial images after head improvement prediction output, the accuracy of the first target recognition and detection of the maneuvering targets is improved, and the processing speed of the first initial images is also improved.

[0103] In one embodiment, the predicted bounding box includes the probability of the presence of the maneuvering target, the probability of the target category, and the bounding box location information. The probability of presence and the probability of the target category are both between 0 and 1. The bounding box location information includes the center x-coordinate, center y-coordinate, pixel width, pixel height, and confidence level of the bounding box. The improved prediction head includes a confidence loss function, a category loss function, and an improved location loss function. The confidence loss function represents whether the maneuvering target was correctly predicted and penalizes the confidence level of the background box. The category loss function represents the prediction accuracy of the maneuvering target category. The improved location loss function represents the positional difference between the predicted bounding box and the ground truth bounding box based on the squared difference of the diagonal of the merged box. This embodiment also sets a confidence threshold. Invalid predicted bounding boxes are filtered out by judging whether the confidence level of the bounding box location information is greater than or equal to the confidence threshold. The confidence level of invalid predicted bounding boxes is less than the confidence threshold, thereby improving the processing speed of the initial image. The confidence threshold is set according to the difference between the first scene and the second scene and the accuracy of training the maneuvering target in different scenes, which improves the prediction accuracy of the maneuvering target. For example, in the first scenario, which is an urban scene, the moving target is a vehicle, and the confidence threshold is set to 0.9. In the second scenario, which is a wilderness scene, the moving target is still a vehicle, and the confidence threshold is set to 0.8.

[0104] In one embodiment, such as Figure 4 As shown, in step S340, the head improvement prediction end of the trained first YOLO neural network model outputs the predicted bounding boxes of the maneuvering targets in each of the preprocessed first initial images, including:

[0105] S341, the head-improved prediction end outputs the probability of recognizing the presence of a moving target in each of the preprocessed first initial images through a confidence loss function.

[0106] S342, the head-improved prediction end outputs the target class probability of the maneuvering target in each of the preprocessed first initial images through the category loss function.

[0107] S343, the head-improved prediction end outputs the bounding box position information of the maneuvering target in each of the preprocessed first initial images through an improved position loss function.

[0108] In this embodiment, since the boundary position information output by the improved position loss function has been filtered by confidence and the positional difference between the ground truth box and the predicted box of the maneuvering target has been reduced, the probability of the existence of the maneuvering target and the probability of the target category output by the improved prediction end of the head will increase with the bounding box position information output by the improved position loss function. This reduces the processing speed of the initial image while improving the prediction accuracy of the maneuvering target.

[0109] In one embodiment, the improved position loss function outputs the bounding box position information of the maneuvering target in each of the preprocessed first initial images, including: determining the improved position loss function value based on the intersection-union ratio, center Euclidean distance, minimum diagonal length, maximum diagonal length, balance parameter, and difference parameter; and outputting the bounding box position information of the maneuvering target based on the improved position loss function value and the movement state information of the maneuvering target in each of the preprocessed first initial images; wherein, the center Euclidean distance is the Euclidean distance between the center points of the predicted box and the ground truth box, the minimum diagonal length is the minimum diagonal length within the minimum closed envelope of the predicted box and the ground truth box, the maximum diagonal length is the maximum diagonal length within the minimum closed envelope of the predicted box and the ground truth box, and the difference parameter is the difference in aspect ratio between the predicted box and the ground truth box.

[0110] In this embodiment, the existing position loss function outputs bounding box position information based on the length of any diagonal within the smallest closed envelope of the predicted and ground truth boxes. Due to changes in the scene, there may be a significant difference in the pixel width to pixel height ratio between the predicted and ground truth boxes, leading to a decrease in the position loss function value and consequently reducing the prediction accuracy of moving targets. In contrast, this embodiment uses the squared difference between the largest and smallest diagonal lengths within the smallest closed envelope of the predicted and ground truth boxes. This reduces the positional difference between the predicted and ground truth boxes, thereby increasing the overlap between them and improving the prediction accuracy of moving targets.

[0111] In one embodiment, the location loss function is calculated as follows:

[0112]

[0113] Among them, L iou To improve the value of the position loss function;

[0114] IoU stands for Intersection over Union (IoU).

[0115] ρ represents the prediction box c gt The Euclidean distance between the center points of the real bounding box c and the center point of the real bounding box c;

[0116] 'a' represents the maximum diagonal length of the smallest closed bounding box between the predicted box and the ground truth box.

[0117] b is the minimum diagonal length of the smallest closed bounding box between the predicted box and the ground truth box;

[0118] α is the equilibrium parameter;

[0119] v is the difference in aspect ratio between the predicted bounding box and the ground truth bounding box.

[0120] The formula for calculating the difference parameter is:

[0121]

[0122] In the formula w p h is the pixel width of the prediction box. p is the pixel height of the predicted bounding box; w is the pixel width of the ground truth bounding box; h is the pixel height of the ground truth bounding box.

[0123] S400, if a moving target is detected for the first time in each of the first initial images, based on the moving target's motion state information and the tracker, determine the matching result between the moving target's motion state information and the predicted position of the moving target in the next frame.

[0124] In one embodiment, if a moving target is detected for the first time in each first initial image, a Kalman filter is used to predict the movement state information of the moving target in the tracker, and the matching result between the movement state information of the moving target and the predicted position of the moving target in the next frame is determined for continuous tracking of the moving target and to improve the accuracy of tracking the moving target.

[0125] In one embodiment, the motion state information includes pixel position, velocity, and acceleration. The pixel position represents the pixel coordinates of the centroid of the maneuvering target in the image frame. The velocity and acceleration are used to determine the predicted position of the maneuvering target in the next frame.

[0126] In one embodiment, such as Figure 5 As shown, in step S400, if a moving target is detected for the first time in each of the first initial images, based on the moving target's motion state information and the tracker, the matching result between the moving target's motion state information and the predicted position of the moving target in the next frame is determined, including:

[0127] S410, if a moving target is detected for the first time in each of the first initial images, the pixel position, velocity and acceleration of the moving target in the current frame and the pixel position, velocity and acceleration in the next frame are obtained respectively.

[0128] S420: Based on the pixel position, velocity, and acceleration of the moving target in the current frame, the tracker obtains the predicted position of the moving target in the next frame.

[0129] S430 matches the pixel position, velocity, and acceleration of the maneuvering target in the next frame with the predicted position of the maneuvering target in the next frame, and determines the matching result.

[0130] In one embodiment, the Hungarian algorithm is used to match the pixel position, velocity, and acceleration of the maneuvering target in the next frame with its predicted position in the next frame. The matching result is determined by the degree of overlap between the pixel position and the predicted position and the similarity of appearance features. The matching result is then normalized, and a matching threshold is set. If the matching result is greater than or equal to the matching threshold, the matching result is considered successful. The weights of the degree of overlap between the pixel position and the predicted position and the similarity of appearance features are set based on the training recognition accuracy of the maneuvering target in the first and second scenes, respectively. For example, if the matching threshold is greater than or equal to 0.9, the weight of the degree of overlap in the matching result is 0.55, and the weight of the similarity of appearance features in the matching result is 0.45.

[0131] S500, if the movement status information of the maneuvering target is successfully matched with the predicted position of the maneuvering target in the next frame, the seeker is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene, which is any scene other than the first scene.

[0132] In this embodiment, if the movement status information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, it indicates that the maneuvering target is still the maneuvering target of the previous frame. The tracking state of the maneuvering target is maintained, and the seeker is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene. The second scene is any scene other than the first scene, which improves the tracking efficiency and accuracy of the maneuvering target.

[0133] In one embodiment, such as Figure 6 As shown, in step S500, if the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, the seeker is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene, including:

[0134] S510, if the pixel position, velocity and acceleration of the maneuvering target in the next frame are successfully matched with the predicted position of the maneuvering target in the next frame, the pixel position, velocity and acceleration of the maneuvering target in the next frame are continuously acquired.

[0135] S520, if the pixel position, velocity and acceleration of the maneuvering target in the next frame are successfully matched with the predicted position of the maneuvering target in the next frame, the attitude and velocity of the seeker corresponding to the maneuvering target are obtained.

[0136] S530, based on the pixel position, velocity and acceleration of the maneuvering target in the next frame, controls the attitude and velocity of the seeker corresponding to the maneuvering target in the first scene until the maneuvering target enters the second scene.

[0137] In this embodiment, after successful matching, the accuracy and precision of continuous target tracking are improved by controlling the attitude and velocity of the seeker corresponding to the pixel position, velocity, and acceleration of the maneuvering target. The seeker's attitude includes azimuth, pitch, and yaw angles, while its velocity includes travel speed and tracking angular velocity.

[0138] It should be noted that when tracking a maneuvering target, the seeker also evaluates the tracking confidence and tracking stability based on the target's pixel position, velocity, and acceleration, as well as the seeker's attitude and velocity, thereby improving the accuracy and precision of the seeker's continuous tracking of maneuvering targets.

[0139] S600 performs second target recognition and detection on the maneuvering target based on each second initial image of the maneuvering target in the second scene and the trained second YOLO neural network model.

[0140] In this embodiment, after the maneuvering target enters the second scene, each second initial image of the maneuvering target is acquired, and the maneuvering target is identified and detected using a trained second YOLO neural network model. Since the second YOLO neural network model uses a training set corresponding to the second scene, the recognition accuracy of the maneuvering target is improved after the maneuvering target changes scenes.

[0141] S700: If a maneuvering target is detected again in each of the second initial images, the seeker head is controlled to align with the maneuvering target until it makes contact with the maneuvering target.

[0142] In this embodiment, if a maneuvering target is detected again in each of the second initial images, the seeker head is controlled to aim at the maneuvering target until it makes contact with the maneuvering target, thereby achieving the purpose of close-range tracking or attacking the maneuvering target.

[0143] S800, if the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames, the trained first YOLO neural network model or the trained second YOLO neural network model is triggered to re-identify and detect the maneuvering target; where N≧3.

[0144] In this embodiment, if the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames, it indicates that the maneuvering target is occluded or temporarily disappears from the view of the seeker in the first or second scene. At this time, the trained first YOLO neural network model or the trained second YOLO neural network model is triggered to re-identify and detect the maneuvering target in order to find the lost maneuvering target. Thus, the seeker can still keep tracking the maneuvering target even when it fails to match in N consecutive frames, which improves the stability of the seeker's continuous tracking of the maneuvering target and also improves the accuracy of identification and detection.

[0145] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0146] Secondly, such as Figure 7 As shown, this application provides a device 100 for a seeker head to identify and track maneuvering targets, applied to a seeker head, comprising:

[0147] The acquisition module 110 is used to acquire multiple first initial images of the maneuvering target in the first scene.

[0148] The acquisition module 120 is used to preprocess each of the received first initial images to obtain preprocessed first initial images.

[0149] The first determining module 130 is used to perform first target recognition and detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model.

[0150] The second determining module 140 is used to determine the matching result between the motion state information of the moving target and the predicted position of the moving target in the next frame, based on the motion state information of the moving target and the tracker, if a moving target is detected for the first time in each of the first initial images.

[0151] The first control module 150 is used to control the seeker to track the maneuvering target in the first scene if the movement state information of the maneuvering target is successfully matched with the predicted position of the maneuvering target in the next frame, until the maneuvering target enters the second scene, which is any scene other than the first scene.

[0152] The detection module 160 is used to perform second target recognition and detection on the maneuvering target based on each second initial image of the maneuvering target in the second scene and the trained second YOLO neural network model.

[0153] The second control module 170 is used to control the seeker to align with the maneuvering target until it makes contact with the maneuvering target if the maneuvering target is detected again in each of the second initial images.

[0154] The trigger module 180 is used to trigger the trained first YOLO neural network model or the trained second YOLO neural network model to re-identify and detect the maneuvering target if the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames; wherein, N≧3, the first YOLO neural network model and the second YOLO neural network model are both improved YOLO neural network models.

[0155] Understandably, during the dynamic tracking of a maneuvering target, the results of identifying and detecting the maneuvering target or the location information of the tracked maneuvering target are encrypted, and then the encrypted data is transmitted to the control module of the identification and tracking device and the control center of the seeker.

[0156] It should be noted that the information interaction and execution process between the above-mentioned devices are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0157] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0158] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of the first aspects above.

[0159] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any one of the first aspects above.

[0160] Fifthly, this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in any one of the first aspects above.

[0161] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0162] The method for identifying and detecting mobile targets provided in this application can be applied to various terminal devices. This application does not impose any restrictions on the specific type of terminal device.

[0163] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.

[0164] The computer-readable medium may include at least: any entity or device capable of carrying computer program code to a photographic device / terminal device, recording media, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical discs. In some jurisdictions, according to legislation and patent practice, computer-readable media may not be electrical carrier signals or telecommunication signals.

[0165] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0166] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0168] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for a seeker head to identify and track maneuvering targets, characterized in that, Applied to seekers, including: Acquire multiple frames of the first initial image of the maneuvering target in the first scene; The received first initial images are preprocessed to obtain preprocessed first initial images; Based on the preprocessed first initial images and the trained first YOLO neural network model, the maneuvering target is identified and detected as a first target. If the moving target is detected for the first time in each of the first initial images, based on the movement state information of the moving target and the tracker, the matching result between the movement state information of the moving target and the predicted position of the moving target in the next frame is determined; If the movement state information of the maneuvering target matches the predicted position of the maneuvering target in the next frame, the seeker head is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene, which is any scene other than the first scene. Based on the second initial images of the maneuvering target in the second scenario and the trained second YOLO neural network model, the maneuvering target is identified and detected as a second target. If the maneuvering target is detected again in each of the second initial images, the seeker head is controlled to align with the maneuvering target until it makes contact with the maneuvering target; If the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames, the trained first YOLO neural network model or the trained second YOLO neural network model is triggered to re-identify and detect the maneuvering target. Where N≧3, both the first YOLO neural network model and the second YOLO neural network model are improved YOLO neural network models.

2. The method as described in claim 1, characterized in that, Preprocessing is performed on each of the received first initial images to obtain preprocessed first initial images, including: Denoising is performed on each of the first initial images based on its noise type to obtain the corresponding first denoised initial image; Each of the first noise-reduced initial images is subjected to histogram processing to obtain the corresponding first histogram initial image. A preset number of the first histogram initial images are randomly stitched together to form a large image, resulting in multiple first stitched initial images. Each of the first initial images is divided into multiple grids with a preset pixel area to obtain the preprocessed first initial images.

3. The method as described in claim 1, characterized in that, The trained first YOLO neural network model includes a backbone network, a neck network, and a head-improving prediction end; Based on the preprocessed first initial images and the trained first YOLO neural network model, the first target recognition and detection of the maneuvering target is performed, including: Based on the training and testing sets of the maneuvering target in the first scenario, the trained first YOLO neural network model is obtained; The backbone network of the trained first YOLO neural network model extracts the features corresponding to the maneuvering target from each of the preprocessed first initial images; The neck network of the trained first YOLO neural network model aggregates and refines the features corresponding to the maneuvering target; The head-improved prediction end of the trained first YOLO neural network model outputs the predicted bounding boxes of the maneuvering targets in each of the preprocessed first initial images, so as to perform first target recognition and detection on the maneuvering targets.

4. The method as described in claim 3, characterized in that, The predicted bounding box includes the probability of the presence of the maneuvering target, the probability of the target category, and the bounding box location information. The improved head prediction end includes a confidence loss function, a category loss function, and an improved location loss function. The head-level improved prediction terminal of the trained first YOLO neural network model outputs the predicted bounding boxes of the maneuvering targets in each of the preprocessed first initial images, including: The improved head prediction terminal outputs the probability of the presence and recognition of the maneuvering target in each of the preprocessed first initial images through a confidence loss function. The improved head prediction terminal outputs the target category probability of the maneuvering target in each of the preprocessed first initial images through a category loss function; The improved head prediction terminal outputs the bounding box position information of the maneuvering target in each of the preprocessed first initial images through an improved position loss function.

5. The method as described in claim 4, characterized in that, The improved position loss function outputs the bounding box position information of the maneuvering target in each of the preprocessed first initial images, including: Based on the intersection-union ratio, center Euclidean distance, minimum diagonal length, maximum diagonal length, balance parameter, and difference parameter, the improved location loss function value is determined; Based on the improved position loss function value and the movement state information of the maneuvering target in each of the preprocessed first initial images, the bounding box position information of the maneuvering target is output. Wherein, the center Euclidean distance is the Euclidean distance between the center points of the predicted box and the ground truth box, the minimum diagonal length is the minimum diagonal length within the minimum closed envelope of the predicted box and the ground truth box, the maximum diagonal length is the maximum diagonal length within the minimum closed envelope of the predicted box and the ground truth box, and the difference parameter is the difference in aspect ratio between the predicted box and the ground truth box.

6. The method as described in claim 1, characterized in that, The movement state information includes pixel position, velocity, and acceleration, and the pixel position represents the pixel coordinates of the centroid of the maneuvering target in the image frame; If the moving target is detected for the first time in each of the first initial images, based on the movement state information of the moving target and the tracker, the matching result between the movement state information of the moving target and the predicted position of the moving target in the next frame is determined, including: If the maneuvering target is detected for the first time in each of the first initial images, the pixel position, velocity and acceleration of the maneuvering target in the current frame and the pixel position, velocity and acceleration in the next frame are obtained respectively. Based on the pixel position, velocity, and acceleration of the maneuvering target in the current frame, the tracker obtains the predicted position of the maneuvering target in the next frame; The pixel position, velocity, and acceleration of the maneuvering target in the next frame are matched with the predicted position of the maneuvering target in the next frame, and the matching result is determined.

7. The method as described in claim 1, characterized in that, The movement status information includes pixel position, velocity, and acceleration; If the movement state information of the maneuvering target successfully matches the predicted position of the maneuvering target in the next frame, the seeker is controlled to track the maneuvering target in the first scene until the maneuvering target enters the second scene, including: If the pixel position, velocity, and acceleration of the maneuvering target in the next frame successfully match the predicted position of the maneuvering target in the next frame, the pixel position, velocity, and acceleration of the maneuvering target after the next frame are continuously acquired. If the pixel position, velocity, and acceleration of the maneuvering target in the next frame successfully match the predicted position of the maneuvering target in the next frame, the attitude and velocity of the seeker corresponding to the maneuvering target are obtained. Based on the pixel position, velocity, and acceleration of the maneuvering target in the next frame, the attitude and velocity of the seeker head corresponding to the maneuvering target in the first scene are controlled until the maneuvering target enters the second scene.

8. A device for a seeker head to identify and track a maneuvering target, characterized in that, Applied to seekers, including: The acquisition module is used to acquire multiple frames of the first initial image of the maneuvering target in the first scene; The acquisition module is used to preprocess each of the received first initial images to obtain preprocessed first initial images; The first determining module is used to perform first target recognition and detection on the maneuvering target based on the preprocessed first initial images and the trained first YOLO neural network model. The second determining module is used to determine, if the moving target is detected for the first time in each of the first initial images, the matching result between the moving target's movement state information and the tracker's predicted position in the next frame; The first control module is configured to control the seeker to track the maneuvering target in the first scene if the movement state information of the maneuvering target matches the predicted position of the maneuvering target in the next frame, until the maneuvering target enters the second scene, wherein the second scene is any scene other than the first scene into which the maneuvering target enters. The detection module is used to perform second target recognition and detection on the maneuvering target based on each second initial image of the maneuvering target in the second scene and the trained second YOLO neural network model; The second control module is used to control the seeker to align with the maneuvering target until it contacts the maneuvering target if the maneuvering target is detected again in each of the second initial images. The triggering module is used to trigger the trained first YOLO neural network model or the trained second YOLO neural network model to re-identify and detect the maneuvering target if the movement state information of the maneuvering target fails to match the predicted position of the maneuvering target in N consecutive frames. Where N≧3, both the first YOLO neural network model and the second YOLO neural network model are improved YOLO neural network models.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images