Photoelectric target identifying and tracking method and device

Abstract

The invention discloses a photoelectric target identifying and tracking method and device, and belongs to the field of target tracking. The method comprises the following steps: performing target identification on a plurality of frames of cached images according to a preset identification algorithm to obtain identification information of a plurality of targets; performing target batch processing on all the targets according to the identification information, and performing tracking initialization on the processed to-be-tracked targets; and correcting the tracking target which does not meet the preset requirement in the tracking algorithm according to the batching result, so that the tracking algorithm is in a correct tracking state. According to the method, target identification, batching and tracking algorithms are organically combined, unmanned tracking of the photoelectric target can be realized, and the tracking performance is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking, and in particular to a photoelectric target recognition and tracking method and device. Background Art

[0002] Target tracking and target type identification are important technologies for realizing unmanned optoelectronic products. The two are controlled by different algorithms: the target recognition algorithm is responsible for detecting the target in a single frame and giving the target classification; the target tracking algorithm requires prior target information and is responsible for finding and predicting the position of the specified target in the image and controlling the servo turntable to pull the target to the center of the field of view.

[0003] In the related art, when tracking optoelectronic targets, it is necessary to manually select the target. This results in that fast-moving targets often cannot be selected exactly at the center. At the same time, small targets are prone to selecting a large amount of background information and causing loss, which in turn leads to poor target tracking effects.

[0004] Based on this, there is an urgent need for an optoelectronic target recognition and tracking method and device to solve the above technical problems. Summary of the invention

[0005] The present invention provides a photoelectric target recognition and tracking method and device, which can effectively improve the target tracking effect and realize unmanned tracking of photoelectric targets.

[0006] On the one hand, a method for photoelectric target recognition and tracking is provided, the method comprising:

[0007] Performing target recognition on the multi-frame buffer images according to a preset recognition algorithm to obtain recognition information of multiple targets;

[0008] Performing target batch processing on all the targets according to the identification information, and initializing tracking on the processed targets to be tracked;

[0009] According to the batching results, the tracking targets that do not meet the preset requirements in the tracking algorithm are corrected to ensure that the tracking algorithm is in the correct tracking state.

[0010] On the other hand, an optoelectronic target recognition and tracking device is provided, which is used to implement the steps of the optoelectronic target recognition and tracking method described above, wherein the device comprises an embedded ARM chip and a multi-core DSP chip deployed on an embedded platform, wherein:

[0011] The ARM chip is used to perform target recognition on the image and to batch the recognized targets;

[0012] The DSP chip is used to initialize and continuously track the target to be tracked after identification and batching.

[0013] On the other hand, a computer device is provided, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of the above-mentioned optoelectronic target recognition and tracking method.

[0014] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned optoelectronic target recognition and tracking method are implemented.

[0015] On the other hand, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the above-mentioned optoelectronic target recognition and tracking method are implemented.

[0016] The technical solution provided by the present invention can at least bring the following beneficial effects: automatically identify the target through the pre-cached images, determine the target to be tracked, then batch the recognition results, and initialize the tracking of the batched targets, and continuously judge whether the target is successfully tracked according to the batch information during the tracking period, and end the current tracking process when receiving the instruction to stop tracking. This method combines the target recognition algorithm, the target tracking algorithm, and the target batching, so that the creation of tracking does not require manual processing, which greatly improves the tracking performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is a flow chart of a photoelectric target recognition and tracking method provided by one embodiment of the present invention;

[0019] Figure 2 It is a structural diagram of an optoelectronic target recognition and tracking device provided by one embodiment of the present invention;

[0020] Figure 3 It is a hardware architecture diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0022] As mentioned above, tracking in current mainstream optoelectronic products requires people to select the target. For fast-moving targets, it is often impossible to select the target exactly in the center, and for smaller targets, it is easy to select a large amount of background information and cause loss.

[0023] Based on this, the concept of the present invention is to combine the tracking algorithm and the recognition algorithm. The tracking algorithm establishes tracking through recognition information as prior information, and corrects the tracking result through subsequent recognition information to combat occlusion, target intersection and the like.

[0024] The specific implementation of the above concept is described below.

[0025] Please refer to Figure 1 , an optoelectronic target recognition and tracking method provided by an embodiment of the present invention comprises:

[0026] Step 100, performing target recognition on multiple frame buffer images according to a preset recognition algorithm to obtain recognition information of multiple targets;

[0027] Step 102, performing target batch processing on all the targets according to the identification information, and initializing tracking of the processed targets to be tracked;

[0028] Step 104, correcting the tracking targets that do not meet the preset requirements in the tracking algorithm according to the compilation and batching results, so that the tracking algorithm is in a correct tracking state.

[0029] In the embodiment of the present invention, the target recognition is automatically performed through the pre-cached images to determine the target to be tracked, and then the recognition results are batched, and the tracking of the batched targets is initialized. During the tracking period, it is continuously determined whether the target is successfully tracked based on the batching information, and the current tracking process is terminated when the instruction to stop tracking is received. This method combines the target recognition algorithm, the target tracking algorithm, and the target batching, so that the creation of tracking does not require manual processing, which greatly improves the tracking performance. .

[0030] Described below Figure 1 How the various steps are performed.

[0031] First, with respect to step 100, target recognition is performed on multiple frame buffer images according to a preset recognition algorithm to obtain recognition information of multiple targets.

[0032] In actual applications, there is a delay in sending the target recognition results to the target tracking algorithm. The target tracking result output delay is about 120ms, and the target tracking initialization position input delay is determined by the frame rate and generally does not exceed 40ms.

[0033] Therefore, in the embodiment of the present invention, 5 frames of images are cached, that is, 5 frames of images before the current moment are cached, so as to reduce the time delay problem between recognition and tracking, and further improve the tracking accuracy.

[0034] Furthermore, after identifying a target of a specified type, the identification information including the target size, target position and target type is passed to the tracking algorithm, and the identification result and the cached image are matched according to the time information. After the matching is successful, subsequent initialization tracking is performed on this image.

[0035] For example, assuming that five different positions of the same target are identified in the image, then each position is combined with the corresponding image according to the order of the positions and the time information of the cached images.

[0036] Then, with respect to step 102, target batch processing is performed on all the targets according to the identification information, and tracking initialization is performed on the processed targets to be tracked.

[0037] In the embodiment of the present invention, the batching process includes determining the same target identified in each frame according to the recognition information of the continuous multiple frames of images, and attaching the same batch number to the same target. That is to say, target batching is a process of associating targets between the upper and lower frames according to the recognition results, that is, judging whether the targets identified between the upper and lower frames are the same. For example, a target is identified in the first frame of the image, and the batch number is set to 1. The batch number of the same target is set to 1 in the subsequent 2-5 frames.

[0038] After completing the batching of all targets, the target tracking needs to be initialized. This process includes determining whether the actual position obtained by identifying the target to be tracked in the current frame image corresponds to the position predicted by the tracking algorithm; if so, the tracking initialization is determined to be successful and the batch number of the target to be tracked is retained, otherwise image recognition and target batching are performed again.

[0039] With respect to step 104, the tracking targets that do not meet the preset requirements in the tracking algorithm are corrected according to the compilation and batching results, so that the tracking algorithm is in a correct tracking state.

[0040] During the tracking process, the target may face multiple target intersections or be obscured, which requires constant verification of the tracking algorithm so that the tracking strategy can be adjusted in time if tracking errors occur in the tracking algorithm.

[0041] In the embodiment of the present invention, the tracking process includes the following steps: judging whether the target tracked by the tracking algorithm is the target to be tracked according to the target batching result; if so, keeping tracking until the task is completed, otherwise stopping tracking of the current target and initializing tracking of the identified target with the same batch number;

[0042] Determine whether the target to be tracked in the current frame image is lost based on the previous frame image; if the target is lost, maintain the motion tracking state within the preset time threshold, and re-initialize the tracking after identifying the same type of target; when the target is lost and still does not appear after exceeding the time threshold, end the current tracking task.

[0043] Specifically, if the difference between the size information of the target tracked by the tracking algorithm and the size information of the target to be tracked after batch processing exceeds a threshold, it can be considered that the target currently being tracked by the tracking algorithm is not the target to be tracked, that is, the tracking is wrong, and it is necessary to re-initialize the tracking of the identified targets with the same batch number.

[0044] For example, assuming that the batch number of the target to be tracked is 1, if in a certain frame during the tracking process the target being tracked is found to be batch number 2, it is considered a tracking error, and the target identified in this frame as batch number 1 is tracked again. This method can solve the problem of target tracking being lost or mistracked due to violent maneuvers or intersections.

[0045] Furthermore, if it is found that the correlation between the target in the previous frame image and the current frame image is lower than the preset threshold, it can be determined that the target is lost. At this time, it is necessary to pause the target tracking algorithm, control the turntable to maintain the speed before the loss, and set a time, for example, 3 seconds. Within 3 seconds, if the same type of target is re-identified, the tracking algorithm will be re-initialized on this target. If it exceeds 3 seconds, it is determined that the target does not appear and the tracking task is terminated.

[0046] In summary, the method provided by the embodiment of the present invention organically combines target recognition, target batching and tracking algorithms, so that no human is required to create tracking. The tracking performance is greatly improved, so that the target tracking creation time does not exceed 120ms at most, and the minimum 8x8 size target can be tracked, the minimum trackable size is less than 3x3 pixels, and the target occlusion of less than 3s can be resisted.

[0047] Please refer to Figure 2 The embodiment of the present invention provides an optoelectronic target recognition and tracking device, which includes an embedded ARM chip and a multi-core DSP chip deployed on an embedded platform, wherein:

[0048] The ARM chip is used to identify targets in images and batch the identified targets; the DSP chip is used to initialize and continuously track the targets to be tracked after identification and batching.

[0049] Specifically, in the DSP chip, three cores are responsible for running the target tracking algorithm, and one core is responsible for controlling image input, identifying and batching data input, target tracking result output, and tracking correction process.

[0050] After the DSP is powered on, each core is initialized, and after the image is input normally, the image is cached in the off-chip DDR and covered in a loop. If target recognition and batching information is received, the information is stored in the on-chip shared memory and covered in a loop; and an inter-core interrupt is given to notify the tracking algorithm to initialize tracking on the cached image. After the tracking is established, the tracking algorithm fills the tracking position, size, miss amount, and status into the specified area of ​​the shared memory, and the present invention verifies based on this information. If the verification passes, the miss amount is forwarded to the servo turntable, and the tracking target position size is sent to the ARM. If it fails, the reinitialization or anti-occlusion process is entered.

[0051] It should be noted that the optoelectronic target recognition and tracking device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the optoelectronic target recognition and tracking device provided in the above embodiment and the optoelectronic target recognition and tracking method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0052] The embodiment of the present application also provides a computer device, please refer to Figure 3 The computer device includes a processor and a memory, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the optoelectronic target recognition and tracking method provided by the above-mentioned method embodiments.

[0053] An embodiment of the present application also provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the optoelectronic target recognition and tracking method provided by the above-mentioned method embodiments.

[0054] An embodiment of the present application also provides a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the optoelectronic target recognition and tracking method described in any of the above embodiments.

[0055] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0056] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0057] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0058] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A photoelectric target recognition and tracking method, characterized in that: The method comprises: Performing target recognition on the multi-frame buffer images according to a preset recognition algorithm to obtain recognition information of multiple targets; Performing target batch processing on all the targets according to the identification information, and initializing tracking on the processed targets to be tracked; According to the batching results, the tracking targets that do not meet the preset requirements in the tracking algorithm are corrected to ensure that the tracking algorithm is in the correct tracking state.

2. The method according to claim 1, characterized in that: The identification information includes target size, target location and target type.

3. The method according to claim 1, characterized in that The batch processing of all the targets according to the identification information and initializing tracking of the processed targets to be tracked includes: According to the recognition information of multiple consecutive frames of images, the same target identified in each frame of image is determined, and the same batch number is attached to the same target; Determine whether the actual position of the target to be tracked identified in the current frame image corresponds to the position predicted by the tracking algorithm; if so, determine that the tracking initialization is successful and retain the batch number of the target to be tracked, otherwise re-perform image recognition and target batching processing.

4. The method according to claim 3, characterized in that The step of correcting the tracking target that does not meet the preset requirements in the tracking algorithm according to the batch compilation result includes: Determine whether the target being tracked by the tracking algorithm is the target to be tracked according to the target batching result; If yes, keep tracking until the task is finished, otherwise stop tracking the current target and initialize tracking for the identified target with the same batch number; Determine whether the target to be tracked in the current frame image is lost according to the previous frame image; If the target is lost, the motion tracking state is maintained within the preset time threshold, and the tracking is re-initialized after the same type of target is identified; when the target is lost and does not appear after the time threshold, the current tracking task is terminated.

5. The method according to claim 4, characterized in that The step of judging whether the target tracked by the tracking algorithm is the target to be tracked according to the target batching result includes: When the difference between the size information of the target tracked by the tracking algorithm and the size information of the target to be tracked after batch processing exceeds a threshold, it is determined that the target tracked by the tracking algorithm is not the target to be tracked.

6. The method according to claim 4, characterized in that The step of judging whether the target to be tracked in the current frame image is lost according to the previous frame image includes: When the target correlation between the two frames of images is lower than a threshold, it is determined that the target to be tracked has been lost.

7. An optoelectronic target recognition and tracking device, characterized in that: Applied to the optoelectronic target recognition and tracking method according to any one of claims 1 to 6, the device comprises an embedded ARM chip and a multi-core DSP chip deployed on an embedded platform, wherein: The ARM chip is used to perform target recognition on the image and to batch the recognized targets; The DSP chip is used to perform tracking initialization and continuous tracking on the target to be tracked that is determined after identification and batching.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of any one of the methods described in claims 1-6.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The method comprises a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images