Target tracking method, device, electronic device and storage medium

Abstract

The present application relates to a target tracking method, device, electronic device and storage medium. The target tracking method includes: inputting an image to be processed into a preset deep learning model to obtain an initial detection area image of the tracking target; determining a quality reference value of the initial detection area image; in response to the quality reference value being less than a preset reference value, determining a historical detection area image with the highest matching degree with the initial detection area image from historical detection area images; based on the determined historical detection area image, correcting the initial detection area image to obtain a target detection area image; and tracking the tracking target based on the target detection area image. Through the present application, the problem of poor tracking trajectory continuity of the tracking target in the related art is solved, and the tracking trajectory continuity of the tracking target is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a target tracking method, device, electronic device and storage medium. Background Art

[0002] With the development of digital society, image data plays an irreplaceable role in video communication, security monitoring and other fields. Multi-target tracking plays a vital role in various application systems. Multi-target tracking is to assign an identifier to each object in each frame of the video and obtain the behavior trajectory of each identifier. Its input is the video sequence and the output is the behavior trajectory and identifier of the object.

[0003] Tracking algorithm is a classic and efficient way to track targets. It associates detection frames through similarity (position, appearance, motion and other information) to obtain tracking trajectories. Due to the complexity of the scenes in the video, the target detector cannot obtain perfect detection results. Missed detection, false detection and detection frame deviation often occur in the scene and are difficult to avoid. In order to deal with the problems of missed detection and false detection, most of the current tracking methods will select a threshold and only retain the detection results above this threshold to associate and obtain the tracking results. The detection results below this threshold are directly discarded. The advantage of this method is that it removes false detections to a large extent, but the detection results below this threshold are directly discarded, which can easily remove the detection results of the real target, resulting in poor continuity of the tracking trajectory of the tracked target.

[0004] With regard to the problem of poor continuity of the tracking trajectory of the tracked target in the related art, no effective solution has been proposed yet. Summary of the invention

[0005] In this embodiment, a target tracking method, device, electronic device and storage medium are provided to solve the problem of poor continuity of the tracking trajectory of the tracked target in the related art.

[0006] In a first aspect, a target tracking method is provided in this embodiment, including:

[0007] Input the image to be processed into the preset deep learning model to obtain the initial detection area image of the tracked target;

[0008] Determining a quality reference value of the initial detection area image;

[0009] In response to the quality reference value being less than a preset reference value, determining, from the historical detection area images, a historical detection area image having the highest matching degree with the initial detection area image;

[0010] Based on the determined historical detection area image, the initial detection area image is corrected to obtain a target detection area image;

[0011] The tracking target is tracked based on the target detection area image.

[0012] In some embodiments, the initial detection area image includes: N component area images of the tracking target, the component area image includes at least one component of the tracking target; and N is an integer greater than 0.

[0013] In some embodiments, determining the quality reference value of the initial detection area image includes:

[0014] Determining the N component area images in the initial detection area image;

[0015] Determining a sub-quality reference value of each of the N component area images;

[0016] Based on the determined sub-quality reference values, a quality reference value of the initial detection area image is determined.

[0017] In some of the embodiments, the tracking target includes a human body; the N component region images include: a head and shoulder region image and a torso region image corresponding to the human body.

[0018] In some embodiments, in response to the quality reference value being less than a preset reference value, determining from the historical detection area images a historical detection area image having the highest matching degree with the initial detection area image comprises:

[0019] When the quality reference value of the initial detection area image is less than the first sub-preset reference value, acquiring the head and shoulder features of the head and shoulder area image of the tracking target;

[0020] According to the head and shoulder features, a historical detection area image corresponding to the associated torso feature having the highest matching degree with the head and shoulder features is determined from the historical detection area.

[0021] In some embodiments, in response to the quality reference value being less than a preset reference value, determining from the historical detection area images a historical detection area image having the highest matching degree with the initial detection area image comprises:

[0022] When the quality reference value of the initial detection area image is less than the second sub-preset reference value, acquiring the head and shoulder features of the head and shoulder area image of the tracking target;

[0023] Determine a current quality reference value of the head and shoulder feature in the image to be processed, and determine an average head and shoulder feature corresponding to the head and shoulder region image in the historical detection region image;

[0024] Obtaining a target head-shoulder feature according to a current quality reference value of the head-shoulder feature in the image to be processed, the head-shoulder feature, and the average head-shoulder feature;

[0025] According to the target head and shoulder features, a historical detection area image corresponding to a torso feature having the highest matching degree with the target head and shoulder features is determined from the historical detection area.

[0026] In some embodiments, tracking the tracking target based on the target detection area image includes:

[0027] Inputting the target detection area image into a preset tracking neural network to obtain a tracking trajectory of the tracking target;

[0028] The tracking target is tracked according to the tracking trajectory.

[0029] In some of the embodiments, tracking the tracking target according to the tracking trajectory further includes:

[0030] Determine a tracking level of the tracking trajectory and a quality reference value level interval within which a quality reference value of the initial detection area image lies;

[0031] The tracking target is tracked according to the tracking level of the tracking trajectory and the quality reference value level interval in which the quality reference value of the initial detection area image lies.

[0032] In a second aspect, a target tracking device is provided in this embodiment, characterized in that it includes:

[0033] An input module is used to input the image to be processed into a preset deep learning model to obtain an initial detection area image of the tracked target;

[0034] A first determination module, used to determine a quality reference value of the initial detection area image;

[0035] A second determination module is used for determining, in response to the quality reference value being less than a preset reference value, a historical detection area image having the highest matching degree with the initial detection area image from the historical detection area images;

[0036] A correction module, used for correcting the initial detection area image based on the determined historical detection area image to obtain a target detection area image;

[0037] A tracking module is used to track the tracking target based on the target detection area image.

[0038] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the target tracking method described in the first aspect is implemented.

[0039] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the target tracking method described in the first aspect is implemented.

[0040] Compared with the related art, the present embodiment provides a target tracking method, device, electronic device and storage medium, by inputting the image to be processed into a preset deep learning model to obtain an initial detection area image of the tracking target; determining a quality reference value of the initial detection area image; in response to the quality reference value being less than a preset reference value, determining a historical detection area image with the highest matching degree with the initial detection area image from historical detection area images; based on the determined historical detection area image, correcting the initial detection area image to obtain a target detection area image; based on the target detection area image, tracking the tracking target in a manner to complete the tracking target, thereby solving the problem of poor tracking trajectory continuity of the tracking target in the related art and improving the tracking trajectory continuity of the tracking target.

[0041] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 is a hardware structure block diagram of a terminal of the target tracking method of this embodiment;

[0044] Figure 2 is a flow chart of the target tracking method of this embodiment;

[0045] Figure 3 is a comparison diagram of the tracking target head-shoulder sequence alignment and the tracking target sequence overall alignment in this embodiment;

[0046] Figure 4 is a schematic diagram of dividing the quality reference values ​​of the initial detection area image of this embodiment;

[0047] Figure 5 is a schematic diagram of matching the quality reference value of the initial detection area image with the tracking trajectory of this embodiment;

[0048] Figure 6 is a schematic diagram of tracking target shooting in the interactive scenario of this embodiment;

[0049] Figure 7 is a schematic diagram of tracking target shooting in different scenarios of this embodiment;

[0050] Figure 8 It is a structural block diagram of the target tracking device of this embodiment. DETAILED DESCRIPTION

[0051] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0052] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0053] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the target tracking method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0054] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the target tracking method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0055] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0056] In this embodiment, a target tracking method is provided. Figure 2 is a flow chart of the target tracking method of this embodiment. Figure 2 As shown, the process includes the following steps:

[0057] Step S201: input the image to be processed into a preset deep learning model to obtain an initial detection area image of the tracking target.

[0058] Before this step, the image to be processed in the image sequence may be obtained. The image sequence may be obtained in real time, such as a video stream obtained in real time may be used as the image sequence; the image sequence may also be obtained from a database storing the image sequence.

[0059] It should be noted that the image to be processed may be randomly acquired from an image sequence, or may be acquired according to actual needs of the user.

[0060] In this step, the preset deep learning model is a deep learning model that has been pre-trained and has a certain degree of recognition. The preset deep learning model can improve the accuracy of initial detection area image extraction to a certain extent.

[0061] Step S202, determining a quality reference value of the image in the initial detection area.

[0062] In this step, the quality reference value of the detection area image can be determined based on at least one of the completeness, clarity, etc. of the initial detection area image. For example, when the corresponding quality reference value is determined based on the completeness of the initial detection area image, the completeness and the corresponding quality reference value can be positively correlated, that is, the higher the completeness of the initial detection area image, the higher the quality reference value of the initial detection area image.

[0063] Step S203 , in response to the quality reference value being less than the preset reference value, determining from the historical detection area images a historical detection area image that has the highest matching degree with the initial detection area image.

[0064] In this step, based on the judgment of the quality reference value of the initial detection area image, and when the quality reference value of the initial detection area image is less than the preset quality reference value, a method of determining the historical detection area image with the highest matching degree with the initial detection area image from the historical detection area images can facilitate the subsequent completion of the initial detection area image, thereby improving the tracking continuity in the initial detection area image.

[0065] It should be noted that the historical detection area image is an image that is in the same image sequence as the image to be processed and is obtained after target tracking is performed before the image to be processed.

[0066] Step S204: Based on the determined historical detection area image, the initial detection area image is corrected to obtain a target detection area image.

[0067] In this step, the initial detection area image is patched based on the determined historical detection area image to obtain the target detection area image, thereby completing the initial detection area image and ensuring the integrity of the tracked target, thereby achieving the effect of improving the continuity of the tracking trajectory of the tracked target.

[0068] It should be noted that the method of repairing the initial detection area image based on the determined historical detection area image can be to repair the undetected area image part in the initial detection area image based on the determined historical detection area image, so as to improve the integrity of the tracking target.

[0069] Step S205: Tracking the tracking target based on the target detection area image.

[0070] In this step, the target is tracked by patching the target detection area image, thereby ensuring the integrity of the target and improving the continuity of the tracking trajectory of the target.

[0071] Based on the above steps S201 to S205, in response to the quality reference value being less than the preset reference value, a historical detection area image with the highest matching degree with the initial detection area image is determined from the historical detection area images, and based on the determined historical detection area image, the initial detection area image is corrected to obtain the target detection area image, and finally based on the target detection area image, the tracking target is tracked. This method realizes the completion of the initial detection area image with a low quality reference value, thereby improving the integrity of the image of the area to be detected corresponding to the tracking target, solving the problem of poor continuity of the tracking trajectory of the tracking target in the related technology, and also improves the continuity of the tracking trajectory of the tracking target.

[0072] In some of the embodiments, the initial detection area image includes: N component area images of the tracking target, the component area image includes at least one component of the tracking target; N is an integer greater than 0.

[0073] In this embodiment, a tracking target may have one or more components. Therefore, in this embodiment, by obtaining at least one component of the tracking target, it is convenient to subsequently implement the tracking target according to the at least one component, thereby achieving improved target tracking effect.

[0074] In some of the embodiments, determining the quality reference value of the initial detection area image includes: determining N component area images in the initial detection area image; determining a sub-quality reference value of each of the N component area images in the N component area images; and determining the quality reference value of the initial detection area image based on the determined sub-quality reference values.

[0075] In this embodiment, the quality reference value of the initial detection area image is calculated by combining the sub-quality reference values ​​of each of the N component area images, thereby improving the accuracy of the quality reference value of the initial detection area image.

[0076] In some of the embodiments, the tracking target includes a human body; the N component region images include: a head and shoulder region image and a torso region image corresponding to the human body.

[0077] In this embodiment, the detection of the tracking target is achieved based on the head and shoulder area image and the torso area image corresponding to the human body. When one of the head and shoulder area image and the torso area image is detected, the other undetected image can be completed based on the association between the two, thereby improving the anti-occlusion of the tracking target and further improving the continuity of the tracking trajectory of the tracking target.

[0078] It should be noted that the head and shoulder region image and the torso region image of each tracking target are in an associated state. For example, in a certain scene, when the head and shoulder region image and the torso region image of the tracking are detected at the same time, the head and shoulder region image and the torso region image can be associated, so that when one of them is detected later, the other undetected one can be supplemented based on the associated relationship.

[0079] In this embodiment, based on the head and shoulder region image and the torso region image, the quality reference value of the initial detection region image can be determined according to the following formula (1):

[0080]

[0081] in, Preset coefficients for users, dconf head Indicates the quality reference value of the head and shoulder area image, dconf body Indicates the quality reference value of the torso area image, dConf total Indicates the quality reference value of the initial detection area image.

[0082] In some of the embodiments, in response to a quality reference value being less than a preset reference value, determining a historical detection area image that has the highest degree of match with an initial detection area image from the historical detection area images includes: acquiring head and shoulder features of a head and shoulder area image of the tracked target when the quality reference value of the initial detection area image is less than a first sub-preset reference value; and determining, based on the head and shoulder features, a historical detection area image that corresponds to an associated torso feature that has the highest degree of match with the head and shoulder features from the historical detection area.

[0083] In most scenarios, the torso of the tracked target may be obstructed, while the head and shoulders are not obstructed. Therefore, in this embodiment, when the quality reference value of the initial detection area image is less than the first sub-preset quality reference value, the historical detection area image corresponding to the associated torso feature with the highest match to the head and shoulder feature is determined from the historical detection area based on the head and shoulder features, and then the historical detection area image corresponding to the associated torso feature with the highest match to the head and shoulder feature is determined, so as to facilitate subsequent completion based on the determined historical detection area image, thereby achieving the effect of improving the continuity of target tracking.

[0084] In some of the embodiments, in response to a quality reference value being less than a preset reference value, determining a historical detection area image with the highest degree of match with an initial detection area image from the historical detection area images includes: obtaining head and shoulder features of a head and shoulder area image of a tracking target when the quality reference value of the initial detection area image is less than a second sub-preset reference value; determining a current quality reference value of the head and shoulder features in the image to be processed, and determining an average head and shoulder feature corresponding to the head and shoulder area image in the historical detection area image; obtaining a target head and shoulder feature based on the current quality reference value, the head and shoulder feature, and the average head and shoulder feature of the head and shoulder feature in the image to be processed; and determining, based on the target head and shoulder feature, a historical detection area image corresponding to a torso feature with the highest degree of match with the target head and shoulder feature from the historical detection area.

[0085] In this embodiment, when the quality reference value of the initial detection area image is less than the second sub-preset quality reference value, the target head and shoulder features are obtained according to the current quality reference value, the head and shoulder features and the average head and shoulder features of the head and shoulder features in the image to be processed; and then, according to the target head and shoulder features, the historical detection area image corresponding to the torso feature with the highest matching degree with the target head and shoulder features is determined from the historical detection area. This method realizes determining the historical detection area image corresponding to the torso feature with the highest matching degree with the target head and shoulder feature based on the current quality reference value, the head and shoulder features and the average head and shoulder features of the head and shoulder features, so as to facilitate subsequent completion according to the determined historical and detection area images, thereby achieving the effect of improving the continuity of target tracking.

[0086] It should be noted that the value of the first sub-preset quality reference value is greater than the second sub-preset quality reference value, so that when the quality reference value of the initial detection area image is less than the second sub-preset quality reference value, the historical detection area image corresponding to the torso feature with the highest matching degree with the target head and shoulder feature can be further determined based on the current quality reference value of the head and shoulder feature, the head and shoulder feature and the average head and shoulder feature, so as to facilitate subsequent completion based on the determined historical detection area image, thereby improving the continuity of target tracking.

[0087] ReID, also known as "cross-camera tracking technology", is a hot topic in computer vision research. It mainly solves the identification and retrieval of pedestrians across cameras and scenes. This technology can be used as an important supplement to face recognition technology. It can continuously track pedestrians across cameras whose faces cannot be clearly photographed, thereby enhancing the spatiotemporal continuity of data. Pedestrian ReID belongs to pedestrian recognition and is an important research direction after face recognition. The object of study is the overall characteristics of a person. The main characteristic changes include clothing, body shape, hair, posture, etc. This case uses ReID technology to solve the problem of target tracking in a single-camera scene.

[0088] The problem of inaccurate ReID feature measurement is largely affected by factors such as occlusion and false background. For this reason, this embodiment no longer uses the detection of the tracking target for ReID like the previous tracking scheme, but instead uses the ReID features of the components (i.e., the head and shoulder features of the above embodiment). This method can weaken the feature degradation of the target; at the same time, it can also solve the problem of tracking target feature comparison and alignment; and reduce the image input size of the network, reduce the number of inputs for feature extraction, and reduce time consumption.

[0089] like Figure 3 As shown in the figure, the comparison of the tracking target parts and tracking target sequences is shown. The interaction between the tracking targets can easily introduce noise into the detection frame of the tracking target, and as the occluded area increases, the introduced noise increases, which can easily lead to the degradation of the tracking target features. However, the researchers found that the head and shoulders of the tracking target will not be occluded in most images, and there is a strong anti-occlusion ability. The image sequence corresponding to the head and shoulder features also introduces less noise during the occlusion interaction of the tracking target. Secondly, in this embodiment, the ReID feature is extracted for the tracking target, and the following formula (2) is used for feature fusion, where conf represents the quality reference value of the head and shoulder area image, Represents the average head and shoulder feature of the head and shoulder feature in the historical detection area image, The head and shoulder features of the image to be processed output through the network are weighted by combining the average head and shoulder features and the head and shoulder features of the image to be processed output through the network, thereby avoiding the fluctuation caused by the obvious degradation of the single-frame image features due to occlusion, and realizing the update of features to the greatest extent, thus avoiding the problem of head and shoulder feature degradation.

[0090]

[0091] The target detection frame of the tracked target usually fluctuates in position, width and height. If the image quality is poor or there is occlusion between the targets, this effect will be more obvious. For example, when the target is not occluded at the beginning of tracking, and then the torso area is occluded, the features of the partial area of ​​the tracked target are extracted at this time. However, the previous features come from the entire tracked target, which will bring about the problem of feature comparison and alignment of the tracked target, thereby causing feature differences; however, in this embodiment, the head and shoulder features with strong anti-occlusion are used instead, and then combined with the previous average head and shoulder features to achieve anti-feature attenuation, so as to avoid the problem of feature alignment.

[0092] At the same time, in this embodiment, by using head and shoulder features with strong anti-occlusion properties for input, the input of the entire tracking target is avoided, thereby reducing the forward reasoning time of the neural network and speeding up the network processing speed; secondly, feature extraction is only performed when the features are needed to detect and track the target during the matching process, reducing the number of ReID features that need to be calculated.

[0093] In some of the embodiments, tracking the tracking target based on the target detection area image includes: inputting the target detection area image into a preset tracking neural network to obtain a tracking trajectory of the tracking target; and tracking the tracking target according to the tracking trajectory.

[0094] In this embodiment, the tracking trajectory of the tracking target is obtained by inputting the target detection area image into a preset tracking neural network; the tracking target is tracked according to the tracking trajectory, thereby realizing target tracking. At the same time, tracking is performed based on the completed target detection area image, thereby improving the continuity of target tracking.

[0095] In some of the embodiments, a tracking level of the tracking trajectory and a quality reference value level interval of a quality reference value of an initial detection area image are determined; and a tracking target is tracked according to the tracking level of the tracking trajectory and the quality reference value level interval of the quality reference value of the initial detection area image.

[0096] like Figure 4 As shown, when the quality reference value of the initial detection area image is lower than the low-score detection frame threshold, the initial detection area image can be discarded. In this case, the initial detection area image can be identified as a false detection; when the quality reference value of the initial detection area image is between (λ1, λ2], it is classified as a low-score detection frame; when the quality reference value of the initial detection area image is between (λ2, λ3], it is classified as a medium-score detection frame; when the quality reference value of the initial detection area image is between (λ3, 1], it is classified as a high-score detection frame.

[0097] The score of the historical tracking target trajectory comes from the mixed detection score. In target tracking, it is necessary to predict the score of the tracking target trajectory in the image to be processed. In this embodiment, the historical tracking target trajectory is used for coefficient attenuation. The following formula (3) uses a window of length 4 to calculate the historical tracking target trajectory score, and then superimposes the score for prediction. When TConf i+1 When it is greater than the high-score trajectory threshold τ, it is judged as a high-score trajectory. i+1 When it is less than the high-score trajectory threshold τ, it is judged as a low-score trajectory.

[0098] TConf i+1 =a*Tconf i-3 +b*Tconf i-2 +c*Tconf i-1 +(1-abc)*Tconf i +(Tconf i -Tconf i-3 ) / 3 (3)

[0099] Wherein, TConf represents the quality reference value of the initial detection area image, a, b and c are the score fusion coefficients (a<b<c), and the subscript i represents the sequence number of the current image to be processed.

[0100] In this embodiment, the detection frames corresponding to each head, shoulder and torso are divided into three categories according to the scores, namely, high-score detection frames, medium-score detection frames and low-score detection frames, and a total of three matches are performed. The entire matching process includes:

[0101] Step 1: The first match is performed using the high-score detection box and all tracking tracks, such as Figure 5 Connection number 1.

[0102] Step 2: Use the middle score detection frame and the remaining high score tracking tracks that did not match the high score detection frame in the first time to match, such as Figure 5 Connection 2.

[0103] Step 3: Extract features of the tracking track components that are not matched to the detection target to construct a feature base library. Figure 5 The secondary residue of the high- and medium-score trajectories and the primary residue of the low-score trajectories are used to extract features; component features are extracted from the unmatched medium- and low-score detection frames, that is, Figure 5 The first remaining and low-score detection frames of the medium-score detection frame are used for feature extraction, and the detection results are matched with the unmatched tracking trajectories using the features. The similarity between the low-score detection frame and the tracking trajectory is integrated, and the features of the parts are used to mine the real objects from the low-score frame.

[0104] Step 4: For the detection boxes that do not match the tracking trajectory but have a high enough score, such as Figure 5The first remaining part in the , it creates a new tracking track object. For the tracking track that does not match the detection box, such as Figure 5 The tertiary residuals of the medium and high-scoring trajectories and the quadratic residuals of the low-scoring trajectories are retained for 36 frames and matched when the detection appears again.

[0105] In this embodiment, the matching between the target detection area image and the tracking trajectory is achieved through the above method, thereby achieving the tracking of the tracking target corresponding to the target detection area image. At the same time, the detection area image and the tracking trajectory are associated multiple times in combination with the quality reference value to avoid the competition error caused by a single mixed association. Secondly, the component feature anti-occlusion weakens the feature degradation and speeds up the network reasoning. At the same time, it is bound to the head and shoulder area image and the torso area image for matching and comparison, so that the anti-occlusion component feature can find the lost target within a certain frame number range.

[0106] Through the above embodiments, the present application obtains the input image to be processed from the image sequence, inputs the image to be processed into the pedestrian multi-part detection algorithm, obtains the head and shoulder area image and the torso area image, and then completes the target detection based on the deep learning model, and simultaneously outputs the head and shoulder area image and the torso area image of each tracking target in the target detection algorithm. Then the detected head and shoulder area image and torso area image are data associated, the relationship between the head and shoulder area image and the torso area image and the corresponding tracking target is determined, and then the tracking target pedestrian is completed based on the quality reference value of the head and shoulder area image and the torso area image. Finally, the quality reference value is mixed and calculated based on the previous association result of the head and shoulder area image and the torso area image to divide the detection result into score intervals, and the sub-parts in the low score interval are sent to the ReID network to preferentially extract the appearance features, and the appearance features possessed by each sub-part are bound. Finally, the position of the target in the next scene is predicted, and the detection result and the tracking trajectory are matched three times using the score interval division method to complete the tracking association.

[0107] In some embodiments, the difficulty of multi-target pedestrian tracking lies in the problem of target detection accuracy caused by occlusion and interaction between targets, which can easily lead to disconnection of tracking trajectories or association errors. When pedestrian A is occluded by other pedestrians and objects, the detection output result quality reference value is low or missed for the detection network. In this case, the current scene cannot be matched to the correct target again for target tracking. Secondly, association errors are easily caused by relying solely on matching between detection frames.

[0108] In this application, the commonly used yolox is used as the target detection network (i.e., the preset deep learning model in the above embodiment). Figure 6As shown in the figure, the target detection results in a densely occluded scene are shown (for clear display, only part of the detection frame is shown). The target corresponding to the smallest detection frame is mostly occluded, which makes it impossible to detect the torso, but the head and shoulder area is still visible, and the head and shoulders of the target can be detected. This is because the head and shoulders of the target in a crowded and dense scene have a stronger ability to resist occlusion than the torso. When the network detects the target, its detection effect is often weakened due to partial occlusion.

[0109] Target detection will output the head, shoulder and torso regions of the tracked target at the same time. The head, shoulder and torso regions need to be related to each other to determine whether they belong to the same identity. At the same time, the head, shoulder and torso regions of the tracked target can be used to make up for the loss of the other detection to increase the continuity of detection. If the quality reference value of one detection result is low, the other can be used to identify whether the real target exists. Figure 7 As shown in the figure, a normal independent tracking target will detect and output its head and shoulder area and torso area at the same time, and determine that they are the same target by associating them. When the targets are interactively occluded, the head and shoulder areas of the tracking targets can be detected. However, since the torso occlusion of some tracking targets is too serious, the torso area cannot be detected. At this time, the torso area associated with the head and shoulder area can be used to complete the torso area to improve the input of the target detection network; when the human body is occluded by an object, the corresponding torso is also output through the detected results.

[0110] According to the head and shoulder area of ​​the tracking target, the basic association condition should be established based on the prior information inside the tracking target. This application will also perform repair association in combination with the quality reference value when associating the head and shoulder area and the torso area, as shown above. Figure 7 In interactive occlusion scenarios, when two head-shoulder regions with higher quality reference values ​​compete for a torso at the same time, they can be assigned to one of the head-shoulder regions based on their positional relationship, while the other head-shoulder region will generate a detection result based on its own detection result and the previously associated one, thereby ensuring the continuity of the detection output.

[0111] In this embodiment, a target tracking device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0112] Figure 8 is a structural block diagram of the target tracking device of this embodiment, such as Figure 8 As shown, the device comprises:

[0113] An input module 81 is used to input the image to be processed into a preset deep learning model to obtain an initial detection area image of the tracked target;

[0114] A first determination module 82, coupled to the input module 81, for determining a quality reference value of the initial detection area image;

[0115] A second determination module 83, coupled to the first determination module 82, is configured to determine, in response to the quality reference value being less than a preset reference value, a historical detection area image having the highest matching degree with the initial detection area image from the historical detection area images;

[0116] A correction module 84, coupled to the second determination module 83, is used to correct the initial detection area image based on the determined historical detection area image to obtain a target detection area image;

[0117] The tracking module 85 is coupled to the correction module 84 and is used to track the tracking target based on the target detection area image.

[0118] In some of the embodiments, the initial detection area image includes: N component area images of the tracking target, the component area image includes at least one component of the tracking target; N is an integer greater than 0.

[0119] In some of the embodiments, the first determination module 82 includes: a first determination unit, used to determine N component area images in the initial detection area image; a second determination unit, used to determine the sub-quality reference value of each of the N component area images; and a third determination module, used to determine the quality reference value of the initial detection area image based on the determined sub-quality reference values.

[0120] In some of the embodiments, the tracking target includes a human body; the N component region images include: a head and shoulder region image and a torso region image corresponding to the human body.

[0121] In some of the embodiments, the second determination module 83 includes: an acquisition unit, used to acquire the head and shoulder features of the head and shoulder area image of the tracked target when the quality reference value of the initial detection area image is less than the first sub-preset reference value; a first matching unit, used to determine, from the historical detection area, based on the head and shoulder features, the historical detection area image corresponding to the associated torso feature with the highest matching degree with the head and shoulder features.

[0122] In some of the embodiments, the second determination module 83 also includes: a second acquisition unit, used to acquire the head and shoulder features of the head and shoulder area image of the tracking target when the quality reference value of the initial detection area image is less than the second sub-preset reference value; a fourth determination module, used to determine the current quality reference value of the head and shoulder features in the image to be processed, and determine the average head and shoulder features corresponding to the head and shoulder area images in the historical detection area images; a calculation unit, used to obtain the target head and shoulder features based on the current quality reference value of the head and shoulder features in the image to be processed, the head and shoulder features and the average head and shoulder features; a second matching unit, used to determine, from the historical detection area, the historical detection area image corresponding to the torso feature with the highest matching degree with the target head and shoulder features.

[0123] In some of the embodiments, the tracking module 85 includes: an input unit, used to input the target detection area image into a preset tracking neural network to obtain a tracking trajectory of the tracking target; and a tracking unit, used to track the tracking target according to the tracking trajectory.

[0124] In some of the embodiments, the tracking unit includes: a determination subunit, used to determine the tracking level of the tracking trajectory and the quality reference value level interval in which the quality reference value of the initial detection area image is located; and a tracking subunit, used to track the tracking target according to the tracking level of the tracking trajectory and the quality reference value level interval in which the quality reference value of the initial detection area image is located.

[0125] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0126] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0127] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0128] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0129] Step S201: input the image to be processed into a preset deep learning model to obtain an initial detection area image of the tracking target.

[0130] Step S202, determining a quality reference value of the image in the initial detection area.

[0131] Step S203 , in response to the quality reference value being less than the preset reference value, determining from the historical detection area images a historical detection area image that has the highest matching degree with the initial detection area image.

[0132] Step S204: Based on the determined historical detection area image, the initial detection area image is corrected to obtain a target detection area image.

[0133] Step S205: Tracking the tracking target based on the target detection area image.

[0134] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0135] In addition, in combination with the target tracking method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the target tracking method. The storage medium stores a computer program, and when the computer program is executed by a processor, any target tracking method in the above embodiments is implemented.

[0136] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0137] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0138] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0139] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A target tracking method, characterized in that: include: Input the image to be processed into the preset deep learning model to obtain the initial detection area image of the tracked target; Determining a quality reference value of the initial detection area image; In response to the quality reference value being less than a preset reference value, determining, from the historical detection area images, a historical detection area image having the highest matching degree with the initial detection area image; Based on the determined historical detection area image, the initial detection area image is corrected to obtain a target detection area image; Tracking the tracking target based on the target detection area image; Among them, in response to the quality reference value being less than a preset reference value, determining from the historical detection area images a historical detection area image that has the highest matching degree with the initial detection area image includes: when the quality reference value of the initial detection area image is less than a first sub-preset reference value, acquiring the head and shoulder features of the head and shoulder area image of the tracking target; and based on the head and shoulder features, determining from the historical detection area a historical detection area image that corresponds to the associated torso features that have the highest matching degree with the head and shoulder features.

2. The target tracking method according to claim 1, characterized in that: The initial detection area image includes: N component area images of the tracking target, and the component area image includes at least one component of the tracking target; N is an integer greater than 0.

3. The target tracking method according to claim 2, characterized in that: Determining the quality reference value of the initial detection area image includes: Determining the N component area images in the initial detection area image; Determining a sub-quality reference value of each of the N component area images; Based on the determined sub-quality reference values, a quality reference value of the initial detection area image is determined.

4. The target tracking method according to claim 3, characterized in that: The tracking target includes a human body; the N component region images include: a head and shoulder region image and a torso region image corresponding to the human body.

5. The target tracking method according to claim 2, characterized in that: In response to the quality reference value being less than a preset reference value, determining from the historical detection area images a historical detection area image having the highest matching degree with the initial detection area image comprises: When the quality reference value of the initial detection area image is less than the second sub-preset reference value, acquiring the head and shoulder features of the head and shoulder area image of the tracking target; Determine a current quality reference value of the head and shoulder feature in the image to be processed, and determine an average head and shoulder feature corresponding to the head and shoulder region image in the historical detection region image; Obtaining a target head-shoulder feature according to a current quality reference value of the head-shoulder feature in the image to be processed, the head-shoulder feature, and the average head-shoulder feature; According to the target head and shoulder features, a historical detection area image corresponding to a torso feature having the highest matching degree with the target head and shoulder features is determined from the historical detection area.

6. The target tracking method according to claim 1, characterized in that: Tracking the tracking target based on the target detection area image includes: Inputting the target detection area image into a preset tracking neural network to obtain a tracking trajectory of the tracking target; The tracking target is tracked according to the tracking trajectory.

7. The target tracking method according to claim 6, characterized in that: Tracking the tracking target according to the tracking trajectory also includes: Determine a tracking level of the tracking trajectory and a quality reference value level interval within which a quality reference value of the initial detection area image lies; The tracking target is tracked according to the tracking level of the tracking trajectory and the quality reference value level interval in which the quality reference value of the initial detection area image lies.

8. A target tracking device, characterized in that: include: An input module is used to input the image to be processed into a preset deep learning model to obtain an initial detection area image of the tracked target; A first determination module, used to determine a quality reference value of the initial detection area image; A second determination module is used for determining, in response to the quality reference value being less than a preset reference value, a historical detection area image having the highest matching degree with the initial detection area image from the historical detection area images; A correction module, used for correcting the initial detection area image based on the determined historical detection area image to obtain a target detection area image; A tracking module, used for tracking the tracking target based on the target detection area image; Among them, in response to the quality reference value being less than a preset reference value, determining from the historical detection area images a historical detection area image that has the highest matching degree with the initial detection area image includes: when the quality reference value of the initial detection area image is less than a first sub-preset reference value, acquiring the head and shoulder features of the head and shoulder area image of the tracking target; and based on the head and shoulder features, determining from the historical detection area a historical detection area image that corresponds to the associated torso features that have the highest matching degree with the head and shoulder features.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the target tracking method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the target tracking method according to any one of claims 1 to 7 are implemented.

Images