A method and system for fast tracking based on multi-person movement

Abstract

The application discloses a kind of method and system based on the quick tracking of multi-person movement, comprising the following steps: establishing a recognition model, the image collected is input into recognition model;Human body area coordinates, human body recognition reliability, face area coordinates, human body area image and face area image are output by recognition model;Face area image output by recognition model is compared with face area image in storage module;If not match, update storage module, and according to the updated storage module, display module is synchronously updated;If match, according to the human body recognition reliability, human body area image output by recognition model is judged, and according to the difference judgment result, storage module is updated, and according to the updated storage module, display module is synchronously updated.The application is used to solve the technical problems of long operation time and ID repetition and waste when multi-target tracking is carried out in prior art.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method and system for fast tracking of multiple human movements. Background Technology

[0002] The most common method for target tracking is feature value tracking. However, when a character moves, it may be occluded, resulting in significantly different target feature values. To address this, existing technologies add new IDs for management, leading to ID duplication and waste, and also causing the loss of target tracking for older IDs.

[0003] The following are the main situations in which target feature values ​​with significant differences are formed:

[0004] (1) When a person's clothes get dirty, it will create target feature values ​​that are significantly different;

[0005] (2) When tracking people, because people move or update their clothing outside the video range, they will form significantly different target feature values.

[0006] (3) When a person’s limbs move too much, the feature value calculation area is relatively large. Too much background is included in the feature value calculation, which will result in significantly different target feature values.

[0007] In addition to the above-mentioned situations where new IDs are added for management, when a target disappears from the video area for a long time, the target tracking will lose feature values ​​based on the duration. When the target re-enters the video area, it will also be tracked according to the new feature value ID, which will cause ID duplication and waste.

[0008] Besides the problems of ID duplication and waste, existing technologies also rely on feature values ​​for updates. Since human body images are relatively large, existing technologies require image scaling before feature value calculation and comparison, resulting in long computation times and hindering fast tracking.

[0009] In summary, current feature-based multi-target tracking computation time is directly proportional to the number of people within the video range. Furthermore, the size of the human image directly determines the complexity of feature calculation. Moreover, the free movement of people and the exchange of external scenes mean that simply using feature values ​​for judgment will lead to the storage waste of duplicate IDs, which consumes resources and is not conducive to improving tracking accuracy. Summary of the Invention

[0010] To overcome the shortcomings of existing technologies, this invention provides a method and system for fast tracking of multiple human movement, which solves the technical problems of long computation time and ID duplication and waste in existing technologies when performing multi-target tracking, thereby achieving the goal of reducing computation time and improving tracking accuracy.

[0011] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0012] A method for fast tracking of multi-person movement, characterized by the following steps:

[0013] A recognition model is established, and the acquired images are input into the recognition model;

[0014] The recognition model outputs the coordinates of the human body region, the confidence level of human body recognition, the coordinates of the face region, the image of the human body region, and the image of the face region.

[0015] The face region image output by the recognition model is compared with the face region image in the storage module.

[0016] If there is a mismatch, the storage module is updated, and the display module is updated synchronously based on the updated storage module.

[0017] If a match is found, the difference is determined based on the human body recognition credibility and human body region image output by the recognition model. The storage module is then updated based on the difference determination result, and the display module is updated synchronously based on the updated storage module.

[0018] In a preferred embodiment of the present invention, when updating the storage module after identifying a mismatch, the following steps are included:

[0019] The facial region feature values ​​are obtained from the facial region image output by the recognition model;

[0020] Human region feature values ​​are obtained from the human region image output by the recognition model;

[0021] The human body region coordinates, human body recognition confidence level, face region coordinates, face region feature values, and human body region feature values ​​output by the recognition model are added to the storage module to complete the update of the storage module.

[0022] In a preferred embodiment of the present invention, when performing difference judgment based on the human body recognition credibility output by the recognition model during identification and matching, the following steps are included:

[0023] Obtain the confidence difference between the human body recognition confidence level output by the recognition model and the human body recognition confidence level in the storage module, and determine whether the confidence difference exceeds a threshold.

[0024] If the value exceeds this limit, it is assumed that there is a large area of ​​human body obstruction, and the storage module is updated accordingly.

[0025] If the difference is not exceeded, then a difference judgment is made based on the human body region image output by the recognition model.

[0026] In a preferred embodiment of the present invention, when it is determined that a large area of ​​the human body is obstructed and the storage module is updated, the following steps are included:

[0027] The human body region coordinates and human body recognition confidence level in the storage module are updated based on the human body region coordinates and human body recognition confidence level output by the recognition model.

[0028] In a preferred embodiment of the present invention, when performing difference judgment based on the human body region image output by the recognition model, the following steps are included:

[0029] Obtain the IoU between the human region image output by the recognition model and the human region image in the storage module, and determine whether the IoU is lower than a threshold.

[0030] If the value is below the threshold, the human limb movement is considered to be large, and the storage module is updated.

[0031] If the value is not below the threshold, the human region feature value is obtained from the human region image output by the recognition model, the difference is judged based on the human region feature value, and the storage module is updated based on the difference judgment result.

[0032] In a preferred embodiment of the present invention, when the human limb movement is considered to be significant and the storage module is updated, the following steps are included:

[0033] The human body region coordinates and human body recognition confidence level in the storage module are updated based on the human body region coordinates and human body recognition confidence level output by the recognition model.

[0034] In a preferred embodiment of the present invention, the difference judgment based on the human body region feature values ​​includes:

[0035] Obtain the human feature difference between the human region feature value and the human region feature value in the storage module, and determine whether the human feature difference exceeds a threshold.

[0036] In a preferred embodiment of the present invention, updating the storage module based on the difference determination result includes:

[0037] If the difference in human body features exceeds the threshold, it is considered that the human body clothing has been updated, and the human body region feature values, human body region coordinates, and human body recognition confidence in the storage module are updated according to the human body region feature values, the human body region coordinates output by the recognition model, and the human body recognition confidence.

[0038] In a preferred embodiment of the present invention, updating the storage module based on the difference determination result includes:

[0039] If the difference in human body features does not exceed the threshold, it is considered that the human body is moving normally, and the human body feature values, human body coordinates, and human body recognition confidence in the storage module are updated according to the human body region feature values, the human body region coordinates output by the recognition model, and the human body recognition confidence.

[0040] A system for rapid tracking of multi-person movement includes:

[0041] Recognition Unit: Used to establish a recognition model, input the acquired image into the recognition model; output the human body region coordinates, human body recognition confidence level, face region coordinates, human body region image, and face region image through the recognition model;

[0042] Comparison unit: used to compare the face region image output by the recognition model with the face region image in the storage module;

[0043] Difference Judgment Unit: Used to make difference judgments based on the human body recognition credibility and human body region image output by the recognition model during identification and comparison matching.

[0044] Update unit: used to update the storage module when a comparison mismatch is identified, and to update the storage module according to the difference judgment result, and to synchronously update the display module according to the updated storage module.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] (1) Existing technologies identify by feature values, which has the technical problem of long computation time. In contrast, the present invention effectively reduces matching time by introducing a comparison of credibility.

[0047] (2) When the target area is occluded and the target is lost, the present invention can quickly match the target by comparing the credibility.

[0048] (3) When the target area is occluded for too long and the target is lost, the present invention can quickly match the target by comparing the facial recognition data.

[0049] (4) The present invention uses face recognition. Since the comparison area of ​​face recognition is small, compared with human body recognition feature values, the storage space is small and the calculation time is short.

[0050] (5) When the feature value matching error is large and the target is lost, the present invention can accurately track the target by comparing the features through face recognition.

[0051] (6) When the target is lost due to large movements of the person, the present invention can quickly and accurately track the target by comparing the confidence area.

[0052] (7) This invention uses the recognition of small facial images, along with a credibility algorithm and an IOU algorithm, to eliminate most scenarios such as occlusion by human movement and excessive limb movements, thereby saving a lot of feature value calculation time and achieving a fast tracking effect.

[0053] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0054] Figure 1 - This is a flowchart illustrating the steps of a method for rapid tracking of multiple human movements according to an embodiment of the present invention;

[0055] Figure 2 - is a logic diagram of the method for rapid tracking of multiple human movement according to an embodiment of the present invention. Detailed Implementation

[0056] The method for fast tracking of multi-person movement provided by this invention, such as Figure 1 As shown, it includes the following steps:

[0057] Step S1: Establish a recognition model and input the acquired images into the recognition model;

[0058] Step S2: Output the human body region coordinates, human body recognition confidence level, face region coordinates, human body region image, and face region image through the recognition model;

[0059] Step S3: Compare the face region image output by the recognition model with the face region image in the storage module;

[0060] Step S4: If there is a mismatch, update the storage module and update the display module synchronously based on the updated storage module;

[0061] Step S5: If a match is found, the difference is judged based on the human recognition credibility and human region image output by the recognition model, and the storage module is updated according to the difference judgment result. The display module is then updated synchronously according to the updated storage module.

[0062] In step S1 above, establishing a recognition model includes training the recognition model. The training process is as follows:

[0063] By segmenting the image into 64x64 pixels, labeling the foreground human body as 1 and the background as 0, and performing 3x3 convolution calculations, 82 layers of extraction, 94 layers of extraction, and 106 layers of extraction, feature dimension maps of 52x52x256, 26x26x512, and 13x13x1024 are formed. Based on the feature dimension maps, 10x10, 5x10, and 10x5 bounding boxes are formed. The results of coordinate pre-judgment and human body recognition confidence are output. The loss function designed by the difference between the human body recognition confidence and coordinate pre-judgment calculation results and the real human body recognition confidence and real coordinates is recursively calculated to generate the final recognition model.

[0064] In step S4 above, when identifying a mismatch and updating the storage module, the following is included:

[0065] The facial region feature values ​​are obtained from the facial region image output by the recognition model;

[0066] Human region feature values ​​are obtained from the human region image output by the recognition model;

[0067] The human body region coordinates, human body recognition confidence, face region coordinates, face region feature values, and human body region feature values ​​output by the recognition model are added to the storage module to complete the update of the storage module.

[0068] In step S5 above, during the identification and matching process, when making a difference judgment based on the human recognition credibility output by the identification model, the following steps are included:

[0069] Obtain the confidence difference between the human body recognition confidence score output by the recognition model and the human body recognition confidence score in the storage module, and determine whether the confidence difference exceeds the threshold.

[0070] If the value exceeds a certain threshold, it is assumed that there is a large area of ​​human body obstruction, and the storage module is updated accordingly.

[0071] If the difference is not exceeded, the difference is judged based on the human body region image output by the recognition model.

[0072] Specifically, to address the issue of discrepancies in human tracking feature values ​​caused by occlusion, this invention uses a decrease in the confidence level of human body recognition as a criterion. If the decrease exceeds a threshold, the person is considered rapidly occluded, and no new feature values ​​are included. This maintains feature value tracking and existing ID management, preventing ID duplication and waste, while ensuring accurate human tracking. Feature value acquisition is based on the convolutional superposition of 512 feature values ​​from a complete human image. Due to occlusion, feature values ​​for the head or other body parts may not be available, causing discrepancies. The recognition model uses the human body as the center for recognition, adding scores from limb recognition and finally head recognition. The more complete the part is displayed, the higher the score, with a maximum value of 1. If occlusion reaches 10%, the human body recognition confidence score drops to 0.9, confirming occlusion. If the human body recognition confidence score determines that an object is occluding the person, feature value extraction from the 512 feature points is not performed.

[0073] Regarding the issue of differences in feature values ​​for tracking people caused by occlusion, if the confidence level of human body recognition is within the threshold range, it proves that the occlusion is caused by a small object. If, in addition to facial recognition, it proves that the large difference is caused by dirt on the person's body or clothing, then the feature value of the person is recalculated to maintain feature value tracking and original ID management, prevent ID duplication and waste, and ensure accurate person tracking.

[0074] When a person moves out of the video frame and their clothing is updated, if the confidence level is high enough, facial recognition is performed, incorporating new feature values ​​while maintaining the tracking and management of existing IDs to prevent ID duplication and waste, and ensuring accurate person tracking. During person tracking, the recognition model returns a person region. If the width or height of this region is 0, it's determined that the person has moved out of the video frame. When the person updates their clothing, a confidence level value provided by the recognition model is used for assessment. If the confidence level value exceeds 0.9, facial recognition is performed, incorporating new feature values ​​while maintaining the tracking and management of existing IDs to prevent ID duplication and waste, and ensuring accurate person tracking.

[0075] When a person disappears from the video area for an extended period, the system saves facial feature values, allowing for long-term tracking with minimal storage. If the person subsequently reappears in the video area, tracking can resume, improving accuracy with minimal storage. During person tracking, the recognition module returns to the human body region. If the width or height of this region equals 0 for more than 60 seconds, the person is considered to have disappeared from the video area for an extended period. Because facial feature values ​​are saved, long-term tracking with minimal storage is possible. If the person subsequently reappears in the video area, tracking can resume, improving accuracy with minimal storage.

[0076] Furthermore, when it is believed that a large area of ​​the human body is obstructed and the storage module is updated, this includes:

[0077] The human body region coordinates and human body recognition confidence scores in the storage module are updated based on the human body region coordinates and human body recognition confidence scores output by the recognition model.

[0078] Furthermore, when making difference judgments based on the human body region images output by the recognition model, the following steps are included:

[0079] Obtain the IoU between the human region image output by the recognition model and the human region image in the storage module, and determine whether the IoU is lower than the threshold.

[0080] If the value is below the threshold, the human limb movement is considered too large, and the storage module is updated.

[0081] If the value is not below the threshold, the human region feature value is obtained from the human region image output by the recognition model, the difference is judged based on the human region feature value, and the storage module is updated based on the difference judgment result.

[0082] Specifically, when a person's body movements are too large, resulting in excessive background being included in feature value calculations, a high-confidence threshold is used for judgment. IoU interval calculation is then performed, and the current feature value calculation is abandoned to ensure fast person tracking while maintaining the original ID management, ensuring fast and accurate person tracking. When a person's body movements are too large, since feature value extraction involves convolutional extraction of the entire person's image region, the larger the image region, the greater the noise error, leading to excessive background being included in feature value calculations. Therefore, when the confidence level exceeds 0.9, a judgment is made, and the IoU ratio of the intersection and union of the current person's image region size and the person's image region size in the previous frame is calculated. If the IoU calculation is below 0.5, it indicates that the person's movement amplitude is twice that of the previous frame, and the current feature value calculation is abandoned to ensure fast person tracking while maintaining the original ID management, ensuring fast and accurate person tracking.

[0083] Furthermore, when a large human limb movement is considered to occur and the storage module is updated, this includes:

[0084] The human body region coordinates and human body recognition confidence scores in the storage module are updated based on the human body region coordinates and human body recognition confidence scores output by the recognition model.

[0085] Furthermore, when making difference judgments based on human body region feature values, this includes:

[0086] The difference between the human body region feature value and the human body region feature value in the storage module is obtained, and it is determined whether the difference exceeds the threshold.

[0087] Furthermore, when updating the storage module based on the difference judgment result, it includes:

[0088] If the difference in human body features exceeds the threshold, it is considered that the human body clothing has been updated, and the human body region feature values, human body region coordinates, and human body recognition credibility in the storage module are updated based on the human body region feature values, the human body region coordinates output by the recognition model, and the human body recognition credibility.

[0089] Furthermore, when updating the storage module based on the difference judgment result, it includes:

[0090] If the difference in human body features does not exceed the threshold, it is considered that the human body is moving normally, and the human body feature values, human body coordinates, and human body recognition credibility in the storage module are updated based on the human body region feature values, the human body region coordinates output by the recognition model, and the human body recognition credibility.

[0091] like Figure 2 As shown, the principle of this invention is as follows:

[0092] By recognizing and comparing small facial images, this invention saves significant computation time. Furthermore, through confidence-based algorithms and Intersection over Union (IoU) algorithms, it eliminates many redundant scenarios, enabling rapid tracking and localization. Feature value calculation is only performed in the final step, further reducing computation time. This invention replaces the calculation of human body feature values ​​with facial recognition, improving accuracy while reducing computational load because the facial recognition image is smaller than the human body image, thus greatly reducing computational complexity. Secondly, it uses human confidence-based recognition to determine occlusion in the image and region IoU for rapid person identification, preventing miscalculation of feature values ​​and improving both accuracy and speed. Finally, it uses feature value differences to update the status of multiple individuals. Compared to existing methods, this invention significantly reduces computation time and improves tracking accuracy. This invention not only achieves accurate and rapid multi-target tracking but also displays abnormal appearances of individuals through feature value difference calculation, providing a foundation for the development of other application scenarios.

[0093] The system for rapid tracking of multiple human movement provided by the present invention includes: an identification unit, a comparison unit, a difference judgment unit, and an update unit.

[0094] The recognition unit is used to establish a recognition model, inputting the acquired images into the recognition model; and outputting the human body region coordinates, human body recognition confidence, face region coordinates, human body region image, and face region image through the recognition model.

[0095] The comparison unit is used to compare the face region image output by the recognition model with the face region image in the storage module.

[0096] When the difference judgment unit is used for recognition and comparison matching, it makes difference judgments based on the human body recognition credibility and human body region image output by the recognition model.

[0097] The update unit is used to update the storage module when a comparison mismatch is identified, and to update the storage module based on the difference judgment result, and to synchronously update the display module based on the updated storage module.

[0098] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0099] (1) Existing technologies identify by feature values, which has the technical problem of long computation time. In contrast, the present invention effectively reduces matching time by introducing a comparison of credibility.

[0100] (2) When the target area is occluded and the target is lost, the present invention can quickly match the target by comparing the credibility.

[0101] (3) When the target area is occluded for too long and the target is lost, the present invention can quickly match the target by comparing the facial recognition data.

[0102] (4) The present invention uses face recognition. Since the comparison area of ​​face recognition is small, compared with human body recognition feature values, the storage space is small and the calculation time is short.

[0103] (5) When the feature value matching error is large and the target is lost, the present invention can accurately track the target by comparing the features through face recognition.

[0104] (6) When the target is lost due to large movements of the person, the present invention can quickly and accurately track the target by comparing the confidence area.

[0105] (7) This invention uses the recognition of small facial images, along with a credibility algorithm and an IOU algorithm, to eliminate most scenarios such as occlusion by human movement and excessive limb movements, thereby saving a lot of feature value calculation time and achieving a fast tracking effect.

[0106] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A method for fast tracking of multi-person movement, characterized in that, Includes the following steps: A recognition model is established, and the acquired images are input into the recognition model; The recognition model outputs the coordinates of the human body region, the confidence level of human body recognition, the coordinates of the face region, the image of the human body region, and the image of the face region. The face region image output by the recognition model is compared with the face region image in the storage module. If there is a mismatch, the storage module is updated, and the display module is updated synchronously based on the updated storage module. If a match is found, the difference is determined based on the human recognition credibility and human region image output by the recognition model. The storage module is then updated based on the difference determination result, and the display module is updated synchronously based on the updated storage module. Specifically, in the identification and matching process, when making a difference judgment based on the human recognition credibility output by the identification model, the following are included: Obtain the confidence difference between the human body recognition confidence level output by the recognition model and the human body recognition confidence level in the storage module, and determine whether the confidence difference exceeds a threshold. If the value exceeds the limit, it is considered that there is a large area of ​​human body obstruction. In this case, no new feature value will be included, feature value tracking and original ID management will be maintained, and the storage module will be updated. If the threshold is not exceeded, the feature value of the person is recalculated to maintain feature value tracking and the original ID management, and the difference is judged based on the human body region image output by the recognition model.

2. The method for fast tracking of multi-person movement according to claim 1, characterized in that, When updating the storage module after identifying a mismatch, the process includes: The facial region feature values ​​are obtained from the facial region image output by the recognition model; Human region feature values ​​are obtained from the human region image output by the recognition model; The human body region coordinates, human body recognition confidence level, face region coordinates, face region feature values, and human body region feature values ​​output by the recognition model are added to the storage module to complete the update of the storage module.

3. The method for fast tracking of multi-person movement according to claim 1, characterized in that, When it is determined that a large area of ​​the human body is obstructed and the storage module is updated, the following steps are included: The human body region coordinates and human body recognition confidence level in the storage module are updated based on the human body region coordinates and human body recognition confidence level output by the recognition model.

4. The method for fast tracking of multi-person movement according to claim 1, characterized in that, When performing difference judgment based on the human body region image output by the recognition model, the following are included: Obtain the IoU between the human region image output by the recognition model and the human region image in the storage module, and determine whether the IoU is lower than a threshold. If the value is below the threshold, the human limb movement is considered to be large, and the storage module is updated. If the value is not below the threshold, the human region feature value is obtained from the human region image output by the recognition model, the difference is judged based on the human region feature value, and the storage module is updated based on the difference judgment result.

5. The method for rapid tracking of multi-person movement according to claim 4, characterized in that, When a large human limb movement is considered to be occurring and the storage module is updated, the following steps are included: The human body region coordinates and human body recognition confidence level in the storage module are updated based on the human body region coordinates and human body recognition confidence level output by the recognition model.

6. The method for fast tracking of multi-person movement according to claim 4, characterized in that, When making difference judgments based on the feature values ​​of the human body region, the following are included: Obtain the human feature difference between the human region feature value and the human region feature value in the storage module, and determine whether the human feature difference exceeds a threshold.

7. The method for fast tracking of multi-person movement according to claim 6, characterized in that, When updating the storage module based on the difference determination result, the following steps are included: If the difference in human body features exceeds the threshold, it is considered that the human body clothing has been updated, and the human body region feature values, human body region coordinates, and human body recognition confidence in the storage module are updated according to the human body region feature values, the human body region coordinates output by the recognition model, and the human body recognition confidence.

8. The method for fast tracking of multi-person movement according to claim 6, characterized in that, When updating the storage module based on the difference determination result, the following steps are included: If the difference in human body features does not exceed the threshold, it is considered that the human body is moving normally, and the human body feature values, human body coordinates, and human body recognition confidence in the storage module are updated according to the human body region feature values, the human body region coordinates output by the recognition model, and the human body recognition confidence.

9. A system for implementing the method for rapid tracking of multi-person movement according to any one of claims 1 to 8, characterized in that, include: Recognition unit: used to establish a recognition model and input the acquired image into the recognition model; The recognition model outputs the coordinates of the human body region, the confidence level of human body recognition, the coordinates of the face region, the image of the human body region, and the image of the face region. Comparison unit: used to compare the face region image output by the recognition model with the face region image in the storage module; Difference Judgment Unit: Used to make difference judgments based on the human body recognition credibility and human body region image output by the recognition model during identification and comparison matching. Update unit: used to update the storage module when a comparison mismatch is identified, and to update the storage module according to the difference judgment result, and to synchronously update the display module according to the updated storage module.

Images