A data processing method, device, apparatus, and computer-readable storage medium

Abstract

Embodiments of the present application disclose a data processing method, device and equipment and a computer readable storage medium. The method comprises: obtaining an image frame corresponding to a video file, performing target detection on the image frame to obtain a target detection result; performing tracking processing on a detection object based on the target detection result to determine motion trajectory information of the detection object; determining a target object passing through a reference line segment during a motion based on the motion trajectory information; determining a motion direction of the target object when passing through the reference line segment based on the motion trajectory information of the target object and position information of the reference line segment, and performing data statistics to obtain target statistical data. In this way, the motion direction of the target object when passing through the reference line is determined based on the motion trajectory of the target object and the position information of the reference line segment, and counting is performed according to the motion direction, which can solve the problem of counting errors caused by the target object moving back and forth near the reference line segment, thereby improving the counting accuracy of the target object.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Currently, with the continuous development of animal husbandry, intelligent and refined breeding methods are gradually becoming the mainstream. These methods have driven the development of intelligent inventory and counting. However, due to the high activity and large number of livestock and poultry, multiple animals may obstruct each other's view, resulting in problems such as low counting efficiency and difficulty in counting.

[0003] In related technologies, livestock and poultry target detection and counting methods mainly include target recognition and counting methods based on features such as color and texture, image density map extraction and estimation methods, and sensor signal acquisition and analysis methods based on the boundary between livestock and poultry targets and intrusion areas. Target recognition and counting methods based on features such as color and texture, and image density map extraction and estimation methods, both have certain limitations in image acquisition, and the static counting accuracy error is relatively high when the target moves. Sensor signal acquisition and analysis methods based on the boundary between livestock and poultry targets and intrusion areas rely on sensor-acquired data, resulting in lower counting accuracy. Summary of the Invention

[0004] To address the aforementioned technical problems, this application aims to provide a data method that can improve the counting accuracy of target objects.

[0005] The technical solution of the application embodiment is implemented as follows:

[0006] This application provides a data processing method, including:

[0007] Obtain at least one image frame corresponding to the video file, perform target detection on the at least one image frame, and obtain the target detection result;

[0008] Based on the target detection results, each detected object is tracked and processed to determine the motion trajectory information of each detected object;

[0009] Based on the motion trajectory information of each detected object, the target object that passes through the reference line segment during the motion is determined from multiple detected objects. The reference line segment is located in the middle position of each image frame.

[0010] The motion direction of each target object when passing through the reference line segment is determined based on the motion trajectory information of each target object and the position information of the reference line segment.

[0011] Target statistics are obtained by statistically analyzing the movement direction of each target object as it passes through the reference line segment.

[0012] This application provides a data processing apparatus, including:

[0013] The first acquisition module acquires at least one image frame corresponding to the video file, performs target detection on the at least one image frame, and obtains the target detection result.

[0014] The first determining module is used to track and process each detected object based on the target detection results, and determine the motion trajectory information of each detected object;

[0015] The second determining module is used to determine, based on the motion trajectory information of each detected object, the target object that passes through the reference line segment during the motion process from among multiple detected objects, wherein the reference line segment is located in the middle position of each image frame;

[0016] The third determining module is used to determine the direction of motion of each target object when it passes through the reference line segment based on the motion trajectory information of each target object and the position information of the reference line segment.

[0017] The second acquisition module is used to perform data statistics based on the movement direction of each target object when it passes through the reference line segment, and obtain target statistical data.

[0018] This application provides a data processing device, including:

[0019] Memory, used to store executable data processing instructions;

[0020] The processor, when executing executable data processing instructions stored in the memory, implements the data processing method provided in the embodiments of this application.

[0021] This application provides a computer-readable storage medium storing computer-executable data processing instructions configured to execute the data processing method provided in this application.

[0022] This application provides a data processing method, apparatus, device, and computer-readable storage medium. Using this technical solution, firstly, at least one image frame corresponding to a video file is acquired, and target detection is performed on each image frame to obtain target detection results. Then, based on the target detection results, tracking processing is performed on each detected object to determine the motion trajectory information of each detected object. Based on the motion trajectory information of each detected object, target objects that pass through a reference line segment during their movement are identified from among multiple detected objects. Next, based on the motion trajectory information of each target object and the position information of the reference line segment, the movement direction of each target object when passing through the reference line segment is determined. Finally, data statistics are performed based on the movement direction of each target object when passing through the reference line segment to obtain target statistical data. Thus, by determining the movement direction of the target object when passing through the reference line segment based on the target object's motion trajectory and the position information of the reference line segment, and counting based on this movement direction, the problem of counting errors caused by the target object moving back and forth near the reference line segment can be solved, thereby improving the counting accuracy of target objects. Attached Figure Description

[0023] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0024] Figure 2 A flowchart illustrating a dynamic target tracking and counting method provided in an embodiment of this application;

[0025] Figure 3 A flowchart illustrating another dynamic target tracking and counting method provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of the composition structure of a data processing device provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the composition structure of a data processing device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] In the following description, references to “some embodiments” or “other embodiments” describe a subset of all possible embodiments. However, it is understood that “some embodiments” or “other embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0031] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0032] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this application is for the purpose of describing embodiments of this application only and is not intended to be limiting of this application.

[0033] In view of the problems in related technologies, this application provides a data processing method that can improve the counting accuracy of target objects.

[0034] The data processing method provided in the embodiments of this application will be described below, such as... Figure 1 The diagram shown is a flowchart of a data processing method provided in an embodiment of this application. The method includes the following steps:

[0035] S101. Obtain at least one image frame corresponding to the video file, perform target detection on at least one image frame, and obtain the target detection result.

[0036] It should be noted that video files can be obtained from a database or captured in real time by image acquisition devices such as cameras. At least one image frame corresponding to a video file can be obtained by extracting image frames from the video file. In practice, image frame extraction can be performed on the video file at a preset frame rate; different frame rates will result in different numbers of image frames obtained.

[0037] In some embodiments, when performing target detection on each image frame, target detection can be performed on each image frame separately. In some embodiments, the YOLOv5 algorithm can be used to perform target detection on the image frames to obtain target detection results. The target detection results may include the detected objects in the image frames, as well as information such as the positions of each detected object. In practice, since some image frames may not contain detected objects, during the process of performing target detection on multiple target image frames, the detected targets may only be detected in a portion of the image frames.

[0038] S102. Based on the target detection results, track and process each detected object to determine the motion trajectory information of each detected object.

[0039] It should be noted that the detected object can be an object in an image frame. For example, if a video file is about the activity of livestock and poultry in a certain area, the detected object can be the livestock and poultry in the corresponding image frame of the video file, such as pigs, cattle, and sheep. The motion trajectory information of the detected object can be the activity trajectory of each detected object, including at least one position of the detected object during the activity, the coordinate information of each position, and the complete motion route of the detected object.

[0040] In some embodiments, tracking processing of each detected target can be performed by target tracking. In practice, the Deep-Sort algorithm can be used to track each target in order to determine the motion trajectory information of each target.

[0041] S103. Based on the motion trajectory information of each detected object, determine the target object that passes through the reference line segment during the motion process from multiple detected objects.

[0042] In some embodiments, the reference line segment can be a pre-set virtual line segment. The reference line segment can be used to count the number of target objects. The reference line segment can be located in the middle of each image frame. When the reference line segment is located in the middle of each image frame, it can be set in the horizontal or vertical direction within the image frame, and the position of the reference line segment is the same in each image frame. In some embodiments, after obtaining the motion trajectory information of each detected object, the detected objects that have passed through the reference line segment during motion can be further identified, i.e., the target objects.

[0043] In some embodiments, multiple detected objects may all pass through the reference line segment during their movement, or only some of the detected objects may pass through the reference line segment, or none of them may pass through the reference line segment. The movement trajectory of the detected objects can be used to determine whether the target objects have passed through the reference line segment. In practice, whether the detected objects have passed through the reference line segment during their movement can be determined by whether their movement trajectory passes through the reference line segment.

[0044] S104. Determine the direction of motion of each target object when it passes through the reference line segment based on the motion trajectory information of each target object and the position information of the reference line segment.

[0045] In some embodiments, after determining that a target object can pass through a reference line segment, the direction of motion of the target object when passing through the reference line segment can be determined based on the motion trajectory information of the target object and the position information of the reference line segment. For example, if the reference line segment is located in the middle of an image frame and is positioned from the top to the bottom of the image frame, and the target object passes through the reference line segment during its movement, then the direction of motion of the target object when passing through the reference line segment can be from the left side of the reference line segment to the right side of the reference line segment, or from the right side of the reference line segment to the left side of the reference line segment.

[0046] S105. Based on the movement direction of each target object when passing through the reference line segment, perform data statistics to obtain target statistical data.

[0047] It should be noted that the target statistics can be the sum of the quantities of all target objects. For example, it can be the total number of a type of livestock, such as the total number of pigs, or it can be the total number of various types of livestock, such as the total number of pigs, cattle, sheep, etc. In some embodiments, after obtaining the movement direction of each target object when passing through the reference line segment, the data of the target objects can be statistically analyzed to obtain the target statistics.

[0048] In this embodiment, firstly, at least one image frame corresponding to the video file is acquired, and target detection is performed on each image frame to obtain target detection results. Then, based on the target detection results, tracking processing is performed on each detected object to determine the motion trajectory information of each detected object. Based on the motion trajectory information of each detected object, target objects that pass through reference line segments during their movement are identified from multiple detected objects. Next, based on the motion trajectory information of each target object and the position information of the reference line segment, the motion direction of each target object when passing through the reference line segment is determined. Finally, data statistics are performed based on the motion direction of each target object when passing through the reference line segment to obtain target statistical data. Thus, by determining the motion direction of the target object when passing through the reference line based on the motion trajectory of the target object and the position information of the reference line segment, and counting based on this motion direction, the problem of counting errors caused by the target object moving back and forth near the reference line segment can be solved, thereby improving the counting accuracy of target objects.

[0049] In some embodiments of this application, the direction of motion of each target object when passing through the reference line segment is determined based on the motion trajectory information of each target object and the position information of the reference line segment. That is, step S104 can also be implemented by the following steps S1041 to S1043. Each step is described in detail below.

[0050] S1041. Based on the motion trajectory information of each target object, determine the first coordinate of the first position and the second coordinate of the second position of the target object on both sides of the reference line segment during the motion process.

[0051] In some embodiments, based on the motion trajectory information of each target object, the first position and the second position of each target object located on both sides of the reference line segment during the motion process can be determined, and the first coordinate of the first position and the second coordinate of the second position can be obtained.

[0052] S1042. Determine the third coordinate of the first endpoint and the fourth coordinate of the second endpoint of the reference line segment based on the position information of the reference line segment.

[0053] In some embodiments, the reference line segment includes a first endpoint and a second endpoint. When the reference line segment is horizontally positioned, the first endpoint and the second endpoint can be the left endpoint and the right endpoint, or the right endpoint and the left endpoint, respectively. When the reference line segment is vertically positioned, the first endpoint and the second endpoint can be the upper endpoint and the lower endpoint, or the lower endpoint and the upper endpoint. After obtaining the position information of the reference line segment, the third coordinate of the first endpoint and the fourth coordinate of the second endpoint can be determined.

[0054] S1043. Determine the reference vector based on the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate.

[0055] In some embodiments, after obtaining the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate, a reference vector can be further determined based on the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate.

[0056] S1044. Determine the direction of motion of each target object when it passes through the reference line segment based on the reference vector.

[0057] In some embodiments, after the reference vector is determined, the direction of motion of each target object when passing through the reference line segment can be determined based on the reference vector. In some embodiments, the direction of motion of the target object when passing through the reference line segment can be determined based on the direction of the reference vector.

[0058] In some embodiments of this application, the reference vector includes a first vector, a second vector, a third vector, and a fourth vector. Based on this, the reference vector is determined based on the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate. That is, step S1042 can also be implemented by the following steps S201 and S202.

[0059] S201. Determine the first vector based on the first coordinate and the third coordinate; determine the second vector based on the second coordinate and the third coordinate.

[0060] In some embodiments, a first vector can be determined based on a first coordinate and a second coordinate, and a second vector can be determined based on a second coordinate and a third coordinate. For example, if the first coordinate is (x1, y1), the second coordinate is (x2, y2), the third coordinate is (x3, y3), and the fourth coordinate is (x4, y4), then the first vector... The size can be represented as (x2-x1, y2-y1), the second vector The size can be represented as (x4-x3, y4-y3).

[0061] S202. Determine the third vector based on the first and fourth coordinates; determine the fourth vector based on the second and fourth coordinates.

[0062] In some embodiments, similar to the determination of the first and second vectors described above, the third vector can be determined based on the first and fourth coordinates, and the fourth vector can be determined based on the second and fourth coordinates. If the first coordinate is (x1, y1), the second coordinate is (x2, y2), the third coordinate is (x3, y3), and the fourth coordinate is (x4, y4), then the third vector... The size can be represented as (x4-x1, y4-y1), the fourth vector The size can be represented as (x⁴-x², y⁴-y²).

[0063] In some embodiments of this application, the direction of motion of each target object when passing through the reference line segment is determined based on the reference vector, that is, step S1044 can also be implemented by the following steps S301 to S304.

[0064] S301. Determine the cross product of the first vector and the third vector as the first vector product.

[0065] In some embodiments, after determining the first vector and the third vector, the outer product of the first vector and the third vector can be used as the product of the first vector. For example, if the first vector is... The third vector is Then the first vector product It can be represented as

[0066] S302. When the first vector to the third vector is determined to be a counterclockwise rotation direction based on the first vector product, the motion direction of each target object is determined to be from the first side to the second side of the reference line segment.

[0067] In some embodiments, since the first vector product is obtained by the cross product of the first vector and the third vector, the direction of the first vector product is perpendicular to the direction of the first vector and also perpendicular to the direction of the third vector. When determining the direction of the first vector product, the rotation direction from the first vector to the third vector can be determined according to the right-hand rule. When the rotation direction from the first vector to the third vector is determined to be counterclockwise, the movement direction of each target object can be determined from the first side to the second side of the reference line segment.

[0068] S303. Determine the outer product of the second vector and the fourth vector as the second vector product.

[0069] In some embodiments, after determining the second vector and the fourth vector, the outer product of the second vector and the fourth vector can be used as the second vector product. For example, if the second vector is... The fourth vector is Then the second vector product can be used It can be represented as

[0070] S304. When the second vector to the fourth vector is determined to be a clockwise rotation direction based on the product of the second vectors, the motion direction of each target object is determined to be from the second side to the first side of the reference line segment.

[0071] In some embodiments, since the first vector product is obtained by the cross product of the first vector and the third vector, the direction of the first vector product is perpendicular to the direction of the first vector and also perpendicular to the direction of the third vector. When determining the direction of the first vector product, the rotation direction from the first vector to the third vector can be determined according to the right-hand rule. When the rotation direction from the first vector to the third vector is determined to be counterclockwise, the movement direction of each target object can be determined from the first side to the second side of the reference line segment.

[0072] In some embodiments of this application, data statistics are performed based on the movement direction of each target object when passing through the reference line segment to obtain target statistical data. That is, step S105 can also be implemented by the following steps S1051 to S1055. Each step is described in detail below.

[0073] S1051. Obtain the first and second count values ​​corresponding to each target object.

[0074] It should be noted that, initially, both the first and second count values ​​can be set to zero. During the counting and statistics process for each target object, the first and second count values ​​may change. The current first and second count values ​​for each target object can be the initial values ​​or other values ​​obtained during the counting process.

[0075] S1052. When the target object moves from the first side to the second side of the reference line segment, update the first count value to obtain the updated first count value.

[0076] In some embodiments, the first side may be the left side of the reference line segment and the second side may be the right side of the reference line segment; in other embodiments, the first side may be the upper side of the reference line segment and the second side may be the lower side of the reference line segment. Of course, the first side and the second side of the reference line segment are merely illustrative examples and are not intended to limit the scope of this application.

[0077] In some embodiments, updating the first count value can be done by incrementing the first count value by one. For example, if the current first count value corresponding to the target object is 4, and the analysis of the target object's motion trajectory determines that the target object moves from the left side of the reference line segment to the right side of the reference line segment, then the first count value is incremented by one, resulting in an updated first count value of 5.

[0078] S1053. When the target object moves from the second side to the first side of the reference line segment, update the second count value to obtain the updated second count value.

[0079] In some embodiments, updating the second count value can be done by incrementing the second count value by one. For example, if the initial value of the second count value corresponding to the target object is 0, then when analyzing the motion trajectory of the target object, if it is determined that the target object has moved from the right side of the reference line segment to the left side of the reference line segment, the second count value is incremented by one, resulting in an updated second count value of 1.

[0080] S1054. Determine the first statistical result corresponding to the updated first count value and the second statistical result corresponding to the updated second count value.

[0081] In some embodiments, after obtaining the updated first count value and the updated second count value, a first statistical result corresponding to the updated first count value and a second statistical result corresponding to the updated second count value can be obtained. The first count result can represent the total number of times each target object crosses the reference line segment from the first side to the second side, and the second count result can represent the total number of times each target object crosses the reference line segment from the second side to the first side.

[0082] S1055. Determine the absolute value of the difference between the first statistical result and the second statistical result, and use the absolute value of the difference as the target statistical data.

[0083] In one embodiment, the target statistical data can represent the target object that passes through the reference line segment among multiple target objects. For example, if the first statistical result is 15 and the second statistical result is 10, then the absolute value of the difference between the first statistical result and the second statistical result can be determined to be 5, that is, the target statistical data is determined to be 5.

[0084] It is understood that in the embodiments of this application, the first and second counting results corresponding to the target object are determined based on whether the target object passes through the reference line segment from the first side to the second side or from the second side to the first side. Based on the first and second counting results, the target statistical data corresponding to the target object is determined. This method can adapt to the situation where the target object moves back and forth, and to a certain extent reduces the counting error caused by the target object moving near the reference line segment, thereby improving the counting accuracy of the target object.

[0085] In some embodiments of this application, the target detection result also includes the detection position information of at least one detected object. Based on this, each detected object is tracked based on the target detection result to determine the motion trajectory information of each detected object. That is, step S102 can also be implemented by the following steps S1021 to S1025. The following describes each step in detail.

[0086] S1021. Obtain the first detection position information corresponding to the i-th detection object in the current image frame and the second detection position information of each detection object in the next image frame.

[0087] It should be noted that at least one image frame in a video file includes one or more. When there are multiple image frames, they are arranged sequentially from front to back according to their corresponding timestamps. The current image frame can be the first image frame, the Nth image frame, or any image frame between the first and Nth image frames, where N represents the total number of image frames. For example, if the current image frame is the first image frame, then the next image frame corresponding to the current image frame can be the second image frame. The i-th detection object in the current image frame can be any detection object in the current image frame. For example, if the current image frame has M detection objects, where M is a positive integer greater than zero, then i = 1, 2, ..., M.

[0088] In some embodiments, after performing target detection on the current image frame and the next image frame corresponding to the current image frame, the first position information corresponding to the i-th detected object in the current image frame and the second position information of each detected object in the next image frame corresponding to the current image frame can be obtained. In some embodiments, the first position may include the position coordinates of the i-th detected object in the current image frame, and the second position information may include the position coordinates of each detected object in the next image frame corresponding to the current image frame.

[0089] S1022. Based on the first detection position information, predict the position of the i-th detection object in the next image frame and obtain the predicted position information of the i-th detection object.

[0090] In some embodiments, the Kalman filter method can be used to predict the position of the i-th detected object in the next image frame. In practice, the first detection position information may include the center point coordinates (u,v) of the detection box corresponding to the i-th detected object, the aspect ratio r of the detection box, and the height h of the detection box, and an eight-dimensional state vector can be constructed. in, Let u, v, r, and h represent the derivatives, respectively. The predicted position information x' of the i-th object in the next image frame can be obtained according to the following formula (1):

[0091] x' = Fx (1);

[0092] Where x is the state vector corresponding to the i-th detected object in the current image frame. x is the predicted state vector corresponding to the i-th detected object in the next image frame, and F is the state transition matrix. The state transition matrix F can be set according to the following formula (2):

[0093]

[0094] S1023. Determine the Mahalanobis distances between the predicted location information and each of the second detection location information, and determine the minimum Mahalanobis distance among the various Mahalanobis distances.

[0095] In some embodiments, after obtaining the predicted position information of the i-th image in the next image frame, the Mahalanobis distance can be used to measure the distance between the predicted position information and each of the second detection position information. In practice, the Mahalanobis distance can be calculated according to formula (3).

[0096] d (1) (i,j)=(d j -y i )S i -1 (d i -y i (3);

[0097] Where, d j For the j-th (j=1,2,...,K) second detection location information, where K is the number of second detection location information, y i This provides the predicted location information for the i-th (i = 1, 2, ..., M) detected object, where M is the number of detected objects in the current image frame, and S... i -1 Let be the covariance between the second detection location information d and the predicted location information y. In some embodiments, after obtaining the predicted location information and each of the Mahalanobis distances between the respective second detection location information, the minimum Mahalanobis distance among a plurality of Mahalanobis distances can be determined.

[0098] S1024. Determine that the minimum Mahalanobis distance is less than the first preset distance threshold, and determine the candidate object of the i-th detection object from each detection object in the next image frame.

[0099] It should be noted that the first preset distance threshold can be a pre-set distance threshold. When the minimum Mahalanobis distance is less than the first preset distance threshold, it can be determined that the difference between the second position information and the predicted position information is small, and the candidate object can be the detection object in the next image frame corresponding to the minimum Mahalanobis distance.

[0100] In some embodiments, if it is determined that the minimum Mahalanobis distance is less than a first preset distance threshold, the detected object in the next image frame corresponding to the minimum Mahalanobis distance can be determined as a candidate object of the i-th detected object in the current image frame.

[0101] S1025. When the shape of the i-th detected object and the candidate object meets the matching condition, it is determined that the i-th detected object and the candidate object have reached the target matching condition. Based on the first detection position information and the second detection position information of the candidate object, the motion trajectory information of the i-th detected object is determined.

[0102] In some embodiments, the shape matching condition between the i-th detected object and the candidate object can be that the appearance information of the i-th detected object and the appearance information of the candidate object meet the matching condition, and the appearance information can be the feature information corresponding to the detected object and the candidate object. In some embodiments, when the shape matching condition between the i-th detected object and the candidate object is met, it can be determined that the i-th detected object and the candidate object have reached the target matching condition, that is, the i-th detected object and the candidate object have completed the matching. Then, the motion trajectory information of the i-th detected object can be determined based on the first detection position information of the i-th detected object and the second detection position information of the candidate object. This motion trajectory information can represent that the i-th detected object moves from the first detection position to the second detection position.

[0103] In some embodiments of this application, when the shape of the i-th detection object and the candidate object meets the matching condition, it is determined that the i-th detection object and the candidate object have reached the target matching condition. Based on the first detection position information and the second detection position information of the candidate object, the motion trajectory information of the i-th detection object is determined. That is, before step S1025, the following steps S301 to S303 can also be executed.

[0104] S301. Use the trained feature extraction model to extract features from the candidate objects and obtain the first feature vector.

[0105] In some embodiments, the trained feature extraction model can be a pedestrian re-identification (Re-ID) model. After determining the candidate objects, the trained Re-ID model can be used to extract features from the candidate objects, obtain the feature information of the candidate objects, and obtain the first feature vector composed of the feature information of the candidate objects.

[0106] S302. When the current image frame is the first frame, the trained feature extraction model is used to extract features from the i-th detected object in the current image frame to obtain the second feature vector.

[0107] In some embodiments, if the current image frame is the first image frame among multiple image frames, the trained feature extraction model can be used to extract features from the i-th detected object in the current image frame, obtain the feature information of the i-th detected object in the current image frame, and obtain a second feature vector composed of the feature information of the i-th detected object in the current image frame.

[0108] S303. Determine the cosine distance between the first feature vector and the second feature vector. When the cosine distance is less than the second preset distance threshold, determine that the shape of the i-th detection object and the candidate object meet the matching condition.

[0109] In some embodiments, the cosine distance C between the first eigenvector A and the second eigenvector B can be calculated using formula (4):

[0110]

[0111] Where ||·||2 represents the L2 norm. After determining the cosine distance, if the cosine distance is less than the second preset distance threshold, it can be determined that the shape of the i-th detection object and the candidate object meet the matching condition.

[0112] In some other embodiments of this application, after using a trained feature extraction model to extract features from candidate objects and obtain a first feature vector, i.e. after executing step S301, the following steps S401 to S403 can also be executed.

[0113] S401. When the current image frame is not the first image frame, determine each matching object from the historical frames before the current image frame that meets the target matching condition with the i-th detected object of the current image frame.

[0114] It should be noted that the historical frames preceding the current image frame can be other image frames that precede the current image frame after sorting them by timestamp. For example, if the current image frame is the fourth image frame, then the historical frames preceding the current image frame can include the first, second, and third image frames. The matching object can be a detection object in the historical frames that meets the target matching condition with the i-th detection object in the current image frame.

[0115] In some embodiments, if the current image frame is not the first image frame among multiple image frames, for example, the current image frame may be the second image frame, the third image frame, etc., then it is necessary to determine the matching object that meets the target matching condition with the i-th detected object of the current image frame from each historical frame before the current image frame.

[0116] S402. Obtain the third feature vector of each matching object corresponding to the i-th detection object.

[0117] In some embodiments, the matching objects corresponding to the i-th detected object may include multiple objects. In practice, the features of each matching object can also be extracted based on a trained feature extraction model, such as a trained Re-ID model, to obtain the feature information corresponding to each matching object, that is, the third feature vector composed of the feature information corresponding to each matching object.

[0118] S403. Determine the cosine distances between the first eigenvector and each of the third eigenvectors, and determine the minimum cosine distance among all the cosine distances.

[0119] In some embodiments, the cosine distance between the first eigenvector and the third eigenvector can be calculated using the following formula (5):

[0120] d (2) (i,j)=1-r i T r k j (5);

[0121] Where, r i Let r represent the first feature vector corresponding to the candidate object of the i-th detection object. k i Let represent the third feature vector corresponding to the j-th matching object, where j = 1, 2, ..., k, and k represents the number of matching objects. In practice, since there may be multiple matching objects, there will also be multiple third feature vectors corresponding to each matching object. Therefore, after obtaining the cosine distances between each third feature vector and the first feature vector, we can further determine the minimum value among the cosine distances and define the minimum cosine distance as the minimum cosine distance.

[0122] S404. When the minimum cosine distance is less than the second distance preset threshold, it is determined that the shape of the i-th detection object and the candidate object meet the matching condition.

[0123] In some embodiments, after determining the minimum cosine distance, if it is determined that the minimum cosine distance is less than a second preset distance threshold, it can be determined that the appearance of the i-th detected object and the candidate object meet the matching condition, that is, the appearance information of the i-th detected object and the candidate object are matched.

[0124] In some embodiments of this application, before performing feature extraction on the candidate object using a trained feature extraction model to obtain the first feature vector, i.e., before executing step S301, the following steps S501 to S504 may also be executed.

[0125] S501, Obtain multiple training images and initial object dimensions.

[0126] It should be noted that the training images can be images obtained from a database. Each training image includes a detection object, which can include livestock such as pigs, cattle, and sheep. The initial object size can be the size information of the pedestrian target (object), such as the height and aspect ratio of the pedestrian.

[0127] S502. Update the initial object size to the size corresponding to the detected object, and construct a training dataset based on multiple training images.

[0128] In some embodiments, after obtaining the training images and the initial object size, the initial object size can be updated to the size of the detected object. Training data can be constructed using the training images. In some embodiments, the training data can be constructed by performing image processing such as image enhancement on each training image to obtain more training data. Multiple training images and the image-processed training images constitute the training dataset.

[0129] S503. Train the initial feature extraction model based on the training dataset and the size corresponding to the detection object to obtain a trained feature extraction model.

[0130] In some embodiments, after obtaining the training dataset, an initial feature extraction model can be trained using the training data and the dimensions corresponding to the detected objects. The initial feature extraction model can be a Re-ID model. During the training of the initial feature extraction model, the detected objects in the training images are extracted using the dimensions corresponding to the detected objects. In practice, each training image in the training dataset corresponds to a sample label. During training, the initial feature extraction model can be backpropagated based on the difference between the output of the initial feature extraction model and the sample labels (e.g., the difference between the feature information of the known objects corresponding to the sample labels and the feature information of the detected objects in the output). This backpropagation training continues until a training termination condition is met (e.g., the difference between the feature information of the known objects corresponding to the sample labels and the feature information of the detected objects in the output is less than a preset threshold). At this point, training of the feature extraction model can be stopped, and a well-trained feature extraction model is obtained.

[0131] It is understood that, in the embodiments of this application, by updating the size of the initial object to the size corresponding to the detected object, and constructing a training dataset based on the training images including the detected object, the feature extraction model trained based on the size corresponding to the detected object in the training dataset can be more suitable for extracting features of the detected object (such as livestock such as pigs), thereby facilitating the subsequent similarity measurement between the detected object and the candidate object corresponding to the detected object, and reducing the counting error caused by the occlusion of multiple detected objects.

[0132] In some embodiments of this application, after obtaining at least one image frame corresponding to the video file, performing target detection on at least one image frame, and obtaining the target detection result, i.e., step S101, the following steps S601 to S604 can also be executed. Each step is described in detail below.

[0133] S601. Determine at least one first region in each image frame.

[0134] In some embodiments, the first region includes at least one detected object obtained through object detection. After performing object detection on each image frame and obtaining the object detection result, one or more first regions in each image frame can be determined. The first region may include one or more detected objects. When the first region includes multiple detected objects, the detection boxes corresponding to the multiple detected objects may be independent of each other, or there may be occlusion.

[0135] S602. Determine the total number of detection objects corresponding to each of the at least one first region.

[0136] In some embodiments, after determining at least one first region in each image frame, the number of detected objects in each first region can be counted to obtain the total number of detected objects corresponding to each region. In some embodiments, when an image frame includes multiple first regions, the total number of detected objects corresponding to each of the multiple first regions may be the same or different.

[0137] S603. Determine that there are target regions whose total number is greater than a preset threshold, and determine the image frame corresponding to the target region as the first image frame.

[0138] It should be noted that the preset threshold can be a pre-defined number of detection objects. When the number of detection objects exceeds the preset threshold, it is determined that the distribution of detection objects is relatively dense; conversely, it indicates that the distribution of detection objects is moderate. In some embodiments, after obtaining the total number of detection objects corresponding to at least one first region, a first region with a total number greater than the preset threshold can be further identified, and the first region with a total number greater than the preset threshold can be identified as the target region, and the image frame containing the target region can be identified as the first image frame.

[0139] In some embodiments, a target region may correspond to only one first image frame, that is, the first image frame includes only one target region; in other embodiments, multiple target regions may each correspond to one first image frame, that is, the first image frame includes multiple target regions.

[0140] S604. Output the target detection results and dense warning information corresponding to the first image frame.

[0141] In some embodiments, the first image frame represents a target area where the number of detected objects is greater than a preset threshold, that is, there are densely distributed detected objects in the first image frame. Therefore, after the first image frame is determined, the target detection result and dense warning information corresponding to the first image frame can be output.

[0142] In this embodiment, at least one image frame corresponding to a video file is acquired, and target detection is performed on each image frame to obtain target detection results. Based on the target detection results, tracking processing is performed on each detected object to determine the motion trajectory information of each detected object. Based on the motion trajectory information of each detected object, target objects that pass through reference line segments during their movement are identified from multiple detected objects. Based on the motion trajectory information of each target object and the position information of the reference line segment, the motion direction of each target object when passing through the reference line segment is determined. Based on the motion direction of each target object when passing through the reference line segment, data statistics are performed to obtain target statistical data. Thus, by determining the motion direction of the target object when passing through the reference line segment based on the motion trajectory of the target object and the position information of the reference line segment, and counting based on this motion direction, the problem of counting errors caused by the target object moving back and forth near the reference line segment can be solved, thereby improving the counting accuracy of target objects.

[0143] The implementation process of the embodiments of this application in a practical application scenario will be described below.

[0144] In some embodiments, such as Figure 2 The diagram shown is a flowchart illustrating a dynamic target tracking and counting method provided in an embodiment of this application. The dynamic target tracking and counting method provided in this application can be implemented through the following steps S701 to S704, which are described in detail below.

[0145] S701. Obtain at least one video frame (image frame) and virtual counting line (reference line segment) corresponding to the video file.

[0146] In some embodiments, video files can be obtained directly from a database, or they can be captured in real time using a camera or similar device. The virtual counting line can be set according to the size of the video frame, and is usually set in the middle area of ​​the video frame.

[0147] S702. Perform target detection on each video frame and obtain the target detection results.

[0148] It should be noted that object detection on each video frame can involve detecting objects within each frame and marking the detected objects. The object detection results can include the location information of the detected objects. In some embodiments, the Yolov5 algorithm can be used to perform object detection on each video frame separately, determining whether an object is detected. If an object is detected, it is then determined whether the number of objects exceeds a set density threshold. If it exceeds the set density threshold, a density warning is output and displayed. Afterward, bounding boxes are drawn around the detected objects, and the object confidence score is displayed.

[0149] It is understood that the embodiments of this application improve the feature recognition part of the Deep-Sort algorithm, Faster RCNN detection, and optimize it to the detection method of the Yolov5 algorithm. Yolov5 uses weighted DIOU_NMS, which can improve the detection rate of occluded objects without additional computational cost. By adopting a lightweight model based on Yolov5 to lighten the feature extraction model, the accuracy and speed of livestock and poultry target tracking can be improved.

[0150] S703. Based on the target detection results, target tracking (tracking processing) is performed on each target (detected object) in the video frame to obtain the motion trajectory information of each target.

[0151] In some embodiments, the YOLOv5 algorithm can be used to track targets in video frames. The first detected target is assigned a corresponding identity document (ID). Then, Kalman filtering is used to predict the trajectory of the target, and the predicted trajectory is matched with the target in the next video frame to establish the association between the previous and next frames. This process continues until all video frames have been processed, or the target has not appeared in a number of consecutive frames, i.e., the target has left the area in the image frame, thus obtaining the motion trajectory information corresponding to the target.

[0152] S704. Determine the statistical counting results of each target based on the motion trajectory information of each target and the position information of the virtual counting line (reference line segment).

[0153] In some embodiments, based on the position information of the virtual counting line, the two endpoints A and B of the virtual counting line, as well as the position coordinates of endpoint A and endpoint B, can be obtained. Simultaneously, the first position C and the second position D located on either side of the virtual counting line, along with their coordinates, can be determined from the motion trajectory information of the moving target. Then, a first vector can be determined based on the position coordinates of endpoint A and the coordinates of the first position C. The second vector is determined based on the coordinates of endpoint B and the coordinates of the first position C. The third vector is determined based on the coordinates of endpoint A and the coordinates of the second position D. The second vector is determined based on the coordinates of endpoint B and the coordinates of the second position D. Furthermore, according to the first vector Second vector The first vector product and the third vector and the fourth vector The second vector product determines the statistical count results for each target.

[0154] For example, if the rotation direction from the first vector to the second vector is determined to be counterclockwise based on the first vector product, and the rotation direction from the third vector to the fourth vector is determined to be clockwise based on the second vector product, then the target can be determined to have moved from the first position C to the second position D, and the first count value is incremented by one. If the rotation direction from the first vector to the second vector is determined to be clockwise based on the first vector product, and the rotation direction from the third vector to the fourth vector is determined to be counterclockwise based on the second vector product, then the target can be determined to have moved from the first position D to the second position C, and the second count value is incremented by one. Then, the statistical count result for each target can be determined based on the sum of the first count values ​​and the sum of the second count values. For example, the absolute value of the difference between the sum of the first count values ​​and the sum of the second count values ​​can be used as the statistical count result for each target. When the sum of the first count values ​​is 20 and the sum of the second count values ​​is 15, the statistical count result for the target is determined to be 5.

[0155] It is understood that the dynamic target tracking and counting method provided in this application embodiment is based on the YOLOv5 algorithm to detect targets in video frames, and combined with the Deep-Sort algorithm to track the detected targets in order to obtain the motion trajectory information of each target. In particular, when tracking each target, the feature extraction model in the Deep-Sort algorithm is optimized to adapt to the input size of livestock and poultry body shape and then retrained, which solves the target occlusion problem in the tracking process. Then, based on the motion trajectory information of each target and the position information of the virtual counting line, the statistical counting result of each target is determined, realizing high-precision and real-time counting of dynamic livestock and poultry targets, and improving the accuracy of detection, tracking and counting of dynamic wandering targets.

[0156] In some embodiments, the dynamic target tracking and counting method provided in this application can be achieved through... Figure 3 The process follows these steps: First, it initializes the weight file for the object detection algorithm, the video frame rate corresponding to the video file, and the display position of the counting results. Then, it sets an appropriate virtual counting line based on the video frame size, typically in the middle of the video frame. Next, it selects to open the video file or open the camera to capture video in real-time. Afterward, it performs real-time detection, first reading a video frame, and the Yolov5 algorithm performs object detection on that frame to determine if a target is detected. If a target is found, it checks if the number of targets exceeds the set density threshold, displaying a density warning. Then, it draws bounding boxes around the detected targets and displays the target confidence score. The process ends when the last frame of the video is read.

[0157] After target detection, real-time tracking is possible. Starting from the first frame, the YOLOv5 algorithm is used to detect pig targets in the image (target detection is performed on at least one image frame). The detected target bounding boxes are fed into the Deep-Sort algorithm. The first target to appear is assigned a new ID and a corresponding tracker is created. Then, Kalman filtering is used to predict its mean and variance, and Kalman filter gain and noise are added for trajectory updates. In the second frame, the updated trajectory is concatenated and matched with the detection bounding boxes of the next frame. Maximum matching is achieved using motion information based on Mahalanobis distance and appearance information based on cosine distance, establishing a correlation between consecutive frames. Before processing each newly generated target, the feature extraction model in the Deep-Sort algorithm saves the target's features. Then, during matching, the similarity between the detected target and the appearance features of all previously stored targets is calculated to solve the problem of long-term target occlusion. Since the feature extraction model in the original Deep-Sort algorithm is a pedestrian re-identification model (Re-ID), the aspect ratio of the pedestrian target in its dataset is not suitable for the body size of livestock and poultry, resulting in poor feature extraction performance. This application optimizes the original feature extraction model by changing the data size of the input model and recreating the livestock and poultry target tracking dataset for training the livestock and poultry feature extraction model, making it more suitable for calculating the feature similarity of livestock and poultry.

[0158] After tracking the target, it can be counted in real time. Unlike the traditional method of determining whether a target has crossed the line by judging whether line segments intersect, this method determines the direction of the target's crossing by judging the direction of the vector product from the target's position to the two ends of the virtual counting line, thus determining whether a target has crossed the virtual counting line. Since livestock targets may run back and forth in the video and linger near the virtual counting line, the direction of crossing the virtual counting line is determined by whether it is a left or right crossing, and the corresponding count value is incremented by 1. The total count value equals the number of left crossings minus the number of right crossings, and the count result is then displayed on the screen. Then, the second frame is read, and this process is repeated until the last frame of the video is read.

[0159] In some embodiments, according to the cross product rule of vectors, when the target is at position C, the vector... with vector vector product The direction is perpendicular to the paper and upwards. When the target is at position D, the vector... with vector vector product The direction is perpendicular to the paper and downwards. The vector product is defined as shown in equations (6) and (7):

[0160]

[0161]

[0162] make Let a vector direction perpendicular to the paper and upwards be positive. Define the operation:

[0163]

[0164] Therefore, when the target moves from position C to position D by passing the virtual counting line from right to left, it satisfies formula (8), and the left count (first count value) is incremented by 1. Similarly, when the following formula (9) is satisfied, it means that the target moves from position D to position C by passing the virtual counting line from left to right, and the right count is incremented by 1 (second count value).

[0165]

[0166] In some embodiments, the counting result should not change when the target moves up and down on the virtual counting line, and the count should be incremented by 1 when the target moves from the virtual counting line to the other side (boundary condition). The final determination formula is shown in formula (10), where left represents the left count, right represents the right count, and the total count value is equal to the absolute value of the difference between the left count value and the right count value (determine the absolute value of the difference between the first statistical result and the second statistical result, and use the absolute value of the difference as the target statistical data).

[0167]

[0168] It is understood that, in the embodiments of this application, the direction of the target crossing the line is determined by the direction of the vector product from the target's location to the two ends of the virtual counting line. This can solve the problem of repeated counting caused by the irregular movement of livestock and poultry targets or the large changes in their movement routes, reduce the error caused by the target moving near the virtual counting line, and improve the counting accuracy of the target.

[0169] This application also provides a data processing apparatus. Figure 4 This is a schematic diagram of the composition structure of a data processing device provided in an embodiment of this application, as shown below. Figure 4 As shown, the data processing device 800 includes:

[0170] The first acquisition module 801 acquires at least one image frame corresponding to the video file, performs target detection on the at least one image frame, and obtains the target detection result.

[0171] The first determining module 802 is used to perform tracking processing on each detected object based on the target detection result and determine the motion trajectory information of each detected object;

[0172] The second determining module 803 is used to determine, based on the motion trajectory information of each of the detected objects, the target objects that pass through the reference line segment during the motion process from multiple detected objects, wherein the reference line segment is used to count the number of the target objects;

[0173] The third determining module 804 is used to determine the direction of motion of each target object when it passes through the reference line segment based on the motion trajectory information of each target object and the position information of the reference line segment.

[0174] The second acquisition module 805 is used to perform data statistics based on the movement direction of each target object when it passes through the reference line segment, and obtain target statistical data.

[0175] It should be noted that the description of the data processing apparatus in this application embodiment is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment; therefore, it will not be repeated. For technical details not disclosed in this apparatus embodiment, please refer to the description of the method embodiment in this application for understanding.

[0176] It should be noted that, in the embodiments of this application, if the above-described control method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0177] Accordingly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method provided in the above embodiments.

[0178] This application also provides a data processing device. Figure 5 This is a schematic diagram of the composition structure of a data processing device provided in an embodiment of this application, such as... Figure 5As shown, the data processing device 900 includes: a memory 901, a processor 902, a communication interface 903, and a communication bus 904. The memory 901 stores executable data processing instructions; the processor 902 executes the executable data processing instructions stored in the memory to implement the data processing method provided in the above embodiment.

[0179] The descriptions of the data processing apparatus and storage medium embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the data processing apparatus and storage medium embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0180] It should be understood that the phrases "some embodiments" or "one embodiment" throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "some embodiments" or "one embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0181] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0182] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0183] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0184] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0185] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0186] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a product to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0187] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, include: Obtain at least one image frame corresponding to the video file, perform target detection on the at least one image frame, and obtain the target detection result; Based on the target detection results, each detected object is tracked and processed to determine the motion trajectory information of each detected object; Based on the motion trajectory information of each detected object, target objects that pass through reference line segments during the motion are determined from multiple detected objects. The reference line segments are used to count the number of target objects. The motion direction of each target object when passing through the reference line segment is determined based on the motion trajectory information of each target object and the position information of the two endpoints of the reference line segment; wherein, the reference line segment includes a horizontal reference line segment or a vertical reference line segment; Target statistics are obtained by statistically analyzing the movement direction of each target object as it passes through the reference line segment. Determining the direction of motion of each target object when it passes through the reference line segment based on the motion trajectory information of each target object and the position information of the two endpoints of the reference line segment includes: Based on the motion trajectory information of each target object, determine the first coordinates of the first position and the second coordinates of the second position of the target object at the first position on both sides of the reference line segment during the motion process; Based on the position information of the reference line segment, determine the third coordinate of the first endpoint and the fourth coordinate of the second endpoint of the reference line segment; Based on the first coordinate and the third coordinate, a first vector is determined; based on the second coordinate and the third coordinate, a second vector is determined. Based on the first coordinate and the fourth coordinate, a third vector is determined; based on the second coordinate and the fourth coordinate, a fourth vector is determined. The outer product of the first vector and the third vector is defined as the first vector product; When it is determined that the first vector to the third vector is a counterclockwise rotation direction based on the first vector product, the motion direction of each target object is determined to be from the first side to the second side of the reference line segment; The outer product of the second vector and the fourth vector is defined as the second vector product; When the second vector to the fourth vector is determined to be a clockwise rotation direction based on the second vector product, the motion direction of each target object is determined to be from the second side to the first side of the reference line segment.

2. The method according to claim 1, characterized in that, The method of statistically analyzing data based on the movement direction of each target object as it passes through the reference line segment to obtain target statistical data includes: Get the first and second count values ​​corresponding to each target object; When the target object moves from the first side to the second side of the reference line segment, the first count value is updated to obtain the updated first count value. When the target object moves from the second side to the first side of the reference line segment, the second count value is updated to obtain the updated second count value; Determine the first statistical result corresponding to the updated first count value and the second statistical result corresponding to the updated second count value; Determine the absolute value of the difference between the first statistical result and the second statistical result, and use the absolute value of the difference as the target statistical data.

3. The method according to claim 1, characterized in that, The target detection results also include the detection location information of each detected object; The step of tracking each detected object based on the target detection results to determine the motion trajectory information of each detected object includes: Obtain the first detection position information corresponding to the i-th detection object in the current image frame and the second detection position information of each detection object in the next image frame; Based on the first detection location information, the position of the i-th detected object in the next image frame is predicted to obtain the predicted location information of the i-th detected object; Determine the Mahalanobis distances between the predicted location information and each of the second detection location information, and determine the minimum Mahalanobis distance among the various Mahalanobis distances; If the minimum Mahalanobis distance is determined to be less than a first preset distance threshold, a candidate object for the i-th detection object is determined from each detection object in the next image frame; When the shape of the i-th detected object and the candidate object meets the matching condition, it is determined that the i-th detected object and the candidate object have reached the target matching condition. Based on the first detection position information and the second detection position information of the candidate object, the motion trajectory information of the i-th detected object is determined.

4. The method according to claim 3, characterized in that, The method further includes: The candidate object is subjected to feature extraction using a trained feature extraction model to obtain a first feature vector; When the current image frame is the first frame image, the trained feature extraction model is used to extract features from the i-th detected object in the current image frame to obtain the second feature vector. Determine the cosine distance between the first feature vector and the second feature vector. When the cosine distance is less than a second preset distance threshold, determine that the shape of the i-th detection object and the candidate object meet the matching condition.

5. The method according to claim 4, characterized in that, The method further includes: When the current image frame is not the first image frame, each matching object that meets the target matching condition with the i-th detected object of the current image frame is determined from each historical frame before the current image frame; Obtain the third feature vector of each matching object corresponding to the i-th detection object; Determine the cosine distances between the first feature vector and each of the third feature vectors, and determine the minimum cosine distance among the cosine distances. When the minimum cosine distance is less than the second preset distance threshold, it is determined that the shape of the i-th detection object and the candidate object meet the matching condition.

6. The method according to claim 4, characterized in that, Also includes: Obtain multiple training images and initial object sizes, with each training image including the object to be detected; The initial object size is updated to the size corresponding to the detected object, and a training dataset is constructed based on the multiple training images; The initial feature extraction model is trained based on the training dataset and the size corresponding to the detection object to obtain the trained feature extraction model.

7. The method according to claim 1, characterized in that, The method further includes: Determine at least one first region in each image frame, wherein the first region includes at least one detected object obtained by the target detection; Determine the total number of detection objects corresponding to each of the at least one first region; A target region is identified whose total number is greater than a preset threshold, and the image frame corresponding to the target region is identified as the first image frame. Output the target detection results and dense warning information corresponding to the first image frame.

8. A data processing apparatus, characterized in that, include: The first acquisition module acquires at least one image frame corresponding to the video file, performs target detection on the at least one image frame, and obtains the target detection result. The first determining module is used to track and process each detected object based on the target detection results, and determine the motion trajectory information of each detected object; The second determining module is used to determine, based on the motion trajectory information of each of the detected objects, the target objects that pass through the reference line segment during the motion process from among the multiple detected objects, wherein the reference line segment is used to count the number of the target objects; The third determining module is used to determine the direction of motion of each target object when it passes through the reference line segment based on the motion trajectory information of each target object and the position information of the two endpoints of the reference line segment; wherein, the reference line segment includes a horizontal reference line segment or a vertical reference line segment; The second acquisition module is used to perform data statistics based on the movement direction of each target object when it passes through the reference line segment, and obtain target statistical data. The first determining module is configured to: determine, based on the motion trajectory information of each target object, the first coordinates of the first position and the second coordinates of the second position of the target object at the first position on both sides of the reference line segment during the motion process; determine, based on the position information of the reference line segment, the third coordinates of the first endpoint and the fourth coordinates of the second endpoint of the reference line segment; determine, based on the first coordinates and the third coordinates, a first vector; determine, based on the second coordinates and the third coordinates, a second vector; determine, based on the first coordinates and the fourth coordinates, a third vector; determine, based on the second coordinates and the fourth coordinates, a fourth vector; determine, by multiplying the first vector and the third vector, a first vector product; when the first vector product determines that the rotation direction from the first vector to the third vector is counterclockwise, determine that the motion direction of each target object is from the first side to the second side of the reference line segment; determine, by multiplying the second vector and the fourth vector, a second vector product; when the second vector product determines that the rotation direction from the second vector to the fourth vector is clockwise, determine that the motion direction of each target object is from the second side to the first side of the reference line segment.

9. A data processing device, characterized in that, include: Memory, used to store executable data processing instructions; A processor, when executing executable data processing instructions stored in the memory, implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The device stores executable data processing instructions for causing a processor to execute, thereby implementing the method as described in any one of claims 1 to 7.

Images