Parcel sorting behavior identification method and device

Abstract

The present disclosure proposes a method and device for identifying package sorting behavior, which relate to the field of computer vision technology. The present disclosure performs target detection on image frames in a target video to obtain at least one human detection frame and at least one package detection frame; tracks the motion trajectories of the human detection frame and the package detection frame respectively; in the process of trajectory tracking, identifies the moment when any package is thrown out based on the tracked current motion trajectory; obtains the motion information of any package from the moment it is thrown out to the current moment, and identifies the sorting behavior of any package based on the motion information. When identifying the package sorting behavior, the present application can effectively eliminate misjudgments such as only people, or only packages, or people moving with packages, or interference from irrelevant background information, by judging the interaction relationship between the human body and the package, and can also identify the main body of the sorting personnel and determine the level of sorting force, making the judgment more accurate.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method and device for identifying package sorting behavior. Background Art

[0002] With the continuous development of e-commerce, online shopping and logistics industries, people's demand for express delivery services is also increasing. In the rapid development of the express delivery industry, when sorting parcels, sorters often engage in violent sorting such as throwing and dropping parcels in order to increase the sorting speed, which will cause damage to the items in the parcels and harm the interests of consumers and merchants. Summary of the Invention

[0003] The present application aims to solve one of the technical problems in the related art at least to a certain extent.

[0004] To this end, one object of the present application is to propose a method for identifying package sorting behavior, by performing target detection on image frames in a target video to obtain at least one human detection frame and at least one package detection frame; tracking the motion trajectories of the human detection frame and the package detection frame respectively; in the process of trajectory tracking, identifying the moment when any package is thrown out based on the tracked current motion trajectory; obtaining the motion information of any package from the moment when it is thrown out to the current moment, and identifying the sorting behavior of any package based on the motion information.

[0005] When identifying package sorting behavior, this application can effectively eliminate misjudgments such as only a person, or only a package, or a person and a package moving together, or interference from irrelevant background information, by judging the interaction between the human body and the package. It can also identify the main body of the sorting person and determine the level of sorting force, making the judgment more accurate.

[0006] The second objective of this application is to provide a device for identifying parcel sorting behavior.

[0007] The third objective of this application is to provide an electronic device.

[0008] A fourth object of the present application is to provide a non-transitory computer-readable storage medium.

[0009] A fifth object of this application is to provide a computer program product.

[0010] To achieve the above-mentioned purpose, the first aspect embodiment of the present application proposes a method for identifying package sorting behavior, including: performing target detection on image frames in a target video to obtain at least one human detection frame and at least one package detection frame; tracking the motion trajectories of the human detection frame and the package detection frame respectively; in the process of trajectory tracking, identifying the moment when any package is thrown out based on the tracked current motion trajectory; obtaining the motion information of any package from the moment when it was thrown out to the current moment, and based on the motion information, identifying the sorting behavior of any package.

[0011] When identifying package sorting behavior, this application can effectively eliminate misjudgments such as only a person, or only a package, or a person and a package moving together, or interference from irrelevant background information, by judging the interaction between the human body and the package. It can also identify the main body of the sorting person and determine the level of sorting force, making the judgment more accurate.

[0012] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a device for identifying package sorting behavior, including: a first acquisition module, used to perform target detection on image frames in a target video, and obtain at least one human detection frame and at least one package detection frame; a trajectory tracking module, used to track the motion trajectories of the human detection frame and the package detection frame respectively; a second acquisition module, used to identify the moment when any package is thrown out based on the tracked current motion trajectory during the trajectory tracking process; a behavior recognition module, used to obtain the motion information of any package from the moment it is thrown out to the current moment, and based on the motion information, identify the sorting behavior of any package.

[0013] The parcel sorting behavior recognition device proposed in this application can effectively eliminate misjudgments such as only a person, only a package, a person and a package moving together, or interference from irrelevant background information by judging the interaction between the human body and the package when identifying parcel sorting behavior. It can also identify the main body of the sorting person and determine the level of sorting force, making the judgment more accurate.

[0014] To achieve the above-mentioned purpose, the third aspect embodiment of the present application proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to implement the method for identifying parcel sorting behavior as described in the first aspect embodiment of the present application.

[0015] To achieve the above-mentioned purpose, the fourth embodiment of the present application proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to implement the method for identifying parcel sorting behavior as described in the first embodiment of the present application.

[0016] To achieve the above-mentioned purpose, the fifth embodiment of the present application proposes a computer program product, including a computer program, which, when executed by a processor, implements the method for identifying parcel sorting behavior as described in the first embodiment of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic diagram of a method for identifying parcel sorting behavior according to an embodiment of the present application.

[0018] Figure 2 This is a schematic diagram of the trajectory of a single person sorting packages according to an embodiment of the present application.

[0019] Figure 3 This is a schematic diagram of an embodiment of the present application for identifying the moment when any package is thrown out.

[0020] Figure 4 This is a schematic diagram of identifying the sorting behavior of any package according to an embodiment of the present application.

[0021] Figure 5 This is a schematic diagram of tracking the motion trajectories of a human detection frame and a package detection frame respectively according to an embodiment of the present application.

[0022] Figure 6 This is a schematic diagram of a method for identifying parcel sorting behavior according to an embodiment of the present application.

[0023] Figure 7 This is a schematic diagram of a device for identifying parcel sorting behavior according to an embodiment of the present application.

[0024] Figure 8 This is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0026] Figure 1 This is an exemplary embodiment of a method for identifying parcel sorting behavior proposed in this application, such as Figure 1As shown, the method for identifying the parcel sorting behavior includes the following steps:

[0027] S101: Perform target detection on image frames in a target video to obtain at least one human detection frame and at least one package detection frame.

[0028] The video to be analyzed for parcel sorting behavior is used as the target video. The target video can be a real-time video of the sorter sorting parcels, or a locally stored or received video of the sorter sorting parcels. The target video is decoded and framed to obtain multiple image frames corresponding to the target video at different times. Object detection is performed on all image frames corresponding to the target video, and the human detection frame corresponding to the sorter in each image frame is obtained. The human detection frame is recorded as P box (t1), and the package detection frame corresponding to the sorted package, the package detection frame is recorded as B box (t2), where t1 and t2 are the frame indexes of the image frames corresponding to the human detection frame or the package detection frame, and each detection frame includes the coordinate information of the detection frame in the image frame.

[0029] Optionally, the target detection algorithm may employ algorithms such as Feature Pyramid Networks (FPN) and Convolutional Neural Networks (CNN). In each image frame, there may be one or more human detection frames and one or more package detection frames. If there are multiple sorting personnel in an image frame, there may be multiple human detection frames in the image frame. If there are multiple sorted packages in an image frame, there may be multiple package detection frames in the image frame.

[0030] S102 , tracking the motion trajectories of the human body detection frame and the package detection frame respectively.

[0031] The human body detection frame and the package detection frame of each frame image obtained above are respectively assigned identification information. According to the identification information of the human body detection frame and the package detection frame, the motion trajectory of the human body detection frame and the package detection frame are respectively tracked to obtain the human body motion trajectory, which is recorded as P track (i, t1), where i represents the identification information of the human body, t1 represents the tracking state of the human body at the latest t1 image frame; and the package motion trajectory, which is recorded as B track (j, t2), where j represents the package identification information and t2 represents the tracking status of the package at the latest image frame t2. Optionally, target tracking can be performed using a tracking algorithm such as nearest neighbor matching or Simple Online and Realtime Tracking (SORT).

[0032] S103 , during the trajectory tracking process, identifying the time at which any package is thrown out based on the tracked current motion trajectory of any package.

[0033] During the trajectory tracking process, the tracking trajectory of each package is traversed to determine the time when each package is thrown out. Figure 2 This is a schematic diagram of a human body sorting packages, such as Figure 2 As shown, taking a package as an example, the image frame corresponding to the package is traced back from the current frame, that is, Figure 2 As shown, starting from the image frame at time t, we trace back to the image frames corresponding to time t-1, time t-2, time t-3, time t-4, time t-5, and time t-6, and analyze the image frames to determine whether the package was thrown out by the sorter. If it is confirmed that the package was thrown out by the sorter, we can identify the image frame where the package was thrown out, and use the time corresponding to the image frame where the package was thrown out as the throwing time. Figure 2 As shown, the distance between the package detection frame and the human body detection frame in the image frame at time t-4 is less than the distance threshold, that is, at time t-4, the sorter threw the package out, and time t-4 is the time when the package was thrown out.

[0034] S104: Obtain movement information of any package from the moment it is thrown to the current moment, and identify the sorting behavior of any package based on the movement information.

[0035] Figure 2 During times t-6, t-5, and t-4, the sorter is moving while holding the package. The package remains within the sorter's body, and any movement during this period is not considered a candidate for sorting behavior. After time t-4, the package begins to separate from the sorter's body, and is subsequently thrown. To eliminate misjudgments due to the sorter's own large movements or the delivery and receiving behaviors between sorters, only the various motion information of the package from the moment it was thrown to the current moment is obtained, such as acceleration, distance traveled, and average speed. Because the acceleration, distance traveled, and average speed of packages corresponding to gentle handling differ significantly from those corresponding to violent throwing, motion quantities such as acceleration, distance traveled, and average speed can be used to reflect the sorter's sorting effort. Based on the acquired motion information, the sorter's sorting behavior parameters can be obtained. By comparing these sorting behavior parameters with parameters in existing sorting behavior specifications, the sorter's sorting behavior can be identified. Alternatively, when obtaining the sorter's sorting behavior parameters, parameters can be compared based on each piece of motion information to identify the sorter's sorting behavior. Among them, the parameters in the sorting behavior specification can be set differently according to different types of package scenarios.

[0036] The embodiment of the present application proposes a method for identifying package sorting behavior. The method performs target detection on image frames in a target video to obtain at least one human detection frame and at least one package detection frame. The motion trajectories of the human detection frame and the package detection frame are tracked respectively. During the trajectory tracking process, the moment when any package is thrown is identified based on the tracked current motion trajectory. The motion information of any package from the moment it is thrown to the current moment is obtained, and based on the motion information, the sorting behavior of any package is identified. When identifying the package sorting behavior, the present application can effectively eliminate misjudgments such as only a person, or only a package, or a person moving with a package, or interference from irrelevant background information by judging the interaction between the human body and the package. It can also identify the main body of the sorting person and determine the level of sorting force, making the judgment more accurate.

[0037] Figure 3 This is an exemplary embodiment of a method for identifying parcel sorting behavior proposed in this application, such as Figure 3 As shown, for any package, identifying the time when the package is thrown out based on the tracked current motion trajectory includes the following steps:

[0038] S301 : Compare the target package motion trajectory corresponding to any package with the motion trajectory of each human body to obtain the target distance between the target package detection frame corresponding to any package and each human body detection frame at the detected moment.

[0039] According to the motion trajectory of the target package, the current image frame is traced back from late to early, and the position information of the target package detection frame on each image frame is obtained as the first position information, and the position information of the human body detection frame on the same image frame is obtained as the second position information.

[0040] As a feasible method, if there is only one sorting personnel, that is, there is only one human body detection frame on the image frame, the distance between the first position information of the target package detection frame and the second position information of the human body detection frame of the human body at the corresponding moment in each image frame is calculated as the target distance.

[0041] As another feasible method, if there are multiple sorting personnel, there will be multiple human body detection frames on the same image frame. Since the human body detection frame is relatively large, when calculating the distance between the first position information of the target package detection frame and the second position information of all human body detection frames on the image frame at the corresponding moment, if a certain sorting personnel throws the package over a certain sorting personnel and throws it relatively far, the first person who passed over will be calculated on the image frame, resulting in a deviation. Therefore, when there are multiple sorting personnel, human body key point detection is performed on the human body detection frame on the image frame to obtain the human hand position information, thereby obtaining the distance between the first position information of the target package detection frame and the human hand position information of all human bodies on the image frame at the corresponding moment as the target distance. Optionally, the human body key point detection algorithm can adopt a human skeleton key point detection algorithm, etc.

[0042] S302, determining the time when the target distance is less than the distance threshold for the first time, and using it as the time when any package is thrown.

[0043] A threshold distance is set in advance, and the moment corresponding to the first image frame in which the target distance is less than the distance threshold is used as the time when the package is thrown out. At the same time, the sorting personnel corresponding to the package can also be determined.

[0044] The embodiment of the present application identifies the moment when any package is thrown based on the tracked current motion trajectory, and can determine the interaction relationship between the package and the human body, thereby determining who threw the package, when, and the throwing trajectory, thereby more accurately identifying the package sorting behavior.

[0045] Figure 4 This is an exemplary embodiment of a method for identifying parcel sorting behavior proposed in this application, such as Figure 4 As shown, based on the motion information, the sorting behavior of any package is identified, including the following steps:

[0046] S401: Generate a sorting force parameter for any package based on the motion information.

[0047] Each package is treated as a target package, and various motion information of each target package from the moment it is thrown to the current moment is obtained. Optionally, the motion information may include the distance value, maximum speed, average speed of each target package from the moment it is thrown to the current moment, and the motion speed and acceleration of each target package at each moment from the moment it is thrown to the current moment.

[0048] As a feasible method, the numerical value of each motion information can be used as the motion parameter of the motion information. For example, if the distance value of a package from the moment it is thrown to the current moment is 3 meters, 3 meters can be used as the motion parameter of the distance value from the moment it is thrown to the current moment.

[0049] S402: Determine the sorting intensity level of the sorting behavior of any package according to the sorting intensity parameter.

[0050] Different sorting behavior specifications can be set for different scenarios. For example, if the package is fresh produce, the items inside are relatively fragile. When the distance value from the time the package was thrown to the current moment is used as a parameter, the parameter range can be set to a smaller value. For example, if the distance value from the time the fresh produce package was thrown to the current moment is less than 0.2 meters, it is normal, 0.2-0.4 meters is mild violence, 0.4-0.7 meters is general violence, and 0.7-1 meter is severe violence. The distance value of any package from the time it was thrown to the current moment is compared with the parameters of the sorting behavior specification to identify the sorting behavior of the sorting personnel corresponding to the package.

[0051] For another example, when a package is a clothing product, it is unlikely to be deformed by being thrown, so the sorting behavior specifications for fresh clothing products are set to be stricter than those for fresh products.

[0052] Similarly, other items of motion information, such as acceleration and average speed, can be compared with sorting behavior specifications to identify the sorting behavior of the sorting personnel corresponding to the target package.

[0053] The embodiment of the present application determines the sorting force level of the sorting behavior of any package based on motion information, and can provide detailed indicators for the sorter when sorting packages. In addition, different specifications can be set for the sorter's sorting behavior in different business scenarios, which can improve the accuracy and versatility of sorting behavior recognition.

[0054] Figure 5 This is an exemplary embodiment of a method for identifying parcel sorting behavior proposed in this application, such as Figure 5 As shown, tracking the motion trajectories of the human detection frame and the package detection frame respectively includes the following steps:

[0055] S501 : Tracking the human body detection frame based on first identification information of the human body detection frame to generate a human body motion trajectory corresponding to the human body detection frame.

[0056] Each human body detection frame in the image frame is assigned an identification information as the first identification information. Based on the first identification information corresponding to each human body detection frame, the human body detection frame in each image frame is tracked to generate the human motion trajectory corresponding to the human body detection frame. Optionally, the target tracking can be performed using a tracking algorithm such as nearest neighbor matching or Simple Online and Realtime Tracking (SORT) algorithm.

[0057] S502 : Track the package detection frame based on the second identification information of the package detection frame to generate a package motion trajectory corresponding to the package detection frame.

[0058] Each package detection frame within an image frame is assigned an identification information as the second identification information. Based on the second identification information corresponding to each package detection frame, the package detection frame in each image frame is tracked to generate the package motion trajectory corresponding to the package detection frame. Optionally, target tracking can be performed using a tracking algorithm such as nearest neighbor matching or Simple Online and Realtime Tracking (SORT).

[0059] This embodiment tracks the motion trajectories of the human body detection frame and the package detection frame to obtain the human body motion trajectory and the package motion trajectory, laying the foundation for realizing the interaction between the human body and the package and obtaining the moment when the package is thrown.

[0060] Figure 6 This is an exemplary embodiment of a method for identifying parcel sorting behavior proposed in this application, such as Figure 6 As shown, the method for identifying the parcel sorting behavior includes the following steps:

[0061] S601: Perform target detection on image frames in a target video to obtain at least one human detection frame and at least one package detection frame.

[0062] S602 : Track the human body detection frame based on the first identification information of the human body detection frame to generate a human body motion trajectory corresponding to the human body detection frame.

[0063] S603: Track the package detection frame based on the second identification information of the package detection frame to generate a package motion trajectory corresponding to the package detection frame.

[0064] Regarding the implementation method of steps S602 to S603, the implementation method in the above embodiments of the present application can be adopted and will not be described in detail here.

[0065] S604: Compare the target package motion trajectory corresponding to any package with the motion trajectory of each human body to obtain a target distance between the target package detection frame corresponding to any package and each human body detection frame at the detected moment.

[0066] S605: Determine the time when the target distance is less than the distance threshold for the first time, and use it as the time when any package is thrown out.

[0067] Regarding the implementation method of steps S604 to S605, the implementation method in the above embodiments of the present application can be adopted and will not be described in detail here.

[0068] S606: Generate a sorting force parameter for any package based on the motion information.

[0069] S607: Determine the sorting intensity level of the sorting behavior of any package according to the sorting intensity parameter.

[0070] The embodiment of the present application proposes a method for identifying package sorting behavior. The method performs target detection on image frames in a target video to obtain at least one human detection frame and at least one package detection frame. The motion trajectories of the human detection frame and the package detection frame are tracked respectively. During the trajectory tracking process, the moment when any package is thrown is identified based on the tracked current motion trajectory. The motion information of any package from the moment it is thrown to the current moment is obtained, and based on the motion information, the sorting behavior of any package is identified. When identifying the package sorting behavior, the present application can effectively eliminate misjudgments such as only a person, or only a package, or a person moving with a package, or interference from irrelevant background information by judging the interaction between the human body and the package. It can also identify the main body of the sorting person and determine the level of sorting force, making the judgment more accurate.

[0071] Figure 7 This is a schematic diagram of a parcel sorting behavior identification device proposed in this application, such as Figure 7 As shown, the parcel sorting behavior recognition device 700 includes a first acquisition module 71, a trajectory tracking module 72, a second acquisition module 73 and a behavior recognition module 74, wherein:

[0072] The first acquisition module 71 is configured to perform target detection on image frames in a target video to obtain at least one human detection frame and at least one package detection frame.

[0073] The trajectory tracking module 72 is used to track the motion trajectories of the human detection frame and the package detection frame respectively.

[0074] The second acquisition module 73 is used to identify the time when any package is thrown out based on the tracked current motion trajectory of any package during the trajectory tracking process.

[0075] The behavior recognition module 74 is used to obtain the movement information of any package from the time it is thrown to the current time, and identify the sorting behavior of any package based on the movement information.

[0076] Furthermore, the second acquisition module 73 is also used to: compare the target package motion trajectory corresponding to any package with the motion trajectory of each human body to obtain the target distance between the target package detection frame corresponding to any package and each human body detection frame at the detected moment; determine the moment when the target distance is less than the distance threshold for the first time, and use it as the moment when any package is thrown.

[0077] Furthermore, the second acquisition module 73 is also used to: obtain first position information of the target package detection frame on each image frame from the target package motion trajectory in order from late to early; obtain second position information of the human body detection frame on the same image frame from the human body motion trajectory; and obtain the target distance based on the first position information and the second position information at the corresponding moment.

[0078] Furthermore, the second acquisition module 73 is further configured to use the time corresponding to the image frame in which the target distance is first smaller than the distance threshold as the time when the package is thrown.

[0079] Furthermore, the second acquisition module 73 is also used to: extract the image area marked by the second position information from the image frame corresponding to the second position information; perform human key point detection on the image area to obtain human hand position information; obtain the distance from the first position information to the human hand position information as the target distance.

[0080] Furthermore, the behavior recognition module 74 is further configured to generate a sorting force parameter of any package based on the motion information, and determine a sorting force level of the sorting behavior of any package according to the sorting force parameter.

[0081] Furthermore, the motion information in the behavior recognition module 74 includes the distance value, maximum speed, average speed of any package from the moment it is thrown to the current moment, and the motion speed and acceleration of any package at each moment from the moment it is thrown to the current moment.

[0082] Furthermore, the trajectory tracking module 72 is also used to: track the human body detection frame based on the first identification information of the human body detection frame to generate a human body motion trajectory corresponding to the human body detection frame; track the package detection frame based on the second identification information of the package detection frame to generate a package motion trajectory corresponding to the package detection frame.

[0083] In order to implement the above embodiment, the present application also provides an electronic device 800, such as Figure 8 As shown, the electronic device 800 includes: a processor 801 and a memory 802 communicatively connected to the processor, the memory 802 stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor 801 to implement the method for identifying package sorting behavior as shown in the above embodiment.

[0084] In order to implement the above embodiment, the embodiment of the present application further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to implement the method for identifying parcel sorting behavior as shown in the above embodiment.

[0085] In order to implement the above embodiments, the embodiments of the present application further provide a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method for identifying the parcel sorting behavior as shown in the above embodiments.

[0086] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0087] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0088] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for identifying parcel sorting behavior, characterized in that: include: Performing target detection on image frames in the target video to obtain at least one human detection frame and at least one package detection frame; Tracking the motion trajectories of the human body detection frame and the package detection frame respectively; During the trajectory tracking process, for any package, identifying the time when the package is thrown out based on the tracked current motion trajectory; Obtaining movement information of the any package from the moment the package was thrown to the current moment, and identifying the sorting behavior of the any package based on the movement information; The current motion trajectory includes a human motion trajectory corresponding to the human detection frame and a package motion trajectory corresponding to the package detection frame, wherein identifying the moment when any package is thrown based on the tracked current motion trajectory includes: Acquire the first position information of the target package detection frame on each image frame from the target package motion trajectory in order from late to early; Acquire second position information of the human body detection frame on the same image frame from the human body motion trajectory; When there is only one sorter, the target distance is obtained according to the first position information and the second position information at the corresponding moment; When there are multiple sorting personnel, extracting the image area marked by the second position information from the image frame corresponding to the second position information; Performing human body key point detection on the image area to obtain human hand position information; Acquire the distance from the first position information to the position information of the human hand as the target distance; The moment when the target distance is less than the distance threshold for the first time is determined and used as the moment when any package is thrown.

2. The method according to claim 1, characterized in that The determining of the time when the target distance is less than the distance threshold for the first time and using the time when any package is thrown out includes: The moment corresponding to the image frame in which the target distance is less than the distance threshold for the first time is taken as the moment when the package is thrown.

3. The method according to claim 1, characterized in that The identifying the sorting behavior of any package based on the movement information includes: Based on the movement information, a sorting force parameter of the any package is generated, and according to the sorting force parameter, a sorting force level of the sorting behavior of the any package is determined.

4. The method according to claim 3, characterized in that The motion information includes the distance value, maximum speed, average speed of any package from the moment of being thrown to the current moment, and the motion speed and acceleration of any package at each moment from the moment of being thrown to the current moment.

5. The method according to any one of claims 1 to 4, characterized in that The tracking of the motion trajectories of the human detection frame and the package detection frame respectively includes: Tracking the human body detection frame based on the first identification information of the human body detection frame to generate a human body motion trajectory corresponding to the human body detection frame; Based on the second identification information of the package detection frame, the package detection frame is tracked to generate a package motion trajectory corresponding to the package detection frame.

6. A device for identifying parcel sorting behavior, characterized in that: include: A first acquisition module is configured to perform target detection on image frames in a target video to obtain at least one human detection frame and at least one package detection frame; A trajectory tracking module, configured to track the motion trajectories of the human detection frame and the package detection frame respectively; A second acquisition module is configured to identify, during the trajectory tracking process, for any package, a time when the package is thrown out based on the tracked current motion trajectory; a behavior recognition module, configured to obtain movement information of the package from the moment the package was thrown to the current moment, and to recognize the sorting behavior of the package based on the movement information; The current motion trajectory includes a human motion trajectory corresponding to the human detection frame and a package motion trajectory corresponding to the package detection frame, wherein the second acquisition module is further configured to: Acquire the first position information of the target package detection frame on each image frame from the target package motion trajectory in order from late to early; Acquire second position information of the human body detection frame on the same image frame from the human body motion trajectory; When there is only one sorter, the target distance is obtained according to the first position information and the second position information at the corresponding moment; When there are multiple sorting personnel, extracting the image area marked by the second position information from the image frame corresponding to the second position information; Performing human body key point detection on the image area to obtain human hand position information; Acquire the distance from the first position information to the position information of the human hand as the target distance; The moment when the target distance is less than the distance threshold for the first time is determined and used as the moment when any package is thrown.

7. The device according to claim 6, characterized in that The second acquisition module is further configured to: The moment corresponding to the image frame in which the target distance is less than the distance threshold for the first time is taken as the moment when the package is thrown.

8. The device according to claim 6, characterized in that The behavior recognition module is further used to: Based on the movement information, a sorting force parameter of the any package is generated, and according to the sorting force parameter, a sorting force level of the sorting behavior of the any package is determined.

9. The device according to claim 8, characterized in that The motion information includes the distance value, maximum speed, average speed of any package from the moment of being thrown to the current moment, and the motion speed and acceleration of any package at each moment from the moment of being thrown to the current moment.

10. The device according to any one of claims 6 to 9, characterized in that The trajectory tracking module is further used to: Tracking the human body detection frame based on the first identification information of the human body detection frame to generate a human body motion trajectory corresponding to the human body detection frame; Based on the second identification information of the package detection frame, the package detection frame is tracked to generate a package motion trajectory corresponding to the package detection frame.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

13. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images