Fence delivery detection method and device, and electronic equipment

Abstract

The application discloses a fence object delivery detection method and device and electronic equipment, and belongs to the technical field of image recognition. The method comprises the following steps: acquiring a to-be-recognized video at a position of a target fence; performing pedestrian detection on the to-be-recognized video; in the case that a first pedestrian detection box and a second pedestrian detection box of a target frame image of the to-be-recognized video meet merging conditions, performing merging to obtain a target detection box; performing key point recognition on the target frame image and images after the target frame image to obtain a key point set sequence corresponding to the target detection box; inputting the key point set sequence into a first neural network model to obtain a pedestrian action detection result output by the first neural network model; wherein the first neural network model is used for sampling the key point set sequence according to a target sampling probability, and identifying the action of a pedestrian in the target detection box based on key point information obtained through sampling, and the target sampling probability is determined based on a pedestrian action frequency in the target detection box.

Description

Technical Field

[0001] The present application belongs to the field of image recognition technology, and in particular relates to a method, device and electronic equipment for detecting objects delivered through a barrier. Background Art

[0002] Fences are often installed in public places such as subways, schools, and hospitals. Fences can physically divide and isolate areas. For fences that are not completely enclosed, pedestrians on both sides can pass objects through the gaps in the fence. When objects are passed into or out of fenced areas, it may cause certain economic losses or endanger public safety. Therefore, it is necessary to detect the behavior of passing objects across the fence.

[0003] Currently, simple logical analysis of the status of pedestrians on both sides of the fence is usually performed to detect fence handover behavior, and the accuracy of the detection results is relatively low. Summary of the Invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a method, device and electronic device for detecting object delivery through a barrier, which can improve the accuracy of the detection results of object delivery through a barrier.

[0005] In a first aspect, the present application provides a method for detecting a barrier object, the method comprising:

[0006] Obtain the video to be identified at the location of the target fence;

[0007] Performing pedestrian detection on the video to be identified;

[0008] When a first pedestrian detection frame and a second pedestrian detection frame appear in the target frame image of the video to be identified and a merging condition is satisfied, merging the first pedestrian detection frame and the second pedestrian detection frame to obtain a target detection frame, where the first pedestrian detection frame includes the pedestrian on one side of the target barrier, and the second pedestrian detection frame includes the pedestrian on the other side of the target barrier;

[0009] Performing key point recognition on the target frame image and an image following the target frame image in the video to be recognized to obtain a key point set sequence corresponding to the target detection frame;

[0010] Inputting the key point set sequence into a first neural network model to obtain a pedestrian action detection result output by the first neural network model, wherein the pedestrian action detection result is used to characterize whether pedestrians on both sides of the target barrier have performed an object-handling behavior;

[0011] Among them, the first neural network model is used to sample the key point set sequence according to the target sampling probability, and identify the actions of pedestrians in the target detection frame based on the key point information obtained by sampling. The target sampling probability is determined based on the frequency of pedestrian actions in the target detection frame.

[0012] According to the barrier object delivery detection method of the present application, the key point set sequence corresponding to the target detection frame is input into the first neural network model, the first neural network model samples the key point set sequence according to the target sampling probability to obtain key point information, and the actions of pedestrians on both sides of the target barrier in the target detection frame are identified based on the key point information, so as to determine whether the pedestrians on both sides of the target barrier have performed object delivery behavior. The detection results of barrier object delivery detection using the first neural network model are more robust, and the first neural network model can adjust the sampling probability according to the pedestrian action frequency in the target detection frame. The key point information obtained can more completely represent the actions performed by the pedestrian, and the accuracy of barrier object delivery detection is high.

[0013] According to one embodiment of the present application, the target sampling probability is determined based on the following steps:

[0014] Dividing the key point set sequence into a target number of key point set subsequences;

[0015] Determining the pedestrian action frequency corresponding to the key point set subset based on the number of key point sets in the key point set subset;

[0016] Based on the pedestrian action frequency, the target sampling probability corresponding to the key point set subset is determined, and the target sampling probability is positively correlated with the pedestrian action frequency.

[0017] According to one embodiment of the present application, dividing the key point set sequence into a target number of key point set subsequences includes:

[0018] Calculating a pedestrian position in each key point image, wherein the pedestrian position is determined based on key point coordinates of the key point image, and each key point set in the key point set sequence corresponds to one key point image;

[0019] Based on the maximum and minimum values ​​of the plurality of pedestrian positions, the key point set sequence is divided into a target number of key point set subsequences.

[0020] According to an embodiment of the present application, the first neural network model comprises a first neural network sub-model and a second neural network sub-model, the first neural network sub-model is configured to identify hand-approaching behavior of two pedestrians, and the second neural network sub-model is configured to identify handshaking behavior of two pedestrians. The inputting of the sequence of sets of key points into the first neural network model to obtain the pedestrian action detection result output by the first neural network model comprises:

[0021] The sequence of sets of key points is inputted into the first neural network sub-model to obtain the first pedestrian action detection result output by the first neural network sub-model.

[0022] In a case where the first pedestrian action detection result indicates that the two pedestrians have hand-approaching behavior, the sequence of sets of key points is inputted into the second neural network sub-model to obtain the second pedestrian action detection result output by the second neural network sub-model.

[0023] In a case where the second pedestrian action detection result indicates that the two pedestrians do not have handshaking behavior, the pedestrian action detection result output by the first neural network model is obtained, and the pedestrian action detection result indicates that the two pedestrians have object-passing behavior.

[0024] According to an embodiment of the present application, the pedestrian detection on the to-be-identified video comprises:

[0025] Determining the position of the target barrier in the to-be-identified video.

[0026] Based on the position of the target barrier, determining a detection region of the to-be-identified video.

[0027] Performing the pedestrian detection in the detection region.

[0028] According to an embodiment of the present application, in a case where the first pedestrian detection box and the second pedestrian detection box in the target frame image of the to-be-identified video meet merging conditions, the first pedestrian detection box and the second pedestrian detection box are merged, which comprises:

[0029] In a case where the distance between the first pedestrian detection box and the second pedestrian detection box is less than a target distance threshold, it is determined that the first pedestrian detection box and the second pedestrian detection box meet the merging conditions, and the first pedestrian detection box and the second pedestrian detection box are merged.

[0030] In a second aspect, the present application provides a barrier object-passing detection device, which comprises:

[0031] An acquisition module configured to acquire a to-be-identified video at a position of a target barrier.

[0032] A first processing module is used to perform pedestrian detection on the video to be identified;

[0033] a second processing module configured to, when a first pedestrian detection frame and a second pedestrian detection frame appear in a target frame image of the video to be identified and a merging condition is satisfied, merge the first pedestrian detection frame and the second pedestrian detection frame to obtain a target detection frame, wherein the first pedestrian detection frame includes the pedestrian on one side of the target barrier, and the second pedestrian detection frame includes the pedestrian on the other side of the target barrier;

[0034] A third processing module is configured to perform key point recognition on the target frame image and an image subsequent to the target frame image in the video to be recognized, to obtain a key point set sequence corresponding to the target detection frame;

[0035] a fourth processing module, configured to input the key point set sequence into a first neural network model to obtain a pedestrian action detection result output by the first neural network model, wherein the pedestrian action detection result is used to indicate whether pedestrians on both sides of the target barrier have performed an object-handling behavior;

[0036] Among them, the first neural network model is used to sample the key point set sequence according to the target sampling probability, and identify the actions of pedestrians in the target detection frame based on the key point information obtained by sampling. The target sampling probability is determined based on the frequency of pedestrian actions in the target detection frame.

[0037] According to the barrier object delivery detection device of the present application, the key point set sequence corresponding to the target detection frame is input into the first neural network model, the first neural network model samples the key point set sequence according to the target sampling probability to obtain key point information, and the actions of pedestrians on both sides of the target barrier in the target detection frame are identified based on the key point information, so as to determine whether the pedestrians on both sides of the target barrier have performed object delivery behavior. The detection result of barrier object delivery detection using the first neural network model is more robust, and the first neural network model can adjust the sampling probability according to the pedestrian action frequency in the target detection frame. The key point information obtained can more completely represent the actions performed by the pedestrian, and the accuracy of barrier object delivery detection is high.

[0038] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting objects passing through a barrier as described in the first aspect above is implemented.

[0039] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the barrier object delivery detection method as described in the first aspect above.

[0040] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method for detecting object delivery through a barrier as described in the first aspect above.

[0041] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0043] Figure 1 This is one of the flow charts of the barrier object detection method provided in the embodiment of the present application;

[0044] Figure 2 This is the second flow chart of the barrier object delivery detection method provided in the embodiment of the present application;

[0045] Figure 3 This is the third flow chart of the barrier object delivery detection method provided in the embodiment of the present application;

[0046] Figure 4 Schematic diagram of the structure of the barrier object detection device provided in an embodiment of the present application;

[0047] Figure 5 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0049] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0050] The application will be described in detail below with reference to the accompanying drawings and specific embodiments and application scenarios thereof.

[0051] The barrier delivery detection method can be applied to a terminal, and can be executed by hardware or software in the terminal.

[0052] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or a tablet computer having a touch-sensitive surface (for example, a touch screen display and / or a touchpad). It should also be understood that in some embodiments, the terminal can not be a portable communication device, but a desktop computer having a touch-sensitive surface (for example, a touch screen display and / or a touchpad).

[0053] In the following embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal can include one or more other physical user interface devices such as physical keyboards, mice, and joysticks.

[0054] The barrier delivery detection method provided by the application can be executed by an electronic device or a functional module or functional entity in the electronic device capable of implementing the barrier delivery detection method. The electronic device mentioned in the application includes, but is not limited to, a mobile phone, a tablet computer, a computer, and the like. The following will take the electronic device as an example to describe the barrier delivery detection method provided by the application.

[0055] As shown in Figure 1 The barrier delivery detection method includes steps 110-150.

[0056] Step 110, obtaining a to-be-recognized video of a position where a target barrier is located.

[0057] The target barrier is a barrier to be detected for delivery. The target barrier can be a non-fully closed structure, that is, pedestrians on both sides of the target barrier can pass through the target barrier to deliver goods.

[0058] The target barrier can be arranged in public places such as subways, schools, and hospitals. A video acquisition device can be arranged near the target barrier, and the video acquisition device is used to monitor the position where the target barrier is located and acquire a video of the position where the target barrier is located.

[0059] In this embodiment, the to-be-recognized video is a video of the position where the target barrier is located, which is acquired in real time by the video acquisition device.

[0060] Step 120, performing pedestrian detection on the to-be-recognized video.

[0061] Pedestrian detection is the process of identifying, locating, and tracking pedestrians that appear in the video to be identified.

[0062] In this embodiment, pedestrian detection is performed on the video to be identified. When a pedestrian appears in the video to be identified, the pedestrian can be identified and located, a pedestrian detection frame corresponding to the pedestrian is generated, and the position of the pedestrian is tracked. According to the position of the pedestrian, the position of the pedestrian detection frame corresponding to the pedestrian is updated.

[0063] In actual implementation, a trained deep learning model can be used to identify and locate pedestrians in the video to be identified. For example, the neural network Yolov5 can be used to identify and locate pedestrians. The position of pedestrians can be tracked by a tracking algorithm. For example, the ByteTrack algorithm can be used to track the position of pedestrians.

[0064] like Figure 2 As shown, the detected pedestrian detection box Pbox is input into the ByteTrack algorithm to determine whether the pedestrian ID in the pedestrian detection box appears for the first time.

[0065] When the pedestrian appears in the pedestrian detection frame for the first time, the pedestrian is targeted and a tracking trajectory corresponding to the pedestrian target is created; when the pedestrian does not appear in the pedestrian detection frame for the first time, the pedestrian's motion trajectory is updated and the position of the corresponding pedestrian detection frame is updated.

[0066] If a pedestrian in the pedestrian detection frame does not appear for M consecutive frames, the pedestrian's track is deleted and the corresponding pedestrian detection frame is deleted.

[0067] Step 130: When the first pedestrian detection frame and the second pedestrian detection frame appear in the target frame image of the video to be identified and meet the merging condition, merge the first pedestrian detection frame and the second pedestrian detection frame to obtain the target detection frame.

[0068] The first pedestrian detection frame includes pedestrians on one side of the target barrier, and the second pedestrian detection frame includes pedestrians on the other side of the target barrier.

[0069] It can be understood that the pedestrian detection frame may include a first pedestrian detection frame and a second pedestrian detection frame, and the first pedestrian detection frame and the second pedestrian detection frame correspond to pedestrians on both sides of the target barrier respectively.

[0070] The target detection frame is a detection frame obtained by merging the first pedestrian detection frame and the second pedestrian detection frame. The target detection frame includes the pedestrian in the merged first pedestrian detection frame and the pedestrian in the merged second pedestrian detection frame.

[0071] In this embodiment, when the first pedestrian detection frame and the second pedestrian detection frame are detected in the video to be identified, and the first pedestrian detection frame and the second pedestrian detection frame satisfy the merging condition, the first pedestrian detection frame and the second pedestrian detection frame satisfying the merging condition are merged.

[0072] Among them, the frame in which the first pedestrian detection frame and the second pedestrian detection frame in the video to be identified meet the merging conditions is taken as the target frame, and the image corresponding to the target frame is the target frame image.

[0073] In this embodiment, the merging condition may be a condition corresponding to the spatial position relationship between the first pedestrian detection frame and the second pedestrian detection frame. For example, the distance between the first pedestrian detection frame and the second pedestrian detection frame in a set direction may be less than a preset distance threshold, or the degree of overlap between the first pedestrian detection frame and the second pedestrian detection frame may be greater than a preset overlap threshold, etc.

[0074] The merging condition may also be a condition obtained by combining information of other pedestrian detection frames or previous frames.

[0075] In this step, merging the first pedestrian detection frame and the second pedestrian detection frame can be performed by calculating the target detection frame based on the boundary point coordinates of the first pedestrian detection frame and the boundary point coordinates of the second pedestrian detection frame, so that the target detection frame can include the area corresponding to the first pedestrian detection frame and the area corresponding to the second pedestrian detection frame.

[0076] In this embodiment, there may be a first pedestrian detection frame and a second pedestrian detection frame that meet the merging condition. The first pedestrian detection frame and the second pedestrian detection frame that meet the merging condition are merged to obtain a target detection frame.

[0077] There may be multiple first pedestrian detection frames and one second pedestrian detection frame that meet the merging condition. The multiple first pedestrian detection frames and one second pedestrian detection frame that meet the merging condition are merged to obtain a target detection frame.

[0078] There may be a first pedestrian detection frame and multiple second pedestrian detection frames that meet the merging condition. The first pedestrian detection frame and the multiple second pedestrian detection frames that meet the merging condition are merged to obtain a target detection frame.

[0079] There may be multiple first pedestrian detection frames and multiple second pedestrian detection frames that meet the merging condition. The multiple first pedestrian detection frames and multiple second pedestrian detection frames that meet the merging condition are merged to obtain a target detection frame.

[0080] It can be understood that in the target detection frame, there may be a situation where a pedestrian on one side of the target barrier hands over an object to a pedestrian on the other side of the target barrier, there may be a situation where a pedestrian on one side of the target barrier hands over an object to multiple pedestrians on the other side of the target barrier, there may be a situation where multiple pedestrians on one side of the target barrier hands over an object to a pedestrian on the other side of the target barrier, and there may be a situation where multiple pedestrians on one side of the target barrier hands over an object to multiple pedestrians on the other side of the target barrier.

[0081] Step 140 : perform key point recognition on the target frame image and the image following the target frame image in the video to be recognized, and obtain a key point set sequence corresponding to the target detection frame.

[0082] Among them, key point recognition is the process of identifying and locating the key points of pedestrians in the image. The key points of pedestrians can be the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle, etc. The key point set can include at least one key point of pedestrians on both sides of the target detection frame, and one frame of image corresponds to one key point set.

[0083] In actual implementation, key point recognition can be performed through a top-down regression pose estimation model with a residual neural network (ResNet) as the backbone network.

[0084] For example, detect the key points in each pedestrian detection box Pbox, use the topdownregression posture estimation model with ResNet as the backbone network as the human key point extractor K, and extract 17 coco key points KPs:

[0085] KPs=K(image,Pbox)

[0086] Among them, the 17 coco key points include nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

[0087] According to the two pedestrians in the target detection box pairBox, the corresponding key point set pairKPs is obtained from KPs.

[0088] In this step, key point recognition is performed on the target frame image and the image after the target frame image in the video to be identified, that is, one or more frames corresponding to the target frame. The corresponding positions of the key points of the pedestrians on both sides of the target detection frame in each frame image can be obtained, and the key point set corresponding to each frame image can be obtained. The key point sets corresponding to consecutive image frames are arranged in chronological order to form a key point set sequence.

[0089] It can be understood that the key point set sequence can be used to characterize the position changes of the key points of the pedestrians on both sides of the target detection frame, so as to infer the motion trajectory, posture changes and other information of the pedestrians on both sides of the target detection frame.

[0090] Step 150: Input the key point set sequence into the first neural network model to obtain the pedestrian action detection result output by the first neural network model.

[0091] Among them, the pedestrian action detection result is used to characterize whether the pedestrians on both sides of the target barrier have delivered objects. The pedestrian action detection result can be that the pedestrians on both sides of the target barrier have delivered objects, or that the pedestrians on both sides of the target barrier have not delivered objects.

[0092] In this embodiment, the first neural network model is used to sample the key point set sequence according to the target sampling probability, and recognize the actions of pedestrians in the target detection frame based on the key point information obtained by sampling.

[0093] Among them, the target sampling probability is determined based on the pedestrian movement frequency in the target detection frame. The target sampling probability can represent the probability of each key point set being selected when sampling the key point set sequence, and the pedestrian movement frequency can represent the speed of change of pedestrian movements on both sides of the target detection frame.

[0094] In this embodiment, the corresponding pedestrian movement frequency can be obtained by changing the key point positions of pedestrians on both sides of the target detection frame represented by the key point set sequence over time. The greater the pedestrian movement frequency, the greater the corresponding target sampling probability.

[0095] In this step, the key point set sequence is input into the first neural network model. The first neural network model samples the key point set sequence according to the target sampling probability, selects multiple key point sets, and forms key point information through the selected multiple key point sets. The key point information can represent the change information of the positions of each key point of the pedestrian samples on both sides of the target detection frame.

[0096] The first neural network model identifies the motion trajectories, posture changes, and mutual proximity of pedestrians on both sides of the target detection frame based on key point information, thereby determining whether the pedestrians on both sides of the target barrier are handing over objects.

[0097] For example, perform key point recognition on F frames in the video to be identified, obtain F key point sets pairKPs corresponding to the pedestrians on both sides of the target detection frame, determine the key point set sequence pairKPs_F, and the first neural network model can be an improved model of the neural network PoseC3D, marked as P. The first neural network model is used to detect whether the pedestrians on both sides of the target barrier have handed over objects:

[0098] ret=P(imageW,imageH,pairKPs_F)

[0099] Where imageW is the width of the image corresponding to the video to be identified, imageH is the height of the image, and ret is the pedestrian action detection result. A value of 1 indicates that the pedestrians on both sides of the target barrier have passed objects, and a value of 0 indicates that the pedestrians on both sides of the target barrier have not passed objects.

[0100] According to the method for detecting object delivery through a barrier provided in an embodiment of the present application, a key point set sequence corresponding to a target detection frame is input into a first neural network model, and the first neural network model samples the key point set sequence according to the target sampling probability to obtain key point information. The actions of pedestrians on both sides of the target barrier in the target detection frame are identified based on the key point information, thereby determining whether the pedestrians on both sides of the target barrier have performed object delivery behavior. The detection results of object delivery through a barrier using the first neural network model are more robust, and the first neural network model can adjust the sampling probability according to the frequency of pedestrian actions in the target detection frame. The obtained key point information can more completely represent the actions performed by pedestrians, and the accuracy of object delivery through a barrier is high.

[0101] In some embodiments, the target sampling probability is determined based on the following steps:

[0102] Divide the key point set sequence into a target number of key point set subsequences;

[0103] Based on the number of key point sets in the key point set subset, determining the pedestrian action frequency corresponding to the key point set subset;

[0104] Based on the pedestrian action frequency, the target sampling probability corresponding to the key point set sequence is determined. The target sampling probability is positively correlated with the pedestrian action frequency.

[0105] The target number is a preset value, and the target number can be determined according to the length of the key point set sequence. When the length of the key point set sequence is longer, a larger target number can be determined.

[0106] In this embodiment, the key point set sequence can be randomly divided, or a time window of fixed length can be set and the time window can be slid on the key point set sequence, each sliding generating a new key point set subsequence, or the key point set sequence can be divided according to the spatial position corresponding to each key point set in the key point set sequence.

[0107] In this embodiment, the relationship between the number of key point sets in the key point set subset and the pedestrian movement frequency can be determined based on the division method of the key point set sequence. For example, when the key point set sequence is divided based on spatial position, it can be determined that the number of key point sets in the key point set subset is negatively correlated with the pedestrian movement frequency corresponding to the key point set subset. Therefore, the pedestrian movement frequency corresponding to the key point set subset can be determined based on the number of key point sets in the key point set subset.

[0108] In this embodiment, a functional relationship in which the pedestrian motion frequency and the target sampling probability are positively correlated can be preset, and the target sampling probability can be obtained based on the pedestrian motion frequency and the corresponding functional relationship.

[0109] In some embodiments, dividing the keypoint set sequence into a target number of keypoint set subsequences includes:

[0110] Calculate the pedestrian position in each keypoint image;

[0111] Based on the maximum and minimum values ​​of multiple pedestrian positions, the key point set sequence is divided into a target number of key point set subsequences.

[0112] The pedestrian position is determined based on the key point coordinates of the key point image, and each key point set in the key point set sequence corresponds to a key point image.

[0113] Each key point set in the key point set sequence corresponds to a frame image in the video to be identified, that is, a key point image. The key point coordinates of the key point image are the coordinates corresponding to the key point set of the key point image. The key point coordinates can be determined based on the coordinates of each key point in the key point set. For example, the key point coordinates can be obtained by summing the coordinates of each key point in the key point set.

[0114] In this embodiment, the key point coordinates of the key point image may be determined as the pedestrian position in the key point image.

[0115] In this embodiment, the pedestrian position in each key point image is calculated to obtain the maximum and minimum values ​​of the pedestrian position. The coordinate interval between the maximum and minimum values ​​is divided into a target number of coordinate sub-intervals with the maximum and minimum values ​​of the pedestrian position as interval boundaries. According to the pedestrian position in the key point image corresponding to each key point set, each key point set is assigned to each coordinate sub-interval, thereby obtaining a target number of key point set subsequences.

[0116] For example, in a key point set sequence, for each key point set, the x-axis coordinates of multiple key points in the key point set are accumulated to obtain a vector X of the corresponding length of the key point set sequence. sum :

[0117]

[0118] in, Indicates the cumulative x-coordinate value of all key points in the key point set corresponding to the paired pedestrian in the target detection frame in the key point image of the i-th frame, pairKPs_F i Representing paired pedestrians in the key point image of the i-th frame, pairKPs k is pairKPs_F i The kth pedestrian in the j is a pedestrian pair KPs k The jth key point of .

[0119] Take vector X sum The maximum value of V max and minimum value V min , divide it into M coordinate sub-intervals, where M is the target number and the size of a single coordinate sub-interval is:

[0120] gap_s=(V max -V min ) / M

[0121] Calculate X sum The value of the i-th element falling into the j-th coordinate subinterval

[0122]

[0123] X sum After the elements in are assigned to the corresponding coordinate subintervals, the key point sets can be assigned to the corresponding intervals to form M key point set subsequences.

[0124] Calculate X sum The number of statistical vectors N that fall into the coordinate subinterval s , the number of j-th coordinate subintervals The number of key point sets in the jth key point set subsequence can be characterized as:

[0125]

[0126] Calculate the probability pro of sampling the coordinate subinterval s , the calculation formula is:

[0127]

[0128] in is the sampling probability of the j-th coordinate subinterval, that is, the target sampling probability of the j-th key point set subsequence.

[0129] In this embodiment, the key point set sequence pairKPs_F can be divided into coordinate sub-intervals ands Sampling with probability.

[0130] In some embodiments, the first neural network model includes a first neural network sub-model and a second neural network sub-model. The first neural network sub-model is used to identify the approaching hand movements of pedestrians on both sides, and the second neural network sub-model is used to identify the handshake movements of pedestrians on both sides. The key point set sequence is input into the first neural network model to obtain the pedestrian action detection result output by the first neural network model, including:

[0131] Inputting the key point set sequence into the first neural network sub-model to obtain a first pedestrian action detection result output by the first neural network sub-model;

[0132] When the first pedestrian action detection result is that pedestrians on both sides have a hand approaching behavior, the key point set sequence is input into the second neural network sub-model to obtain a second pedestrian action detection result output by the second neural network sub-model;

[0133] When the second pedestrian action detection result shows that the pedestrians on both sides do not shake hands, the pedestrian action detection result output by the first neural network model is obtained, and the pedestrian action detection result shows that the pedestrians on both sides perform an object-handing behavior.

[0134] Among them, the first pedestrian action detection result can represent whether the pedestrians on both sides of the target barrier have hand approaching behavior, and the hand approaching behavior is the behavior of the hands of pedestrians on both sides of the target barrier approaching each other or touching each other.

[0135] The second pedestrian action detection result can indicate whether the pedestrians on both sides of the target barrier have a handshake behavior. The handshake behavior is the behavior in which the pedestrians on both sides of the target barrier touch each other's hands but do not pass any items.

[0136] In this embodiment, the key point set sequence is input into the first neural network sub-model, and the first neural network sub-model identifies the hand approaching behavior of pedestrians on both sides. When the first pedestrian action detection result output by the first neural network sub-model is that the pedestrians on both sides do not have the hand approaching behavior, the first neural network model can directly output the pedestrian action detection result that the pedestrians on both sides do not have the object handing behavior.

[0137] When the first pedestrian action detection result is that the pedestrians on both sides are approaching each other with their hands, the key point set sequence is input into the second neural network sub-model, and the second neural network sub-model recognizes the handshake behavior of the pedestrians on both sides. When the second pedestrian action detection result output by the second neural network sub-model is that the pedestrians on both sides are shaking hands, the first neural network model can output a pedestrian action detection result that the pedestrians on both sides do not pass objects.

[0138] When the second pedestrian action detection result shows that the pedestrians on both sides do not shake hands, the first neural network model can output a pedestrian action detection result showing that the pedestrians on both sides have handed over objects.

[0139] For example, when the first pedestrian action detection result is that the pedestrians on both sides have a hand-to-hand approach behavior, the key point set sequence is input into the second neural network sub-model, which can be the neural network Yolov5. The neural network Yolov5 for handshake detection is marked as Y3, and the handshake behavior of the pedestrians on both sides is detected in the target detection box pairBox of the image image of the video to be identified:

[0140] H=Y3(image,pairBox)

[0141] In this embodiment, when it is detected that the hands of pedestrians on both sides of the target barrier are close to each other, it is further detected whether the hands of pedestrians on both sides of the target barrier are close to each other as a handshake behavior. If it is determined that it is not a handshake behavior, it is determined that the pedestrians on both sides are passing objects, which can improve the accuracy of the object passing detection results.

[0142] In some embodiments, performing pedestrian detection on a video to be identified includes:

[0143] Determine the location of the target barrier in the video to be identified;

[0144] Determine the detection area of ​​the video to be identified based on the position of the target fence;

[0145] Perform pedestrian detection in the detection area.

[0146] In this embodiment, the position of the target fence can be detected by the neural network Yolov5.

[0147] Input the multi-frame image image included in the video to be identified into the neural network Yolov5, and the position of the target barrier can be obtained barrierBox(x, y, w, h), where x is the horizontal coordinate of the upper left corner of the target barrier, y is the vertical coordinate of the upper left corner of the target barrier, w is the width of the target barrier, and h is the height of the target barrier. The neural network Yolov5 that detects the position of the target barrier is denoted as Y1, and the position of the target barrier is obtained:

[0148] barrierBox=Y1(image)

[0149] Determining the position of the target barrier in the video to be identified is a preprocessing stage for barrier object detection and can be performed once every N seconds.

[0150] In this embodiment, the area corresponding to the target barrier may be expanded to obtain a detection area of ​​the video to be identified.

[0151] For example, the horizontal expansion coefficient is afsw, and the vertical expansion coefficient is afsh, and the barrier expansion area, that is, the detection area barrierRoiBox(x r ,y r ,w r ,h r ),in

[0152] x r =xx*asfw / 2

[0153] y r =yy*asfh / 2

[0154] w r =w+w*asfw

[0155] h r =h+h*asfh

[0156] Pedestrian detection is performed in the detection area barrierRoiBox using the neural network Yolov5. The neural network Yolov5 for pedestrian detection is set to Y2. Pedestrians are detected within the barrierRoiBox range of the image to obtain the pedestrian detection box Pbox:

[0157] Pbox=Y2(image,barrierRoiBox)

[0158] According to the position of the target barrier barrierBox, the pedestrian detection box Pbox is divided into left and right sets. The left set includes the first pedestrian detection box boxL(x L ,y L ,w L ,h L ), the right set includes the second pedestrian detection box boxR(x R ,y R ,w R ,h R ).

[0159] In this embodiment, pedestrian detection is performed in the detection area, which can avoid detecting pedestrians that are far away from the target barrier, reduce the range of pedestrian detection, and reduce unnecessary pedestrian detection frames, thereby improving detection efficiency.

[0160] In some embodiments, when a first pedestrian detection frame and a second pedestrian detection frame appear in a target frame image of a video to be identified and meet a merging condition, merging the first pedestrian detection frame and the second pedestrian detection frame includes:

[0161] When the distance between the first pedestrian detection frame and the second pedestrian detection frame is less than the target distance threshold, it is determined that the first pedestrian detection frame and the second pedestrian detection frame meet a merging condition, and the first pedestrian detection frame and the second pedestrian detection frame are merged.

[0162] The target distance threshold may be a preset value.

[0163] For example, the first pedestrian detection frame boxL(x L ,y L ,w L ,h L ) and the second pedestrian detection box boxR(x R ,y R ,w R ,h R ) is less than the target distance threshold, meeting the merging condition:

[0164] |(y L +h L )-(y R +h R )| <D1

[0165] x R -(x L +w L ) <D2

[0166] Where D1 and D2 are constants set based on the target distance threshold.

[0167] Merge the first pedestrian detection frame and the second pedestrian detection frame to obtain the target detection frame pairBox(x p ,y p ,w p ,h p ),in

[0168] x p =x L

[0169] y p =min(y L ,y R )

[0170] w p =x R +w R -x L

[0171] h p =max(y L +h L ,y R+h R )-y p

[0172] A specific embodiment of a method for detecting object delivery between barriers is described below.

[0173] like Figure 3 As shown, in step 1, fence position detection, the neural network Yolov5 is used to detect the position of the target fence in the video to be identified.

[0174] Step 2: Pedestrian detection: Use the neural network Yolov5 to detect the first pedestrian detection frame and the second pedestrian detection frame on both sides of the target barrier.

[0175] Step 3: Pedestrian tracking: track the detected pedestrians.

[0176] Step 4: Pair the pedestrians on both sides of the barrier and analyze the positions of the pedestrians on both sides of the target barrier. When the first pedestrian detection frame and the second pedestrian detection frame meet the merging conditions, merge the first pedestrian detection frame and the second pedestrian detection frame to obtain the target detection frame.

[0177] Step 5: Pair pedestrian key point extraction, perform key point recognition on the target frame image and the image after the target frame image in the video to be identified, and obtain the key point set sequence corresponding to the target detection frame.

[0178] Step 6: Hand approach detection: Use the first neural network sub-model to detect whether pedestrians on both sides of the target barrier have any hand approach behavior.

[0179] Step 7: Paired pedestrian handshake detection. When it is detected in step 6 that the pedestrians on both sides of the target barrier are approaching with their hands, the second neural network sub-model is used to detect whether the pedestrians on both sides of the target barrier are shaking hands. If there is no handshake behavior, it is determined that an object handover behavior has occurred.

[0180] The method for detecting object delivery between barriers provided in the embodiment of the present application uses a neural network to detect the position of barriers and pedestrians, determines the pedestrians on both sides of the barrier, performs key point recognition on paired pedestrians in the target detection frame, and extracts a sequence of key point sets, thereby realizing object delivery between barriers based on skeletal motion recognition.

[0181] The key point set sequence is input into the neural network PoseC3D, and the neural network PoseC3D is set to sample the key point set sequence according to the target sampling probability to detect whether there is a hand approach behavior. In the case of hand approach behavior, the handshake behavior is further detected. This can eliminate interference and screen out the true object handover behavior. The detection result is more accurate and the detection result is more robust when using a neural network.

[0182] When pedestrians perform human body movements, the video frames occupied by the entire process are not evenly distributed. The probability of sampling in the fast-moving stage is higher, which can more completely represent the entire movement and is more accurate than the detection results obtained by random sampling.

[0183] The embodiment of the present application provides a method for detecting object delivery between barriers, which can be performed by a detection device for object delivery between barriers. The embodiment of the present application takes the detection device for object delivery between barriers as an example to illustrate the detection method for object delivery between barriers.

[0184] The embodiment of the present application also provides a device for detecting objects delivered through a barrier.

[0185] like Figure 4 As shown, the barrier object delivery detection device includes:

[0186] An acquisition module 410 is used to acquire a video to be identified at a location of a target fence;

[0187] A first processing module 420 is configured to perform pedestrian detection on the video to be identified;

[0188] The second processing module 430 is configured to merge the first pedestrian detection frame and the second pedestrian detection frame to obtain a target detection frame when a first pedestrian detection frame and a second pedestrian detection frame appear in the target frame image of the video to be identified and meet a merging condition, where the first pedestrian detection frame includes the pedestrian on one side of the target barrier and the second pedestrian detection frame includes the pedestrian on the other side of the target barrier;

[0189] The third processing module 440 is used to perform key point recognition on the target frame image and the image after the target frame image in the video to be recognized, and obtain a key point set sequence corresponding to the target detection frame;

[0190] A fourth processing module 450 is configured to input the key point set sequence into the first neural network model to obtain a pedestrian action detection result output by the first neural network model. The pedestrian action detection result is used to indicate whether pedestrians on both sides of the target barrier are performing an object-handling behavior.

[0191] Among them, the first neural network model is used to sample the key point set sequence according to the target sampling probability, and identify the actions of pedestrians in the target detection frame based on the key point information obtained by sampling. The target sampling probability is determined based on the frequency of pedestrian actions in the target detection frame.

[0192] According to the device for detecting object delivery through a barrier provided in an embodiment of the present application, a key point set sequence corresponding to a target detection frame is input into a first neural network model. The first neural network model samples the key point set sequence according to the target sampling probability to obtain key point information. The actions of pedestrians on both sides of the target barrier in the target detection frame are identified based on the key point information, thereby determining whether the pedestrians on both sides of the target barrier have performed object delivery. The detection result of object delivery through a barrier using the first neural network model is more robust, and the first neural network model can adjust the sampling probability according to the frequency of pedestrian actions in the target detection frame. The obtained key point information can more completely represent the actions performed by pedestrians, and the accuracy of object delivery through a barrier is high.

[0193] In some embodiments, the target sampling probability is determined based on the following steps:

[0194] Divide the key point set sequence into a target number of key point set subsequences;

[0195] Based on the number of key point sets in the key point set subset, determining the pedestrian action frequency corresponding to the key point set subset;

[0196] Based on the pedestrian action frequency, the target sampling probability corresponding to the key point set sequence is determined. The target sampling probability is positively correlated with the pedestrian action frequency.

[0197] In some embodiments, the fourth processing module 450 is configured to calculate the position of a pedestrian in each keypoint image, where the pedestrian position is determined based on the keypoint coordinates of the keypoint image, and each keypoint set in the keypoint set sequence corresponds to a keypoint image;

[0198] Based on the maximum and minimum values ​​of multiple pedestrian positions, the key point set sequence is divided into a target number of key point set subsequences.

[0199] In some embodiments, the first neural network model includes a first neural network sub-model and a second neural network sub-model, the first neural network sub-model is used to identify the hand approaching behavior of pedestrians on both sides, and the second neural network sub-model is used to identify the handshake behavior of pedestrians on both sides. The fourth processing module 450 is used to input the key point set sequence into the first neural network sub-model to obtain a first pedestrian action detection result output by the first neural network sub-model;

[0200] When the first pedestrian action detection result is that pedestrians on both sides have a hand approaching behavior, the key point set sequence is input into the second neural network sub-model to obtain a second pedestrian action detection result output by the second neural network sub-model;

[0201] In a case where the second pedestrian action detection result is that the two pedestrians do not have a handshaking behavior, the pedestrian action detection result output by the first neural network model is obtained, and the pedestrian action detection result is that the two pedestrians have a handover behavior.

[0202] In some embodiments, the first processing module 420 is configured to determine a position of a target barrier in the video to be recognized.

[0203] Based on the position of the target barrier, a detection region of the video to be recognized is determined.

[0204] Pedestrian detection is performed in the detection region.

[0205] In some embodiments, the second processing module 430 is configured to, in a case where a distance between the first pedestrian detection frame and the second pedestrian detection frame is less than a target distance threshold, determine that the first pedestrian detection frame and the second pedestrian detection frame satisfy a merging condition, and merge the first pedestrian detection frame and the second pedestrian detection frame.

[0206] The barrier handover detection apparatus in the embodiments of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and the like. The electronic device can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the like. The embodiments of the present application are not limited in this regard.

[0207] The barrier handover detection apparatus in the embodiments of the present application can be a device with an operating system. The operating system can be an Android operating system, an IOS operating system, or other possible operating systems, and the embodiments of the present application are not limited in this regard.

[0208] The barrier handover detection apparatus provided in the embodiments of the present application can implement the method embodiments Figures 1 to 3 The method embodiments, and each process of the method embodiments is not repeated here to avoid repetition.

[0209] In some embodiments, as Figure 5 As shown, an embodiment of the present application further provides an electronic device 500, including a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, each process of the above-mentioned embodiment of the barrier object detection method is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0210] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0211] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned barrier object delivery detection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0212] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0213] An embodiment of the present application further provides a computer program product, including a computer program, which implements the above-mentioned barrier object delivery detection method when executed by a processor.

[0214] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0215] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned barrier object detection method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0216] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0217] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0218] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0219] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

[0220] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0221] While the embodiments of the application have been shown and described, it is to be understood that the embodiments can be varied, modified, substituted and changed by those skilled in the art without departing from the principles and spirit of the application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting objects passing through a barrier, characterized in that: include: Obtain the video to be identified at the location of the target fence; Performing pedestrian detection on the video to be identified; When a first pedestrian detection frame and a second pedestrian detection frame appear in the target frame image of the video to be identified and a merging condition is satisfied, merging the first pedestrian detection frame and the second pedestrian detection frame to obtain a target detection frame, where the first pedestrian detection frame includes the pedestrian on one side of the target barrier, and the second pedestrian detection frame includes the pedestrian on the other side of the target barrier; Performing key point recognition on the target frame image and an image following the target frame image in the video to be recognized to obtain a key point set sequence corresponding to the target detection frame; Inputting the key point set sequence into a first neural network model to obtain a pedestrian action detection result output by the first neural network model, wherein the pedestrian action detection result is used to characterize whether pedestrians on both sides of the target barrier have performed an object-handling behavior; The first neural network model is used to sample the key point set sequence according to a target sampling probability, and recognize the action of the pedestrian in the target detection frame based on the key point information obtained by the sampling, wherein the target sampling probability is determined based on the frequency of the pedestrian action in the target detection frame; The target sampling probability is determined based on the following steps: Dividing the key point set sequence into a target number of key point set subsequences; Determining the pedestrian action frequency corresponding to the key point set subset based on the number of key point sets in the key point set subset; Determining the target sampling probability corresponding to the key point set subset based on the pedestrian action frequency, wherein the target sampling probability is positively correlated with the pedestrian action frequency; The step of dividing the key point set sequence into a target number of key point set subsequences includes: Calculating a pedestrian position in each key point image, wherein the pedestrian position is determined based on key point coordinates of the key point image, and each key point set in the key point set sequence corresponds to one key point image; Based on the maximum and minimum values ​​of the plurality of pedestrian positions, the key point set sequence is divided into a target number of key point set subsequences.

2. The method for detecting object delivery through a barrier according to claim 1, wherein: The first neural network model includes a first neural network sub-model and a second neural network sub-model. The first neural network sub-model is used to identify the approaching hand behavior of pedestrians on both sides, and the second neural network sub-model is used to identify the handshake behavior of pedestrians on both sides. The key point set sequence is input into the first neural network model to obtain the pedestrian action detection result output by the first neural network model, including: Inputting the key point set sequence into the first neural network sub-model to obtain a first pedestrian action detection result output by the first neural network sub-model; When the first pedestrian action detection result is that pedestrians on both sides have engaged in hand-approaching behavior, inputting the key point set sequence into the second neural network sub-model to obtain a second pedestrian action detection result output by the second neural network sub-model; When the second pedestrian action detection result shows that the pedestrians on both sides do not shake hands, the pedestrian action detection result output by the first neural network model is obtained, and the pedestrian action detection result shows that the pedestrians on both sides pass objects.

3. The method for detecting object transfer through a barrier according to claim 1 or 2, wherein: The performing pedestrian detection on the video to be identified includes: Determining the position of the target barrier in the video to be identified; Determining a detection area of ​​the video to be identified based on the position of the target barrier; The pedestrian detection is performed in the detection area.

4. The method for detecting object transfer through a barrier according to claim 1 or 2, wherein: The merging of the first pedestrian detection frame and the second pedestrian detection frame when the first pedestrian detection frame and the second pedestrian detection frame appear in the target frame image of the video to be identified and meet the merging condition includes: When the distance between the first pedestrian detection frame and the second pedestrian detection frame is less than a target distance threshold, it is determined that the first pedestrian detection frame and the second pedestrian detection frame meet the merging condition, and the first pedestrian detection frame and the second pedestrian detection frame are merged.

5. A barrier object detection device, characterized in that: include: An acquisition module is used to acquire the video to be identified at the location of the target fence; A first processing module is used to perform pedestrian detection on the video to be identified; a second processing module configured to, when a first pedestrian detection frame and a second pedestrian detection frame appear in a target frame image of the video to be identified and a merging condition is satisfied, merge the first pedestrian detection frame and the second pedestrian detection frame to obtain a target detection frame, wherein the first pedestrian detection frame includes the pedestrian on one side of the target barrier, and the second pedestrian detection frame includes the pedestrian on the other side of the target barrier; A third processing module is configured to perform key point recognition on the target frame image and an image subsequent to the target frame image in the video to be recognized, to obtain a key point set sequence corresponding to the target detection frame; a fourth processing module, configured to input the key point set sequence into a first neural network model to obtain a pedestrian action detection result output by the first neural network model, wherein the pedestrian action detection result is used to indicate whether pedestrians on both sides of the target barrier have performed an object-handling behavior; The first neural network model is used to sample the key point set sequence according to a target sampling probability, and recognize the action of the pedestrian in the target detection frame based on the key point information obtained by the sampling, wherein the target sampling probability is determined based on the frequency of the pedestrian action in the target detection frame; The target sampling probability is determined based on the following steps: Dividing the key point set sequence into a target number of key point set subsequences; Determining the pedestrian action frequency corresponding to the key point set subset based on the number of key point sets in the key point set subset; Determining the target sampling probability corresponding to the key point set subset based on the pedestrian action frequency, wherein the target sampling probability is positively correlated with the pedestrian action frequency; a fourth processing module, configured to calculate a pedestrian position in each key point image, wherein the pedestrian position is determined based on key point coordinates of the key point image, and each key point set in the key point set sequence corresponds to one key point image; Based on the maximum and minimum values ​​of the plurality of pedestrian positions, the key point set sequence is divided into a target number of key point set subsequences.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the barrier object transfer detection method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting object transfer between barriers as described in any one of claims 1 to 4 is implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for detecting object transfer between barriers as described in any one of claims 1 to 4 is implemented.

Images