Barrier passing object detection method and device and electronic equipment

Abstract

The invention discloses a barrier passing object detection method and device and electronic equipment, and belongs to the technical field of image recognition. The method comprises the following steps: acquiring a to-be-identified video at the position of a target barrier; performing pedestrian detection on the to-be-identified video; under the condition that the first pedestrian detection frame and the second pedestrian detection frame of the target frame image of the to-be-identified video meet the merging condition, merging is carried out to obtain a target detection frame; performing key point identification on the target frame image and an image after the target frame image to obtain a key point set sequence corresponding to the target detection frame; inputting the key point set sequence into a first neural network model to obtain a pedestrian action detection result output by the first neural network model; wherein the first neural network model is used for sampling the key point set sequence according to a target sampling probability and identifying the action of the pedestrian in the target detection frame based on the sampled key point information, and the target sampling probability is determined based on the action frequency of the pedestrian in the target detection frame.

Description

Technical Field

[0001] The present application belongs to the field of image recognition technology, and in particular, relates to a method, device and electronic equipment for detecting objects delivered through barriers. Background Art

[0002] Fences are usually set up in public places such as subways, schools and hospitals. Fences can divide and isolate areas at the physical level. For fences that are not completely enclosed, pedestrians on both sides of the fence can pass items through the gaps in the fence. When items are passed in or out of the fenced place, it may cause certain economic losses or endanger public safety. Therefore, it is necessary to detect the behavior of passing items through the fence.

[0003] Currently, simple logical analysis of the status of pedestrians on both sides of the fence is usually performed to detect fence handover behavior, and the accuracy of the detection result is low. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a method, device and electronic device for detecting objects delivered through a fence, which can improve the accuracy of the detection result of objects delivered through a fence.

[0005] In a first aspect, the present application provides a method for detecting a barrier object, the method comprising:

[0006] Obtain the video to be identified at the location of the target fence;

[0007] Performing pedestrian detection on the video to be identified;

[0008] When a first pedestrian detection frame and a second pedestrian detection frame appear in the target frame image of the video to be identified and meet a merging condition, merging the first pedestrian detection frame and the second pedestrian detection frame to obtain a target detection frame, wherein the first pedestrian detection frame includes pedestrians on one side of the target barrier, and the second pedestrian detection frame includes pedestrians on the other side of the target barrier;

[0009] Performing key point recognition on the target frame image and the image after the target frame image in the video to be recognized to obtain a key point set sequence corresponding to the target detection frame;

[0010] Inputting the key point set sequence into a first neural network model to obtain a pedestrian action detection result output by the first neural network model, wherein the pedestrian action detection result is used to characterize whether pedestrians on both sides of the target barrier have performed an object-handling behavior;

[0011] Among them, the first neural network model is used to sample the key point set sequence according to the target sampling probability, and identify the actions of pedestrians in the target detection frame based on the key point information obtained by sampling, and the target sampling probability is determined based on the frequency of pedestrian actions in the target detection frame.

[0012] According to the method for detecting objects being handed over through barriers of the present application, a key point set sequence corresponding to a target detection frame is input into a first neural network model, and the first neural network model samples the key point set sequence according to the target sampling probability to obtain key point information, and the actions of pedestrians on both sides of the target barrier in the target detection frame are identified based on the key point information, thereby determining whether the pedestrians on both sides of the target barrier have performed any object handing behavior. The detection result of object handing over through barriers using the first neural network model is more robust, and the first neural network model can adjust the sampling probability according to the frequency of pedestrian actions in the target detection frame. The obtained key point information can more completely represent the actions performed by pedestrians, and the accuracy of object handing over through barriers detection is high.

[0013] According to one embodiment of the present application, the target sampling probability is determined based on the following steps:

[0014] Dividing the key point set sequence into a target number of key point set subsequences;

[0015] Determining the pedestrian action frequency corresponding to the key point set subset based on the number of key point sets in the key point set subset sequence;

[0016] Based on the pedestrian action frequency, the target sampling probability corresponding to the key point set subset sequence is determined, and the target sampling probability is positively correlated with the pedestrian action frequency.

[0017] According to an embodiment of the present application, dividing the key point set sequence into a target number of key point set subsequences includes:

[0018] Calculating the position of a pedestrian in each key point image, wherein the pedestrian position is determined based on the key point coordinates of the key point image, and each key point set in the key point set sequence corresponds to one key point image;

[0019] Based on the maximum and minimum values ​​of the plurality of pedestrian positions, the key point set sequence is divided into a target number of key point set sub-sequences.

[0020] According to an embodiment of the present application, the first neural network model includes a first neural network sub-model and a second neural network sub-model, the first neural network sub-model is used to identify the hand approaching behavior of pedestrians on both sides, and the second neural network sub-model is used to identify the handshake behavior of pedestrians on both sides, and the key point set sequence is input into the first neural network model to obtain the pedestrian action detection result output by the first neural network model, including:

[0021] Inputting the key point set sequence into the first neural network sub-model to obtain a first pedestrian action detection result output by the first neural network sub-model;

[0022] When the first pedestrian action detection result is that pedestrians on both sides have a hand approaching behavior, inputting the key point set sequence into the second neural network sub-model to obtain a second pedestrian action detection result output by the second neural network sub-model;

[0023] When the second pedestrian action detection result is that the pedestrians on both sides do not shake hands, the pedestrian action detection result output by the first neural network model is obtained, and the pedestrian action detection result is that the pedestrians on both sides pass objects.

[0024] According to an embodiment of the present application, the performing pedestrian detection on the video to be identified includes:

[0025] Determine the position of the target barrier in the video to be identified;

[0026] Based on the position of the target barrier, determining the detection area of ​​the video to be identified;

[0027] The pedestrian detection is performed in the detection area.

[0028] According to an embodiment of the present application, when a first pedestrian detection frame and a second pedestrian detection frame appear in a target frame image of the video to be identified and a merging condition is satisfied, merging the first pedestrian detection frame and the second pedestrian detection frame includes:

[0029] When the distance between the first pedestrian detection frame and the second pedestrian detection frame is less than a target distance threshold, it is determined that the first pedestrian detection frame and the second pedestrian detection frame meet the merging condition, and the first pedestrian detection frame and the second pedestrian detection frame are merged.

[0030] In a second aspect, the present application provides a barrier object detection device, the device comprising:

[0031] An acquisition module is used to acquire the video to be identified at the location of the target fence;

[0032] A first processing module, used for performing pedestrian detection on the video to be identified;

[0033] A second processing module is configured to merge the first pedestrian detection frame and the second pedestrian detection frame to obtain a target detection frame when the first pedestrian detection frame and the second pedestrian detection frame appear in the target frame image of the video to be identified and the merging condition is satisfied, wherein the first pedestrian detection frame includes the pedestrian on one side of the target barrier, and the second pedestrian detection frame includes the pedestrian on the other side of the target barrier;

[0034] A third processing module is used to perform key point recognition on the target frame image and the image after the target frame image in the video to be recognized, so as to obtain a key point set sequence corresponding to the target detection frame;

[0035] a fourth processing module, configured to input the key point set sequence into the first neural network model to obtain a pedestrian action detection result output by the first neural network model, wherein the pedestrian action detection result is used to characterize whether pedestrians on both sides of the target barrier have performed an object-handling behavior;

[0036] Among them, the first neural network model is used to sample the key point set sequence according to the target sampling probability, and identify the actions of pedestrians in the target detection frame based on the key point information obtained by sampling, and the target sampling probability is determined based on the frequency of pedestrian actions in the target detection frame.

[0037] According to the device for detecting object delivery through barriers of the present application, a key point set sequence corresponding to a target detection frame is input into a first neural network model, and the first neural network model samples the key point set sequence according to the target sampling probability to obtain key point information, and the actions of pedestrians on both sides of the target barrier in the target detection frame are identified based on the key point information, thereby determining whether the pedestrians on both sides of the target barrier have delivered objects. The detection result of object delivery through barriers using the first neural network model is more robust, and the first neural network model can adjust the sampling probability according to the frequency of pedestrian actions in the target detection frame. The obtained key point information can more completely represent the actions performed by pedestrians, and the accuracy of object delivery through barriers detection is high.

[0038] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting objects passing through a barrier as described in the first aspect above is implemented.

[0039] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for detecting object delivery through a barrier as described in the first aspect above is implemented.

[0040] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for detecting object transfer through a barrier as described in the first aspect above.

[0041] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0043] Figure 1 This is one of the flow charts of the barrier object delivery detection method provided in the embodiment of the present application;

[0044] Figure 2 This is the second flow chart of the barrier object delivery detection method provided in the embodiment of the present application;

[0045] Figure 3 This is the third flow chart of the barrier object delivery detection method provided in the embodiment of the present application;

[0046] Figure 4 is a schematic diagram of the structure of a barrier object detection device provided in an embodiment of the present application;

[0047] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0049] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0050] The following, in conjunction with the accompanying drawings, describes in detail the barrier object delivery detection method, barrier object delivery detection device, electronic device and readable storage medium provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0051] The barrier object delivery detection method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0052] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0053] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0054] The method for detecting object delivery through a barrier provided in the embodiments of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the method for detecting object delivery through a barrier. The electronic devices mentioned in the embodiments of the present application include but are not limited to mobile phones, tablet computers, and computers. The method for detecting object delivery through a barrier provided in the embodiments of the present application is described below using an electronic device as an example of the execution entity.

[0055] like Figure 1 As shown, the barrier object delivery detection method includes: steps 110 to 150.

[0056] Step 110: Obtain the video to be identified at the location of the target fence.

[0057] The target barrier is a barrier to be inspected for object delivery, and the target barrier may be a non-completely enclosed structure, that is, pedestrians on both sides of the target barrier may pass objects through the target barrier.

[0058] The target fence can be set up in public places such as subways, schools and hospitals. A video acquisition device can be set up near the target fence. The video acquisition device is used to monitor the location of the target fence and collect video of the location of the target fence.

[0059] In this embodiment, the video to be identified is a video of the location of the target fence captured in real time by a video capture device.

[0060] Step 120: Perform pedestrian detection on the video to be identified.

[0061] Pedestrian detection is the process of identifying, locating and tracking pedestrians appearing in the video to be identified.

[0062] In this embodiment, pedestrian detection is performed on the video to be identified. When a pedestrian appears in the video to be identified, the pedestrian can be identified and located, a pedestrian detection frame corresponding to the pedestrian is generated, and the position of the pedestrian is tracked. According to the position of the pedestrian, the position of the pedestrian detection frame corresponding to the pedestrian is updated.

[0063] In actual implementation, the trained deep learning model can be used to identify and locate pedestrians in the video to be identified. For example, the neural network Yolov5 can be used to identify and locate pedestrians. The position of pedestrians can be tracked by a tracking algorithm. For example, the position of pedestrians can be tracked by a ByteTrack algorithm.

[0064] like Figure 2 As shown, the detected pedestrian detection box Pbox is input into the ByteTrack algorithm to determine whether the pedestrian id in the pedestrian detection box appears for the first time.

[0065] When the pedestrian in the pedestrian detection frame appears for the first time, the pedestrian is targeted and a tracking trajectory corresponding to the pedestrian target is created; when the pedestrian in the pedestrian detection frame does not appear for the first time, the motion trajectory of the pedestrian is updated and the position of the corresponding pedestrian detection frame is updated.

[0066] When a pedestrian in the pedestrian detection frame does not appear for M consecutive frames, the pedestrian's track is deleted and the corresponding pedestrian detection frame is deleted.

[0067] Step 130: When the first pedestrian detection frame and the second pedestrian detection frame appear in the target frame image of the video to be identified and meet the merging condition, merge the first pedestrian detection frame and the second pedestrian detection frame to obtain the target detection frame.

[0068] The first pedestrian detection frame includes pedestrians on one side of the target barrier, and the second pedestrian detection frame includes pedestrians on the other side of the target barrier.

[0069] It can be understood that the pedestrian detection frame may include a first pedestrian detection frame and a second pedestrian detection frame, and the first pedestrian detection frame and the second pedestrian detection frame correspond to pedestrians on both sides of the target barrier respectively.

[0070] The target detection frame is a detection frame obtained by merging the first pedestrian detection frame and the second pedestrian detection frame. The target detection frame includes pedestrians in the merged first pedestrian detection frame and pedestrians in the merged second pedestrian detection frame.

[0071] In this embodiment, when the first pedestrian detection frame and the second pedestrian detection frame are detected in the video to be identified, and when the first pedestrian detection frame and the second pedestrian detection frame satisfy the merging condition, the first pedestrian detection frame and the second pedestrian detection frame satisfying the merging condition are merged.

[0072] Among them, the frame in which the first pedestrian detection frame and the second pedestrian detection frame in the video to be identified meet the merging condition is taken as the target frame, and the image corresponding to the target frame is the target frame image.

[0073] In this embodiment, the merging condition may be a condition corresponding to the spatial position relationship between the first pedestrian detection frame and the second pedestrian detection frame. For example, the distance between the first pedestrian detection frame and the second pedestrian detection frame in a set direction may be less than a preset distance threshold, or the degree of overlap between the first pedestrian detection frame and the second pedestrian detection frame may be greater than a preset overlap degree threshold, etc.

[0074] The merging condition may also be a condition obtained by combining information of other pedestrian detection frames or previous frames.

[0075] In this step, merging the first pedestrian detection frame and the second pedestrian detection frame can be performed by calculating a target detection frame based on the boundary point coordinates of the first pedestrian detection frame and the boundary point coordinates of the second pedestrian detection frame, so that the target detection frame can include the area corresponding to the first pedestrian detection frame and the area corresponding to the second pedestrian detection frame.

[0076] In this embodiment, there may be a first pedestrian detection frame and a second pedestrian detection frame that meet the merging condition, and the first pedestrian detection frame and the second pedestrian detection frame that meet the merging condition are merged to obtain a target detection frame.

[0077] There may be multiple first pedestrian detection frames and one second pedestrian detection frame that meet the merging condition, and the multiple first pedestrian detection frames and one second pedestrian detection frame that meet the merging condition are merged to obtain a target detection frame.

[0078] There may be a first pedestrian detection frame and multiple second pedestrian detection frames that meet the merging condition, and the first pedestrian detection frame and the multiple second pedestrian detection frames that meet the merging condition are merged to obtain a target detection frame.

[0079] There may be multiple first pedestrian detection frames and multiple second pedestrian detection frames that meet the merging condition, and the multiple first pedestrian detection frames and multiple second pedestrian detection frames that meet the merging condition are merged to obtain a target detection frame.

[0080] It can be understood that in the target detection frame, there may be a situation where a pedestrian on one side of the target barrier hands over an object to a pedestrian on the other side of the target barrier, there may be a situation where a pedestrian on one side of the target barrier hands over an object to multiple pedestrians on the other side of the target barrier, there may be a situation where multiple pedestrians on one side of the target barrier hands over an object to a pedestrian on the other side of the target barrier, and there may be a situation where multiple pedestrians on one side of the target barrier hands over an object to multiple pedestrians on the other side of the target barrier.

[0081] Step 140: perform key point recognition on the target frame image and the image after the target frame image in the video to be recognized, and obtain a key point set sequence corresponding to the target detection frame.

[0082] Among them, key point recognition is the process of identifying and locating the key points of pedestrians in the image. The key points of pedestrians can be the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle, etc. The key point set can include at least one key point of pedestrians on both sides of the target detection frame, and one frame of image corresponds to one key point set.

[0083] In actual implementation, key point recognition can be performed through a top-down regression posture estimation model with a residual neural network (ResNet) as the backbone network.

[0084] For example, detect the key points in each pedestrian detection box Pbox, use the topdownregression posture estimation model with ResNet as the backbone network as the human key point extractor K, and extract 17 coco key points KPs:

[0085] KPs=K(image,Pbox)

[0086] Among them, the 17 coco key points include nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

[0087] According to the two pedestrians in the target detection box pairBox, the corresponding key point set pairKPs is obtained from KPs.

[0088] In this step, key point recognition is performed on the target frame image and the image after the target frame image in the video to be identified, that is, one or more frames corresponding to the target frame. The corresponding positions of the key points of the pedestrians on both sides of the target detection frame in each frame image can be obtained, and the key point set corresponding to each frame image can be obtained. The key point sets corresponding to consecutive image frames are arranged in chronological order to form a key point set sequence.

[0089] It can be understood that the key point set sequence can be used to characterize the position changes of the key points of the pedestrians on both sides of the target detection frame, so as to infer the motion trajectory, posture changes and other information of the pedestrians on both sides of the target detection frame.

[0090] Step 150: input the key point set sequence into the first neural network model to obtain the pedestrian action detection result output by the first neural network model.

[0091] Among them, the pedestrian action detection result is used to characterize whether the pedestrians on both sides of the target barrier have delivered objects. The pedestrian action detection result can be that the pedestrians on both sides of the target barrier have delivered objects, or that the pedestrians on both sides of the target barrier have not delivered objects.

[0092] In this embodiment, the first neural network model is used to sample the key point set sequence according to the target sampling probability, and recognize the action of the pedestrian in the target detection frame based on the key point information obtained by the sampling.

[0093] Among them, the target sampling probability is determined based on the pedestrian action frequency in the target detection frame. The target sampling probability can characterize the probability of each key point set being selected when sampling the key point set sequence, and the pedestrian action frequency can characterize the speed of change of pedestrian actions on both sides of the target detection frame.

[0094] In this embodiment, the corresponding pedestrian movement frequency can be obtained by the change of the key point positions of pedestrians on both sides of the target detection frame represented by the key point set sequence over time. The greater the pedestrian movement frequency, the greater the corresponding target sampling probability.

[0095] In this step, the key point set sequence is input into the first neural network model, the first neural network model samples the key point set sequence according to the target sampling probability, selects multiple key point sets, and forms key point information through the selected multiple key point sets. The key point information can characterize the change information of the positions of each key point of the pedestrian samples on both sides of the target detection frame.

[0096] The first neural network model identifies the movement trajectories, posture changes, and mutual proximity of pedestrians on both sides of the target detection frame based on the key point information, thereby determining whether the pedestrians on both sides of the target barrier are delivering objects.

[0097] For example, the key points of F frames in the video to be identified are identified, and F key point sets pairKPs corresponding to the pedestrians on both sides of the target detection frame are obtained. The key point set sequence pairKPs_F is determined. The first neural network model can be an improved model of the neural network PoseC3D, marked as P. The first neural network model is used to detect whether the pedestrians on both sides of the target barrier have handed over objects:

[0098] ret=P(imageW,imageH,pairKPs_F)

[0099] Where imageW is the width of the image image corresponding to the video to be identified, imageH is the height of the image image, and ret is the pedestrian action detection result. A value of 1 indicates that the pedestrians on both sides of the target barrier have delivered objects, and a value of 0 indicates that the pedestrians on both sides of the target barrier have not delivered objects.

[0100] According to the method for detecting objects being handed over through barriers provided in an embodiment of the present application, a key point set sequence corresponding to a target detection frame is input into a first neural network model, and the first neural network model samples the key point set sequence according to the target sampling probability to obtain key point information, and the actions of pedestrians on both sides of the target barrier in the target detection frame are identified based on the key point information, thereby determining whether the pedestrians on both sides of the target barrier have performed any object handover behavior. The detection result of detecting objects being handed over through barriers using the first neural network model is more robust, and the first neural network model can adjust the sampling probability according to the frequency of pedestrian actions in the target detection frame. The obtained key point information can more completely represent the actions performed by pedestrians, and the accuracy of detecting objects being handed over through barriers is high.

[0101] In some embodiments, the target sampling probability is determined based on the following steps:

[0102] Divide the key point set sequence into a target number of key point set subsequences;

[0103] Based on the number of key point sets in the key point set subset sequence, determine the pedestrian action frequency corresponding to the key point set subset sequence;

[0104] Based on the pedestrian action frequency, the target sampling probability corresponding to the key point set subset sequence is determined, and the target sampling probability is positively correlated with the pedestrian action frequency.

[0105] The target number is a preset value, and the target number can be determined according to the length of the key point set sequence. A larger target number can be determined when the length of the key point set sequence is longer.

[0106] In this embodiment, the key point set sequence can be randomly divided, or a time window of fixed length can be set and the time window can be slid on the key point set sequence, each sliding generates a new key point set subsequence, or the key point set sequence can be divided according to the spatial position corresponding to each key point set in the key point set sequence.

[0107] In this embodiment, the relationship between the number of key point sets in the key point set subset and the pedestrian movement frequency can be determined according to the division method of the key point set sequence. For example, when the key point set sequence is divided based on the spatial position, it can be determined that the number of key point sets in the key point set subset is negatively correlated with the pedestrian movement frequency corresponding to the key point set subset. Therefore, based on the number of key point sets in the key point set subset, the pedestrian movement frequency corresponding to the key point set subset can be determined.

[0108] In this embodiment, a functional relationship in which the pedestrian motion frequency and the target sampling probability are positively correlated can be preset, and the target sampling probability can be obtained based on the pedestrian motion frequency and the corresponding functional relationship.

[0109] In some embodiments, dividing the key point set sequence into a target number of key point set sub-sequences includes:

[0110] Calculate the position of pedestrians in each key point image;

[0111] Based on the maximum and minimum values ​​among multiple pedestrian positions, the key point set sequence is divided into a target number of key point set subsequences.

[0112] The position of the pedestrian is determined based on the key point coordinates of the key point image, and each key point set in the key point set sequence corresponds to a key point image.

[0113] Each key point set in the key point set sequence corresponds to a frame image in the video to be identified, that is, a key point image. The key point coordinates of the key point image are the coordinates corresponding to the key point set of the key point image. The key point coordinates can be determined based on the coordinates of each key point in the key point set. For example, the key point coordinates can be obtained by summing the coordinates of each key point in the key point set.

[0114] In this embodiment, the key point coordinates of the key point image may be determined as the pedestrian position in the key point image.

[0115] In this embodiment, the pedestrian position in each key point image is calculated to obtain the maximum and minimum values ​​of the pedestrian position. The coordinate interval between the maximum and minimum values ​​is divided into a target number of coordinate sub-intervals using the maximum and minimum values ​​of the pedestrian position as interval boundaries. According to the pedestrian position in the key point image corresponding to each key point set, each key point set is assigned to each coordinate sub-interval, thereby obtaining a target number of key point set subsets.

[0116] For example, in the key point set sequence, for each key point set, the x-axis coordinates of multiple key points in the key point set are accumulated to obtain a vector X of the corresponding length of the key point set sequence. sum :

[0117]

[0118] in, represents the cumulative x-coordinate value of all key points in the key point set corresponding to the paired pedestrian in the target detection box in the key point image of the i-th frame, pairKPs_F i Characterize the paired pedestrians in the key point image of the i-th frame, pairKPs k is pair KPs_F i The kth pedestrian in the j It is a pedestrian pair KPs k The jth key point of .

[0119] Take vector X sum The maximum value V max and minimum value V min , divide it into M coordinate sub-intervals, where M is the target number and the size of a single coordinate sub-interval is:

[0120] gap_s=(V max -V min ) / M

[0121] Calculate X sum The value of the i-th element in the j-th coordinate subinterval

[0122]

[0123] X sum After the elements in are assigned to the corresponding coordinate sub-intervals, the key point sets can be assigned to the corresponding intervals to form M key point set sub-sequences.

[0124] Calculate X sum The number of statistical vectors N that fall into the coordinate subinterval s , the number of j-th coordinate subintervals The number of key point sets in the jth key point set subsequence can be characterized as:

[0125]

[0126] Calculate the probability pro of sampling a coordinate subinterval s , the calculation formula is:

[0127]

[0128] in is the sampling probability of the j-th coordinate subinterval, that is, the target sampling probability of the j-th key point set subsequence.

[0129] In this embodiment, the key point set sequence pairKPs_F can be divided into coordinate sub-intervals and divided into pros Probability sampling.

[0130] In some embodiments, the first neural network model includes a first neural network sub-model and a second neural network sub-model, the first neural network sub-model is used to identify the hand approaching behavior of pedestrians on both sides, and the second neural network sub-model is used to identify the handshake behavior of pedestrians on both sides, and the key point set sequence is input into the first neural network model to obtain the pedestrian action detection result output by the first neural network model, including:

[0131] Inputting the key point set sequence into the first neural network sub-model to obtain a first pedestrian action detection result output by the first neural network sub-model;

[0132] When the first pedestrian action detection result is that pedestrians on both sides have a hand approaching behavior, the key point set sequence is input into the second neural network sub-model to obtain a second pedestrian action detection result output by the second neural network sub-model;

[0133] When the second pedestrian action detection result shows that the pedestrians on both sides do not shake hands, the pedestrian action detection result output by the first neural network model is obtained, and the pedestrian action detection result shows that the pedestrians on both sides pass objects.

[0134] Among them, the first pedestrian action detection result can represent whether the pedestrians on both sides of the target barrier have hand approaching behavior, and the hand approaching behavior is the behavior of the hands of the pedestrians on both sides of the target barrier approaching each other, or touching each other.

[0135] The second pedestrian action detection result can characterize whether the pedestrians on both sides of the target barrier have a handshake behavior. The handshake behavior is the behavior of the pedestrians on both sides of the target barrier touching each other's hands but not passing any items.

[0136] In this embodiment, the key point set sequence is input into the first neural network sub-model, and the first neural network sub-model identifies the hand approaching behavior of pedestrians on both sides. When the first pedestrian action detection result output by the first neural network sub-model is that the pedestrians on both sides do not have any hand approaching behavior, the first neural network model can directly output the pedestrian action detection result that the pedestrians on both sides do not have any object handing behavior.

[0137] When the first pedestrian action detection result is that pedestrians on both sides are approaching each other with their hands, the key point set sequence is input into the second neural network sub-model, and the second neural network sub-model identifies the handshake behavior of pedestrians on both sides. When the second pedestrian action detection result output by the second neural network sub-model is that pedestrians on both sides are shaking hands, the first neural network model can output a pedestrian action detection result that pedestrians on both sides do not hand over objects.

[0138] When the second pedestrian action detection result shows that the pedestrians on both sides did not shake hands, the first neural network model can output a pedestrian action detection result showing that the pedestrians on both sides handed over objects.

[0139] For example, when the first pedestrian action detection result is that the pedestrians on both sides have a hand approaching behavior, the key point set sequence is input into the second neural network sub-model, the second neural network sub-model can be the neural network Yolov5, the neural network Yolov5 for handshake detection is marked as Y3, and the handshake behavior of the pedestrians on both sides is detected in the target detection box pairBox of the image image of the video to be identified:

[0140] H=Y3(image,pairBox)

[0141] In this embodiment, when it is detected that the hands of pedestrians on both sides of the target barrier are close to each other, it is further detected whether the hands of pedestrians on both sides of the target barrier are close to each other for a handshake behavior. If it is determined that it is not a handshake behavior, it is determined that the pedestrians on both sides are passing objects, which can improve the accuracy of the object passing detection results.

[0142] In some embodiments, performing pedestrian detection on a video to be identified includes:

[0143] Determine the position of the target fence in the video to be identified;

[0144] Based on the position of the target fence, determine the detection area of ​​the video to be identified;

[0145] Pedestrian detection is performed in the detection area.

[0146] In this embodiment, the position of the target fence can be detected by the neural network Yolov5.

[0147] Input the multi-frame image image included in the video to be identified into the neural network Yolov5, and the position of the target barrier barrierBox(x,y,w,h) can be obtained, where x is the horizontal coordinate of the upper left corner of the target barrier, y is the vertical coordinate of the upper left corner of the target barrier, w is the width of the target barrier, and h is the height of the target barrier. The neural network Yolov5 that detects the position of the target barrier is denoted as Y1, and the position of the target barrier is obtained:

[0148] barrierBox=Y1(image)

[0149] Determining the position of the target barrier in the video to be identified is a preprocessing stage for barrier object detection, which can be performed once every N seconds.

[0150] In this embodiment, the area corresponding to the target barrier may be enlarged to obtain a detection area of ​​the video to be identified.

[0151] For example, if the horizontal expansion factor is afsw and the vertical expansion factor is afsh, the barrier expansion area, i.e., the detection area barrierRoiBox(x r ,y r ,w r ,h r ),in

[0152] x r =xx*asfw / 2

[0153] y r =yy*asfh / 2

[0154] w r =w+w*asfw

[0155] h r =h+h*asfh

[0156] Pedestrian detection is performed in the detection area barrierRoiBox through the neural network Yolov5. The neural network Yolov5 for pedestrian detection is Y2. Pedestrians are detected within the barrierRoiBox range of the image to obtain the pedestrian detection box Pbox:

[0157] Pbox=Y2(image,barrierRoiBox)

[0158] According to the position of the target barrier barrierBox, the pedestrian detection box Pbox is divided into left and right sets. The left set includes the first pedestrian detection box boxL(x L ,y L ,w L ,h L ), the right set includes the second pedestrian detection box boxR(x R ,y R ,w R ,h R ).

[0159] In this embodiment, pedestrian detection is performed in the detection area, which can avoid detecting pedestrians who are far away from the target barrier, reduce the range of pedestrian detection, and reduce unnecessary pedestrian detection frames, thereby improving detection efficiency.

[0160] In some embodiments, when a first pedestrian detection frame and a second pedestrian detection frame appear in a target frame image of a video to be identified and meet a merging condition, merging the first pedestrian detection frame and the second pedestrian detection frame includes:

[0161] When the distance between the first pedestrian detection frame and the second pedestrian detection frame is less than the target distance threshold, it is determined that the first pedestrian detection frame and the second pedestrian detection frame meet a merging condition, and the first pedestrian detection frame and the second pedestrian detection frame are merged.

[0162] The target distance threshold may be a preset value.

[0163] For example, the first pedestrian detection frame boxL(x L ,y L ,w L ,h L ) and the second pedestrian detection box boxR(x R ,y R ,w R ,h R ) is less than the target distance threshold, satisfying the merging condition:

[0164] |(y L +h L )-(y R +h R )| <D1

[0165] x R -(x L +w L ) <D2

[0166] Where D1 and D2 are constants set based on the target distance threshold.

[0167] Merge the first pedestrian detection frame and the second pedestrian detection frame to obtain the target detection frame pairBox(x p ,y p ,w p ,h p ),in

[0168] x p =x L

[0169] y p =min(y L ,y R )

[0170] w p =x R +w R -x L

[0171] h p =max(y L +h L ,y R+h R )-y p

[0172] A specific embodiment of a method for detecting object delivery between barriers is introduced below.

[0173] like Figure 3 As shown, step 1, fence position detection, uses the neural network Yolov5 to detect the position of the target fence in the video to be identified.

[0174] Step 2: Pedestrian detection: Use the neural network Yolov5 to detect the first pedestrian detection frame and the second pedestrian detection frame on both sides of the target barrier.

[0175] Step 3: Pedestrian tracking: tracking the detected pedestrians.

[0176] Step 4: Pair the pedestrians on both sides of the barrier, analyze the positions of the pedestrians on both sides of the target barrier, and when the first pedestrian detection frame and the second pedestrian detection frame meet the merging conditions, merge the first pedestrian detection frame and the second pedestrian detection frame to obtain the target detection frame.

[0177] Step 5: Paired pedestrian key point extraction, key point recognition is performed on the target frame image and the image after the target frame image in the video to be recognized, and a key point set sequence corresponding to the target detection frame is obtained.

[0178] Step 6: Hand approach detection: Use the first neural network sub-model to detect whether pedestrians on both sides of the target barrier have hand approach behavior.

[0179] Step seven, paired pedestrian handshake detection. When it is detected in step six that the pedestrians on both sides of the target barrier are approaching with their hands, the second neural network sub-model is used to detect whether the pedestrians on both sides of the target barrier are shaking hands. If there is no handshake behavior, it is determined that an object handover behavior has occurred.

[0180] The method for detecting objects being handed over via a barrier provided in an embodiment of the present application uses a neural network to detect the position of the barrier and pedestrians, determines the pedestrians on both sides of the barrier, performs key point recognition on paired pedestrians in the target detection frame, and extracts a sequence of key point sets, thereby realizing detection of objects being handed over via a barrier based on skeletal motion recognition.

[0181] The key point set sequence is input into the neural network PoseC3D, and the neural network PoseC3D is set to sample the key point set sequence according to the target sampling probability to detect whether there is a hand approach behavior. In the case of hand approach behavior, the handshake behavior is further detected. Interference can be eliminated and the real object handover behavior can be screened out. The detection result is more accurate. Using the neural network for detection, the detection result is more robust.

[0182] When pedestrians perform human body movements, the video frames occupied by the entire process are not evenly distributed. The probability of sampling in the fast-moving stage is higher, which can more completely represent the entire movement, and the detection results are more accurate than those obtained by random sampling.

[0183] The barrier object delivery detection method provided in the embodiment of the present application can be executed by a barrier object delivery detection device. In the embodiment of the present application, the barrier object delivery detection device executing the barrier object delivery detection method is taken as an example to illustrate the barrier object delivery detection device provided in the embodiment of the present application.

[0184] The embodiment of the present application also provides a device for detecting objects delivered through a fence.

[0185] like Figure 4 As shown, the barrier object delivery detection device comprises:

[0186] An acquisition module 410 is used to acquire a video to be identified at a location of a target fence;

[0187] A first processing module 420, configured to perform pedestrian detection on the video to be identified;

[0188] The second processing module 430 is used to merge the first pedestrian detection frame and the second pedestrian detection frame to obtain a target detection frame when the first pedestrian detection frame and the second pedestrian detection frame appear in the target frame image of the video to be identified and meet the merging condition, wherein the first pedestrian detection frame includes the pedestrian on one side of the target barrier, and the second pedestrian detection frame includes the pedestrian on the other side of the target barrier;

[0189] The third processing module 440 is used to perform key point recognition on the target frame image and the image after the target frame image in the video to be recognized, and obtain a key point set sequence corresponding to the target detection frame;

[0190] The fourth processing module 450 is used to input the key point set sequence into the first neural network model to obtain the pedestrian action detection result output by the first neural network model, and the pedestrian action detection result is used to characterize whether the pedestrians on both sides of the target barrier have performed the object delivery behavior;

[0191] Among them, the first neural network model is used to sample the key point set sequence according to the target sampling probability, and identify the actions of pedestrians in the target detection frame based on the key point information obtained by sampling. The target sampling probability is determined based on the frequency of pedestrian actions in the target detection frame.

[0192] According to the device for detecting object delivery through a barrier provided in an embodiment of the present application, a key point set sequence corresponding to a target detection frame is input into a first neural network model, and the first neural network model samples the key point set sequence according to the target sampling probability to obtain key point information, and the actions of pedestrians on both sides of the target barrier in the target detection frame are identified based on the key point information, thereby determining whether the pedestrians on both sides of the target barrier have delivered objects. The detection result of object delivery through a barrier using the first neural network model is more robust, and the first neural network model can adjust the sampling probability according to the frequency of pedestrian actions in the target detection frame. The obtained key point information can more completely represent the actions performed by pedestrians, and the accuracy of object delivery through a barrier is high.

[0193] In some embodiments, the target sampling probability is determined based on the following steps:

[0194] Divide the key point set sequence into a target number of key point set subsequences;

[0195] Based on the number of key point sets in the key point set subset sequence, determine the pedestrian action frequency corresponding to the key point set subset sequence;

[0196] Based on the pedestrian action frequency, the target sampling probability corresponding to the key point set subset sequence is determined, and the target sampling probability is positively correlated with the pedestrian action frequency.

[0197] In some embodiments, the fourth processing module 450 is used to calculate the pedestrian position in each key point image, the pedestrian position is determined based on the key point coordinates of the key point image, and each key point set in the key point set sequence corresponds to a key point image;

[0198] Based on the maximum and minimum values ​​among multiple pedestrian positions, the key point set sequence is divided into a target number of key point set subsequences.

[0199] In some embodiments, the first neural network model includes a first neural network sub-model and a second neural network sub-model, the first neural network sub-model is used to identify the hand approaching behavior of pedestrians on both sides, and the second neural network sub-model is used to identify the handshake behavior of pedestrians on both sides, and the fourth processing module 450 is used to input the key point set sequence into the first neural network sub-model to obtain the first pedestrian action detection result output by the first neural network sub-model;

[0200] When the first pedestrian action detection result is that pedestrians on both sides have a hand approaching behavior, the key point set sequence is input into the second neural network sub-model to obtain a second pedestrian action detection result output by the second neural network sub-model;

[0201] When the second pedestrian action detection result shows that the pedestrians on both sides do not shake hands, the pedestrian action detection result output by the first neural network model is obtained, and the pedestrian action detection result shows that the pedestrians on both sides pass objects.

[0202] In some embodiments, the first processing module 420 is used to determine the position of the target barrier in the video to be identified;

[0203] Based on the position of the target fence, determine the detection area of ​​the video to be identified;

[0204] Pedestrian detection is performed in the detection area.

[0205] In some embodiments, the second processing module 430 is used to determine that the first pedestrian detection frame and the second pedestrian detection frame meet the merging condition when the distance between the first pedestrian detection frame and the second pedestrian detection frame is less than the target distance threshold, and merge the first pedestrian detection frame and the second pedestrian detection frame.

[0206] The barrier object detection device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a palmtop computer, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0207] The barrier object detection device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0208] The barrier object detection device provided in the embodiment of the present application can achieve Figures 1 to 3 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0209] In some embodiments, Figure 5 As shown, an embodiment of the present application further provides an electronic device 500, including a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, each process of the above-mentioned barrier object delivery detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0210] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0211] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned barrier object delivery detection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0212] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0213] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned barrier object delivery detection method when executed by a processor.

[0214] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0215] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned barrier object detection method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0216] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0217] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0218] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0219] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0220] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0221] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for detecting a barrier object, characterized in that: include: Obtain the video to be identified at the location of the target fence; Performing pedestrian detection on the video to be identified; When a first pedestrian detection frame and a second pedestrian detection frame appear in the target frame image of the video to be identified and meet a merging condition, merging the first pedestrian detection frame and the second pedestrian detection frame to obtain a target detection frame, wherein the first pedestrian detection frame includes pedestrians on one side of the target barrier, and the second pedestrian detection frame includes pedestrians on the other side of the target barrier; Performing key point recognition on the target frame image and the image after the target frame image in the video to be recognized to obtain a key point set sequence corresponding to the target detection frame; Inputting the key point set sequence into a first neural network model to obtain a pedestrian action detection result output by the first neural network model, wherein the pedestrian action detection result is used to characterize whether pedestrians on both sides of the target barrier have performed an object-handling behavior; Among them, the first neural network model is used to sample the key point set sequence according to the target sampling probability, and identify the actions of pedestrians in the target detection frame based on the key point information obtained by sampling, and the target sampling probability is determined based on the frequency of pedestrian actions in the target detection frame.

2. The method for detecting object transfer through a barrier according to claim 1, characterized in that: The target sampling probability is determined based on the following steps: Dividing the key point set sequence into a target number of key point set subsequences; Determining the pedestrian action frequency corresponding to the key point set subset based on the number of key point sets in the key point set subset sequence; Based on the pedestrian action frequency, the target sampling probability corresponding to the key point set subset sequence is determined, and the target sampling probability is positively correlated with the pedestrian action frequency.

3. The method for detecting object transfer through a barrier according to claim 2, characterized in that: The step of dividing the key point set sequence into a target number of key point set subsequences comprises: Calculating the position of a pedestrian in each key point image, wherein the pedestrian position is determined based on the key point coordinates of the key point image, and each key point set in the key point set sequence corresponds to one key point image; Based on the maximum and minimum values ​​of the plurality of pedestrian positions, the key point set sequence is divided into a target number of key point set sub-sequences.

4. The method for detecting object transfer through a barrier according to claim 1, characterized in that: The first neural network model includes a first neural network sub-model and a second neural network sub-model, the first neural network sub-model is used to identify the hand approaching behavior of pedestrians on both sides, and the second neural network sub-model is used to identify the handshake behavior of pedestrians on both sides, and the key point set sequence is input into the first neural network model to obtain the pedestrian action detection result output by the first neural network model, including: Inputting the key point set sequence into the first neural network sub-model to obtain a first pedestrian action detection result output by the first neural network sub-model; When the first pedestrian action detection result is that pedestrians on both sides have a hand approaching behavior, inputting the key point set sequence into the second neural network sub-model to obtain a second pedestrian action detection result output by the second neural network sub-model; When the second pedestrian action detection result is that the pedestrians on both sides do not shake hands, the pedestrian action detection result output by the first neural network model is obtained, and the pedestrian action detection result is that the pedestrians on both sides pass objects.

5. The method for detecting object transfer through a barrier according to any one of claims 1 to 4, characterized in that: The performing pedestrian detection on the video to be identified includes: Determine the position of the target barrier in the video to be identified; Based on the position of the target barrier, determining the detection area of ​​the video to be identified; The pedestrian detection is performed in the detection area.

6. The method for detecting object transfer through a barrier according to any one of claims 1 to 4, characterized in that: When the first pedestrian detection frame and the second pedestrian detection frame appear in the target frame image of the video to be identified and meet the merging condition, merging the first pedestrian detection frame and the second pedestrian detection frame includes: When the distance between the first pedestrian detection frame and the second pedestrian detection frame is less than a target distance threshold, it is determined that the first pedestrian detection frame and the second pedestrian detection frame meet the merging condition, and the first pedestrian detection frame and the second pedestrian detection frame are merged.

7. A barrier object detection device, characterized in that: include: An acquisition module is used to acquire the video to be identified at the location of the target fence; A first processing module, used for performing pedestrian detection on the video to be identified; A second processing module is configured to merge the first pedestrian detection frame and the second pedestrian detection frame to obtain a target detection frame when the first pedestrian detection frame and the second pedestrian detection frame appear in the target frame image of the video to be identified and the merging condition is satisfied, wherein the first pedestrian detection frame includes the pedestrian on one side of the target barrier, and the second pedestrian detection frame includes the pedestrian on the other side of the target barrier; A third processing module is used to perform key point recognition on the target frame image and the image after the target frame image in the video to be recognized, so as to obtain a key point set sequence corresponding to the target detection frame; a fourth processing module, configured to input the key point set sequence into the first neural network model to obtain a pedestrian action detection result output by the first neural network model, wherein the pedestrian action detection result is used to characterize whether pedestrians on both sides of the target barrier have performed an object-handling behavior; Among them, the first neural network model is used to sample the key point set sequence according to the target sampling probability, and identify the actions of pedestrians in the target detection frame based on the key point information obtained by sampling, and the target sampling probability is determined based on the frequency of pedestrian actions in the target detection frame.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the barrier object delivery detection method as described in any one of claims 1-6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting object delivery through a barrier as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the barrier object delivery detection method as described in any one of claims 1-6 is implemented.

Images