Reception event detection method and related device

Abstract

The invention discloses a reception event detection method and a related device, and the method comprises the steps: firstly obtaining the track information and behavior information of shop assistants and customers moving in a target region within a fixed time, enabling the track information to identify the position changes of each person at different times, and enabling the behavior information to identify the behavior characteristics of each person at different times; the method comprises the steps that firstly, one or more interaction events occurring in different combinations are analyzed according to the combinations of the shop assistants and the customers, finally, a pre-constructed classification model can recognize the interaction events and determine whether the interaction events are reception events or not, and therefore based on tracking information of a long-time sequence, interaction characteristics of the shop assistants and the customers can be comprehensively analyzed; therefore, the reception event can be evaluated more accurately, information such as the occurrence object and time of the reception event can be recorded accurately, confusion of the object and time of the reception event caused by single-frame image detection is avoided, and the detection accuracy is effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and particularly to a method for detecting reception events and related devices. Background Art

[0002] A reception event refers to the behavior of a store clerk receiving a customer in a store, which plays a key role in evaluating the enthusiasm of the clerk and conducting customer portrait analysis.

[0003] In related technologies, based on computer vision technology, the behavior of personnel can be detected to analyze whether a reception event has occurred.

[0004] However, the current detection method is usually based on a single-frame image for judgment, resulting in inaccurate detection results. Summary of the Invention

[0005] In view of this, this application provides a method for detecting reception events and related devices, which can realize the automatic recognition of reception events through the trajectory information and behavior information of store clerks and customers.

[0006] To solve the above problems, the technical solutions provided in this application are as follows:

[0007] On the one hand, this application provides a method for detecting reception events, and the method includes:

[0008] Obtain the trajectory information and behavior information corresponding to multiple personnel respectively, where the multiple personnel include M store clerks and N customers who are active in the target area within a fixed time, the trajectory information is used to identify the position changes of each person at different times, and the behavior information is used to identify the behavior characteristics of each person at different times;

[0009] For a target store clerk among the M store clerks and a target customer among the N customers, determine the interaction event between the target store clerk and the target customer according to the trajectory information and the behavior information;

[0010] Based on multiple interaction events among the multiple personnel, determine the reception event in the target area through a pre-constructed classification model.

[0011] In a possible implementation manner, the obtaining the trajectory information and behavior information corresponding to multiple personnel respectively includes:

[0012] Obtain the videos of multiple cameras in the target area within the fixed time;

[0013] Detect the personnel in multiple video frames of the video, and extract the human body pose key points of the personnel in each camera;

[0014] By using multi-object tracking technology, obtain the trajectory segments corresponding to the people in each camera;

[0015] According to the positions of each camera, map the human pose key points and the trajectory segments to the three-dimensional space of the target area to obtain the trajectory information and behavior information corresponding to the multiple people respectively.

[0016] In a possible implementation, the step of mapping the human pose key points and the trajectory segments to the three-dimensional space of the target area according to the positions of each camera to obtain the trajectory information and behavior information corresponding to the multiple people respectively includes:

[0017] According to the positions of each camera, map the human pose key points and the trajectory segments to the three-dimensional space of the target area to obtain the three-dimensional information of the human pose key points and the trajectory segments;

[0018] Based on the information of the human pose key points, obtain the behavior information corresponding to the multiple people respectively.

[0019] Respectively merge the trajectory segments corresponding to each person through a dynamic programming algorithm, and obtain the trajectory information corresponding to the multiple people respectively based on the three-dimensional information of the trajectory segments.

[0020] In a possible implementation, for the target clerk among the M clerks and the target customer among the N customers, determining the interaction event between the target clerk and the target customer according to the trajectory information and the behavior information includes:

[0021] For the target clerk among the M clerks and the target customer among the N customers, according to the trajectory information and behavior information, based on a preset scoring rule, obtain the interaction scores of the combinations of the target clerk and the target customer at different times, and the interaction scores are used to identify the interaction intensity;

[0022] According to the interaction scores, parse the interaction events of the combinations through a pre-constructed three-pointer algorithm. The first pointer of the three-pointer algorithm is used to identify the current moment, the second pointer is used to identify the start moment of the interaction event, and the third pointer is used to identify the end moment of the interaction event.

[0023] In a possible implementation, the preset scoring rule includes:

[0024] One or more of a distance-based scoring rule, a region-based scoring rule, and a line-of-sight interaction-based scoring rule.

[0025] In a possible implementation, the determining the reception events of the target area based on the multiple interaction events among the multiple persons by using a pre-built classification model includes:

[0026] Based on the multiple interaction pending events among the multiple persons, determining multiple scores corresponding to the multiple interaction events respectively through a pre-built classification model;

[0027] In the same time period, if a clerk has P interactive events with multiple customers who are in the same business relationship, among the scores corresponding to the P interactive events, the P-1 scores that are not the maximum values are suppressed;

[0028] The interactive events corresponding to the scores greater than the preset threshold among the multiple scores are determined as reception events of the target area.

[0029] In another aspect, the present application provides a reception event detection device, the device comprising an acquisition unit and a determination unit:

[0030] The acquisition unit is used to acquire trajectory information and behavior information corresponding to a plurality of persons, wherein the plurality of persons include M store employees and N customers active in the target area within a fixed time, the trajectory information is used to identify the position change of each person at different times, and the behavior information is used to identify the behavior characteristics of each person at different times;

[0031] The determining unit is used to determine, for a target clerk among the M clerks and a target customer among the N customers, an interaction event between the target clerk and the target customer according to the trajectory information and the behavior information;

[0032] The determining unit is further configured to determine the reception events of the target area based on the multiple interaction events among the multiple persons by using a pre-built classification model.

[0033] In a possible implementation manner, the acquiring unit is specifically configured to:

[0034] Acquire videos of multiple cameras in the target area within the fixed time;

[0035] Detecting people in multiple video frames of the video, and extracting key points of human body postures of the people in each camera;

[0036] By using multi-target tracking technology, a trajectory segment corresponding to the person in each camera is obtained;

[0037] According to the position of each camera, the key points of the human body posture and the trajectory segments are mapped to the three-dimensional space of the target area to obtain the trajectory information and behavior information corresponding to the multiple persons respectively.

[0038] In a possible implementation manner, the acquiring unit is specifically configured to:

[0039] According to the position of each camera, the key points of the human body posture and the track segments are mapped to the three-dimensional space of the target area to obtain the three-dimensional information of the key points of the human body posture and the track segments;

[0040] Based on the information of the key points of the human body posture, the behavior information corresponding to the multiple persons is obtained.

[0041] The trajectory segments corresponding to each person are merged respectively through a dynamic programming algorithm, and the trajectory information corresponding to the multiple persons is obtained based on the three-dimensional information of the trajectory segments.

[0042] In a possible implementation manner, the determining unit is specifically configured to:

[0043] For a target clerk among the M clerks and a target customer among the N customers, according to the trajectory information and the behavior information, based on a preset scoring rule, an interaction score of the combination of the target clerk and the target customer at different times is obtained, and the interaction score is used to identify the interaction intensity;

[0044] According to the interaction score, the combined interaction event is parsed by a pre-constructed three-pointer algorithm, wherein the first pointer of the three-pointer algorithm is used to identify the current time, the second pointer is used to identify the start time of the interaction event, and the third pointer is used to identify the end time of the interaction event.

[0045] In a possible implementation, the preset scoring rules include:

[0046] One or more of a distance-based scoring rule, an area-based scoring rule, and a line-of-sight interaction-based scoring rule.

[0047] In a possible implementation manner, the determining unit is specifically configured to:

[0048] Based on the multiple interaction pending events among the multiple persons, determining multiple scores corresponding to the multiple interaction events respectively through a pre-built classification model;

[0049] In the same time period, if a clerk has P interactive events with multiple customers who are in the same business relationship, among the scores corresponding to the P interactive events, the P-1 scores that are not the maximum values are suppressed;

[0050] Determine the interaction event corresponding to the score greater than the preset threshold among the multiple scores as the reception event of the target area.

[0051] In another aspect, the present application provides a computer device, which includes a processor and a memory:

[0052] The memory is used to store a computer program;

[0053] The processor is used to execute the method described in any one of the above according to the computer program.

[0054] In another aspect, the present application provides a computer-readable storage medium, which is used to store a computer program, and the computer program realizes the execution of the method described in any one of the above when executed by a computer device.

[0055] In another aspect, the present application provides a computer program product including a computer program, which when running on a computer device causes the computer device to execute the method described in any one of the above.

[0056] It can be seen from the above technical solutions that this technical solution first obtains the trajectory information and behavior information of the personnel (including store clerks and customers) active in the target area within a fixed time. The trajectory information is used to identify the position changes of each person at different times, and the behavior information identifies the behavior characteristics of each person at different times, so as to achieve global and uninterrupted tracking of each person in the target area. Then, for the combination of store clerks and customers, according to their trajectory information and behavior information, one or more interaction events occurring between this combination at different times can be analyzed. Finally, the pre-constructed classification model can identify the interaction events and determine whether they are reception events. Therefore, based on the long-time sequence tracking information, it can help comprehensively analyze the interaction characteristics between store clerks and customers, so as to more accurately evaluate the reception events, and can accurately record information such as the object and time of the occurrence of the reception event, avoiding the confusion of the object and time of the reception event caused by single-frame image detection, and effectively improving the accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0058] Figure 1 It is a schematic flowchart of a method for detecting a reception event provided by an embodiment of the present application;

[0059] Figure 2 It is a schematic diagram of the principle of a three-pointer algorithm provided by an embodiment of the present application;

[0060] Figure 3 It is a schematic diagram of a detection device for reception events provided by an embodiment of the present application. Detailed implementation manners

[0061] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0062] As described in the background art, in the related art, based on computer vision technology, the behavior of personnel can be analyzed, and reception events can be detected by whether the shop assistant and the customer interact. However, this detection method usually detects events based on a single frame of image, and this method has many defects:

[0063] 1. It is difficult to associate the same event between frames, resulting in confusion of event objects and time misalignment.

[0064] 2. It is impossible to associate multiple cameras, resulting in duplicate events or inaccurate event time periods.

[0065] Therefore, these methods cannot accurately locate the reception time period and the identification (ID) of the shop assistant and the customer corresponding to the reception event. Even if it can be recognized in a certain frame that a reception event has occurred, it is impossible to correspond to who specifically received the customer, or who is being received, and how long the reception lasted. This is basically invalid information for analyzing user portraits.

[0066] The present application provides a method for detecting reception events. First, it obtains the trajectory information and behavior information of the personnel (including store clerks and customers) who are active in the target area within a fixed time. The trajectory information is used to identify the position changes of each person at different times, and the behavior information is used to identify the behavior characteristics of each person at different times, thereby achieving global and uninterrupted tracking of each person in the target area. Then, for the combination of store clerks and customers, based on their trajectory information and behavior information, one or more interaction events occurring between this combination at different times can be analyzed. Finally, a pre-constructed classification model can identify the interaction events to determine whether they are reception events. Thus, based on the tracking information of a long time series, it can help comprehensively analyze the interaction characteristics between store clerks and customers, thereby more accurately evaluating reception events, and can accurately record information such as the objects and times when reception events occur, avoiding the confusion of the objects and times of reception events caused by single-frame image detection, and effectively improving the accuracy of detection.

[0067] The solution provided by the embodiments of the present application relates to the field of computer vision technology, and is specifically described through the following embodiments.

[0068] See Figure 1 As shown, it is a schematic flowchart of a method for detecting reception events provided by the embodiments of the present application. In this embodiment, it can be executed by taking a computer device as an example.

[0069] S101: Obtain the trajectory information and behavior information respectively corresponding to multiple persons.

[0070] The target area is the area where reception event detection is carried out, such as a store where commercial transactions are carried out, etc.

[0071] Among them, N persons are used to identify M store clerks and N customers who are active in the target area within a fixed time. The fixed time can be set by the merchant of the store himself, such as one day, one week or one month, etc., or it can be a period of time from a certain moment to the present, so as to achieve real-time detection.

[0072] The trajectory information is used to identify the position changes of each person at different times, including the position information of each person in the target area and the corresponding time, which can be presented in the form of a dynamic trajectory. Through this trajectory information, a globally unique identifier can be established for each person who has been active in this target area, realizing uninterrupted tracking of each person.

[0073] The behavior information is used to identify the behavior characteristics of each person at different times, and the behavior characteristics can be reflected by the body state of the person, such as the orientation of the head, the shape of the limbs, etc.

[0074] In order to break the limitations of traditional manual entry of fellow travelers, multiple cameras are required to cover the target area to obtain image data in the target area, so as to realize automatic detection of fellow travelers through computer equipment.

[0075] In a possible implementation, S101 may include the following steps:

[0076] A1: Obtain videos from multiple cameras in the target area within a fixed time.

[0077] Multiple cameras are installed in the target area to cover the target area as much as possible, and in actual scenarios, multiple cameras inevitably have overlapping coverage areas. Multiple cameras in the target area can obtain multiple videos within a fixed time, so as to obtain the information of people in the target area within the fixed time. Among them, the video obtained by the camera is a two-dimensional (2D) picture.

[0078] A2: Detect people in multiple video frames of the video and extract the human posture key points of the people in each camera.

[0079] The key points of human posture refer to some representative points on the human body, based on which the specific shape of the person can be analyzed. They usually include key points of 17 parts: head, eyes, nose, ears, shoulders, elbows, wrists, hips, knees and ankles.

[0080] The video stream of each camera channel is input into a computer device. Through the machine learning algorithm, people in multiple video frames can be detected to obtain the appearance representation and location information of each person, and the key points of the human body posture of the person in each camera can be extracted.

[0081] The machine learning algorithm includes a person detection algorithm, a pedestrian re-identification (ReID) algorithm, and a human posture estimation algorithm.

[0082] The person detection algorithm uses a neural network to convolutionally encode the video stream of each camera channel, and finally passes through a detection head with a linear fully connected layer to output the bounding box of each person on the 2D screen of the camera. The ReID algorithm inputs the image in the bounding box of the detected person into the neural network and outputs a feature vector to represent the appearance representation of the person to distinguish from other people. The human posture estimation algorithm inputs the video stream of each camera channel into the neural network and outputs the key points of the human posture of each person in the 2D screen. The neural network can be a traditional encoding layer or an attention model with a self-attention structure.

[0083] A3: Through multi-target tracking technology, the trajectory segments corresponding to the people in each camera are obtained.

[0084] After detecting people in the 2D image for each camera channel, the multi-object tracking module associates the detection boxes of the same person between frames to form a continuous trajectory, so as to obtain trajectory segments that are dozens of seconds or minutes long in each camera.

[0085] A4: According to the position of each camera, map the human body pose key points and the trajectory segments to the three-dimensional space of the target area, so as to obtain the trajectory information and behavior information corresponding to N individuals respectively.

[0086] Through camera calibration technology, the internal parameter matrix and external parameter matrix of the camera can be obtained, so as to know the angle and position of the corresponding 2D video stream of each camera in the three-dimensional space.

[0087] The commonly used calibration tools are checkerboards or other geometric pattern calibration plates. The calibration plate is photographed by the camera from different angles and positions to ensure that the images of the calibration plate at different positions can cover the entire imaging field of view. Camera calibration mainly calibrates the internal and external parameters. The purpose of internal parameter calibration is to obtain the internal parameter matrix of the camera, which describes the internal characteristics of the camera, such as focal length, principal point offset, and distortion coefficient; the purpose of external parameter calibration is to obtain the pose of the camera (rotation and translation relative to the calibration plate or the world coordinate system), which describes the geometric relationship between the camera and the external world. Cameras usually introduce radial and tangential distortions, and the distortion coefficients are corrected using the internal parameter matrix and calibration data.

[0088] Through the internal and external parameter calibration data of the camera, the connection with the real three-dimensional space can be established. Therefore, according to the pose of the 2D video stream corresponding to the camera in the three-dimensional space, the human body pose key points and trajectory segments corresponding to each person at different times can be mapped to the three-dimensional space of the target area, and the human body pose key points corresponding to N individuals at different times can be obtained. Then, the endpoints of different trajectory segments at the same time and the same position are connected to merge the trajectory segments of the same person, so as to obtain the three-dimensional information of the trajectories corresponding to N individuals at different times.

[0089] Thus, the position information and behavior information of each person at different times can be obtained, so as to perform global tracking on each person in the target area, avoid the problem of assigning multiple identifiers to the same person at different times, and ensure the continuity and accuracy of tracking.

[0090] Among them, since the trajectories of people at different times may overlap, and the same person in a single camera may have multiple trajectory segments, when merging the trajectory segments obtained from different cameras, it is necessary to fuse the trajectory segments according to the actual situation.

[0091] In a possible implementation, A4 can be executed through the following steps:

[0092] B1: Map the human body pose key points and trajectory segments to the three-dimensional space of the target area according to the positions of each camera, and obtain the three-dimensional information of the human body pose key points and trajectory segments.

[0093] B2: Obtain the behavior information corresponding to N individuals based on the information of the human body pose key points.

[0094] B3: Merge the trajectory segments corresponding to each individual through the dynamic programming algorithm respectively, and obtain the trajectory information corresponding to N individuals based on the three-dimensional information of the trajectory segments.

[0095] Among them, the dynamic programming algorithm merges the trajectory segments corresponding to each individual with the aim of maximizing the trajectory path. When the same person has overlapping trajectories, the trajectory information of the same person can be merged in two steps.

[0096] 1. Trajectory segment fusion across cameras: The basis for fusion is the correlation degree of the three-dimensional (3D) position coordinates of the trajectory points. It is necessary to map the trajectory points from the positions in the 2D images of the cameras to the three-dimensional space based on camera calibration technology. Then, according to the relationship between the world coordinate system and the actual coordinate system, map the 3D coordinates to the position coordinates on the ground plane to obtain the positions of the human trajectory points on the ground plane of the target area. After that, judge whether it is the same person by calculating the distance constraint between the two 3D trajectory points corresponding to each frame of the two trajectories under different cameras, and fuse the trajectory segments of the same person across cameras.

[0097] 2. Connect the fused trajectory segments into a continuous trajectory: Connect the fused trajectory segments into a continuous trajectory according to the spatial position information of the trajectory points, so that a person will only have one trajectory. The merging rule is to establish a graph based on the distance relationship between the 3D trajectory points in the two end trajectories and the relationship between the speed and acceleration in the motion model. The nodes of the graph represent a trajectory, and the edges of the graph represent the correlation degree between two trajectories. The cost of the edge is also calculated using the spatial distance between the 3D trajectory points between the trajectories and the features obtained by ReID between the people. Then, use the dynamic programming algorithm to find the path of the maximum flow of the graph, and determine the final trajectory information through this path.

[0098] Therefore, by using the dynamic programming algorithm to merge the trajectory segments of the same person with the aim of maximizing the trajectory path, it is possible to avoid the situation where the trajectory segments cannot be completely merged due to trajectory overlap, thus ensuring the integrity and coherence of the trajectory and improving the accuracy of obtaining trajectory information.

[0099] After obtaining the trajectory information and behavior information of each person in the target area through the above computer vision method, the identity of each person as a store clerk or customer can be distinguished based on the detection of other external characteristics of each person, such as work badges, work uniforms, etc., so as to detect reception events.

[0100] S102: For the target store clerk among the M store clerks and the target customer among the N customers, determine the interaction events of the combination of the target store clerk and the target customer according to the trajectory information and the behavior information.

[0101] For the M store clerks and the N customers, they can be combined arbitrarily to obtain M×N combination results. According to the trajectory information and the behavior information, the interaction events that occur between the store clerk and the customer in each combination can be determined, so as to judge whether the store clerk in different combinations is receiving customers. It is also possible to filter the M×N combination results, filter out the combinations of store clerks who do not receive customers or the corresponding combinations of customers who do not need to be received, and then judge the filtered combinations.

[0102] Taking the combination of one store clerk and one customer as an example, through the trajectory information and the behavior information, it is possible to analyze whether other store clerks have long-term or close-range interaction events with this person through a machine learning algorithm, so as to determine one or more interaction events that occur in this combination and the corresponding time periods.

[0103] In a possible implementation manner, S102 is executed through the following steps:

[0104] C1: For the target store clerk among the M store clerks and the target customer among the N customers, based on the trajectory information and the behavior information, and based on a preset scoring rule, obtain the interaction scores of the combination of the target store clerk and the target customer at different times.

[0105] Among them, the interaction score is used to identify the interaction intensity. The higher the score, the greater the interaction intensity between the store clerk and the target customer. When the interaction intensity exceeds a certain level, it is considered that this combination is in an interaction state.

[0106] Based on different times, according to the distance change, the area where the two are located, and the coordinates of the key points of the human body postures of the two in three-dimensional space between different combinations, based on a preset scoring rule, a computer device can score different combinations in real time to obtain the interaction scores of each combination at each moment.

[0107] In a possible implementation manner, the preset scoring rule includes:

[0108] One or more of a distance-based scoring rule, a region-based scoring rule, and a line-of-sight interaction-based scoring rule.

[0109] The distance-based scoring rule scores according to the distance between the store clerk and the customer. The closer the distance, the higher the score. The distance-score curve presents an exponential distribution. When the distance between the store clerk and the customer exceeds the preset distance, the score approaches 0. Among them, the preset distance can be set and adjusted according to the actual scenario.

[0110] The area-based scoring rule means that when the store clerk and the customer are both in a certain preset area, a certain score increase or penalty is given. In the area where reception often occurs, such as the area where cars are parked in a 4S store, a certain score is increased. In the area where reception rarely occurs, such as the warehouse, logistics department, etc., a certain score is decreased.

[0111] The line-of-sight interaction-based scoring rule means that the orientation is calculated based on the key points of the body postures of the store clerk and the customer. When there is a line-of-sight interaction between the two, different scores are given at the moments when the customer looks at the store clerk, the store clerk looks at the customer, and they look at each other.

[0112] The preset scoring rules can include one or more of the above rules. When multiple rules are included, different penalty factors can be set for different scoring rules, and then the scores under different rules are summed based on the penalty factors to obtain the final interaction score.

[0113] The above multiple scoring rules are set based on the interaction attributes between the store clerk and the customer when an actual interaction event occurs. The more rules used, the higher the detection accuracy of the interaction event.

[0114] C2: According to the interaction score, parse the combined interaction events through a pre-constructed three-pointer algorithm.

[0115] Based on the interaction scores of the combination of the store clerk and the customer at different times, it can help determine whether a reception event occurs. However, since the store clerk's reception of the customer is a long-time sequence event, the judgment of the reception event is a complex process. For example, during the reception process, there may be a low interaction score, or a high interaction score but a short duration, etc. Determining that a reception event occurs based on the interaction state of the duration may not obtain an accurate detection result.

[0116] The embodiment of the present application proposes a three-pointer algorithm, which can first parse out the time period when the store clerk and the customer may have an interaction based on the interaction score in chronological order, and consider the interaction events occurring within this time period as one interaction event, and then identify whether the interaction event is a reception event.

[0117] Among them, the first pointer of the three-pointer algorithm is used to identify the current moment, the second pointer is used to identify the start moment of the interaction event, and the third pointer is used to identify the end moment of the interaction event.

[0118] In the process of applying the algorithm, it is necessary to preset the minimum reception duration, the minimum reception interval, and the reception score threshold in advance to analyze a complete interaction event.

[0119] The minimum reception duration refers to the minimum duration of an interaction event. The minimum reception interval means that the interaction events between the same group are not necessarily continuous. Whether to split an interaction event into multiple events is determined according to the minimum reception interval. The reception score threshold means that when the score of the group at a certain moment is greater than this threshold, it is considered that the clerk is interacting with the customer at this moment.

[0120] Reference Figure 2 , which is a schematic diagram of the principle of a three-pointer algorithm provided by an embodiment of the present application. The broken line represents the interaction score at different times. The ts pointer is the first pointer pointing to the current moment, and sequentially compares the interaction score with the reception score threshold backward in chronological order. The start_ts pointer is the second pointer representing the start moment of the current event, and the last_ts pointer is the third pointer representing the end moment of the current event. The specific principle of this three-pointer algorithm is as follows:

[0121] In chronological order, determine whether the interaction score at the current moment is greater than the reception score threshold. If the interaction score is greater than the reception score threshold for the first time, move both the start_ts pointer and the last_ts pointer to this moment. If it is not the first time that the interaction score is greater than the reception score threshold, only move the last_ts pointer to this moment.

[0122] 2. Determine whether to add it to the candidate reception event list. When the time between the start_ts~last_ts pointers meets the minimum reception duration, it is considered that an interaction event is formed within this time period, and the information of this event is recorded in the candidate reception event list. Otherwise, the ts pointer continues to move backward in chronological order.

[0123] 3. Determine whether it is an event interval. When the time between the last_ts~ts meets the condition of being greater than the minimum reception interval, update the positions of the start_ts and the last_ts to the moment where the ts is located, and then the ts pointer continues to move backward to determine the next interaction event.

[0124] Thus, through the three-pointer algorithm, the events between the clerk and the customer can be analyzed, so that the interaction status between the clerk and the customer can be evaluated in real time and automatically through a consistent measurement standard, and one or more interaction events generated between different combinations of clerks and customers can be determined to more accurately detect the reception events.

[0125] S103: Based on the multiple interaction events among the multiple persons, determine the reception events in the target area through a pre-constructed classification model.

[0126] For each interactive event, the trajectory and behavior information of the customer and the clerk can be used to construct the interaction features between the two, such as the distance change information between the customer and the clerk, the length of time they look at each other, etc., which can be used as the basis for classification to identify the interactive events.

[0127] The classification model can be a traditional machine learning algorithm, such as random forest or support vector machine, or a multi-layer perceptron or neural network. Generally speaking, the larger the model, the better the classification effect, but the load will also be greater. Therefore, a compromise needs to be considered in practical applications.

[0128] The classification model is trained based on labeled training data, where the training data includes the above-mentioned interaction feature information. The prediction result of the classification model can be a value in the range of 0-1. The larger the value, the greater the probability that the interaction event is a reception event. When the score reaches a certain threshold, the interaction event is considered to be a reception event.

[0129] Therefore, computer vision technology can be used to track each person in the target area globally and uninterruptedly. Then, for the combination of store clerks and customers, one or more interaction events that occurred between the combination at different times can be analyzed based on their trajectory information and behavior information. Finally, the pre-built classification model can identify the interaction event and determine whether it is a reception event. Therefore, based on the tracking information of a long time series, it can help to comprehensively analyze the interaction characteristics between store clerks and customers, thereby more accurately evaluating the reception events, and can accurately record information such as the object and time of the reception event, avoiding confusion of the object and time of the reception event due to single-frame image detection, and effectively improving the accuracy of detection.

[0130] In a possible implementation, S103 may be performed by the following steps:

[0131] D1: Based on multiple interaction events among multiple persons, a pre-built classification model is used to determine multiple scores corresponding to the multiple interaction events.

[0132] D2: In the same time period, if a clerk has P interactions with multiple customers who are in the same business relationship, among the scores corresponding to the P interactions, the P-1 scores that are not maximum values are suppressed.

[0133] Among them, whether customers have a peer relationship can be judged in the following ways:

[0134] Based on the trajectory and behavior information of each customer, and based on the interaction events between the two paired groups, the peer relationship between the two people in the group is determined, and then all people with peer relationships are classified into the same group through a greedy algorithm. Customers in the same group are said to have a peer relationship.

[0135] In the same time period, if the scores between the clerk and multiple customers with a trade relationship obtained by the classification model are all very high, the main reception object is determined by the maximum score, and the scores between other customers and the clerk are suppressed, for example, directly setting them to 0, or increasing the penalty factor, etc., to make the score smaller.

[0136] D3: Determine the interactive events corresponding to the scores greater than the preset threshold among the multiple scores as reception events of the target area.

[0137] The preset threshold is generally greater than the above-mentioned suppressed score, so that reception events corresponding to non-main reception objects and store clerks in the same group of people can be filtered.

[0138] Therefore, for customers with a business relationship, the main reception object can be determined, and the reception event can only be recorded for the main reception object, reducing repeated records of the same reception event, which is conducive to the subsequent optimization and improvement of store services.

[0139] Based on the above embodiments, the present application embodiment provides a reception event detection device, referring to Figure 3 FIG. 3 is a schematic diagram of a reception event detection device provided in an embodiment of the present application. The device 300 includes an acquisition unit 301 and a determination unit 302:

[0140] The acquisition unit is used to acquire trajectory information and behavior information corresponding to a plurality of persons, wherein the plurality of persons include M store employees and N customers active in the target area within a fixed time, the trajectory information is used to identify the position change of each person at different times, and the behavior information is used to identify the behavior characteristics of each person at different times;

[0141] The determining unit is used to determine, for a target clerk among the M clerks and a target customer among the N customers, an interaction event between the target clerk and the target customer according to the trajectory information and the behavior information;

[0142] The determining unit is further configured to determine the reception events of the target area based on the multiple interaction events among the multiple persons by using a pre-built classification model.

[0143] Thus, through computer vision technology, each person in the target area can be globally and continuously tracked. Then, for the combination of the store clerk and the customer, based on their trajectory information and behavior information, one or more interaction events occurring between this combination at different times can be analyzed. Finally, the pre-constructed classification model can identify the interaction events and determine whether they are reception events. Thus, based on the long-term sequence of tracking information, it can help comprehensively analyze the interaction characteristics between the store clerk and the customer, thereby more accurately evaluating the reception events, and can accurately record information such as the objects and time when the reception events occur, avoiding the confusion of the objects and time of the reception events caused by single-frame image detection, and effectively improving the accuracy of detection.

[0144] In a possible implementation manner, the obtaining unit is specifically configured to:

[0145] Obtain videos of multiple cameras in the target area within the fixed time;

[0146] Detect the people in multiple video frames of the video, and extract the body pose key points of the people in each camera;

[0147] By using multi-object tracking technology, obtain the trajectory segments corresponding to the people in each camera;

[0148] According to the positions of each camera, map the body pose key points and the trajectory segments to the three-dimensional space of the target area, and obtain the trajectory information and behavior information corresponding to the multiple people respectively.

[0149] Thus, the position information and behavior information of each person at different times can be obtained, so as to globally track each person in the target area, avoid the problem of assigning multiple identifiers to the same person at different times, and ensure the continuity and accuracy of tracking.

[0150] In a possible implementation manner, the obtaining unit is specifically configured to:

[0151] According to the positions of each camera, map the body pose key points and the trajectory segments to the three-dimensional space of the target area, and obtain the three-dimensional information of the body pose key points and the trajectory segments;

[0152] Based on the information of the body pose key points, obtain the behavior information corresponding to the multiple people respectively.

[0153] Respectively merge the trajectory segments corresponding to each person through a dynamic programming algorithm, and obtain the trajectory information corresponding to the multiple people respectively based on the three-dimensional information of the trajectory segments.

[0154] Thus, by using the dynamic programming algorithm to maximize the trajectory path, the trajectory segments of the same person are merged, which can avoid the situation where the trajectory segments cannot be completely merged due to trajectory overlap, thus ensuring the integrity and coherence of the trajectory and improving the accuracy of obtaining trajectory information.

[0155] In a possible implementation manner, the determining unit is specifically configured to:

[0156] For the target clerk among the M clerks and the target customer among the N customers, according to the trajectory information and behavior information, based on a preset scoring rule, obtain the interaction scores of the combinations of the target clerk and the target customer at different times, where the interaction scores are used to identify the interaction intensity;

[0157] According to the interaction scores, parse the interaction events of the combination through a pre-constructed three-pointer algorithm. The first pointer of the three-pointer algorithm is used to identify the current moment, the second pointer is used to identify the start moment of the interaction event, and the third pointer is used to identify the end moment of the interaction event.

[0158] Thus, through the three-pointer algorithm, the events between the clerk and the customer can be parsed, so that the interaction state between the clerk and the customer can be evaluated in real time and automatically through a consistent measurement standard, and one or more interaction events generated between different combinations of clerks and customers can be determined, so as to detect the reception events more accurately.

[0159] In a possible implementation manner, the preset scoring rule includes:

[0160] One or more of a distance-based scoring rule, a region-based scoring rule, and a line-of-sight interaction-based scoring rule.

[0161] The above-mentioned multiple scoring rules are set according to the interaction attributes between the clerk and the customer when the interaction event actually occurs. The more rules used, the higher the detection accuracy of the interaction event.

[0162] In a possible implementation manner, the determining unit is specifically configured to:

[0163] Based on the multiple interaction pending events among the multiple persons, through a pre-constructed classification model, determine the multiple scores respectively corresponding to the multiple interaction events;

[0164] In the same time period, if a clerk has P interaction events with multiple customers having a peer relationship, among the scores respectively corresponding to the P interaction events, suppress the P-1 scores that are not the maximum values;

[0165] Determine the interaction events corresponding to the scores greater than the preset threshold among the multiple scores as the reception events in the target area.

[0166] Thus, for customers with a peer relationship, the main reception object can be determined, and only the reception events of the main reception object are recorded, reducing the repeated recording of the same reception event, which is beneficial to the subsequent optimization and improvement of store services.

[0167] Based on the above embodiments, an embodiment of the present application provides a computer device, which includes a processor and a memory:

[0168] The memory is used to store a computer program;

[0169] The processor is configured to execute the above-mentioned detection method of reception events according to the computer program.

[0170] Based on the above embodiments, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program, and the computer program realizes the execution of the above-mentioned detection method of reception events when executed by a computer device.

[0171] Based on the above embodiments, an embodiment of the present application provides a computer program product including a computer program. When it runs on a computer device, it enables the computer device to execute the above-mentioned detection method of reception events.

[0172] It should be noted that the various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0173] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A detection method for reception events, characterized in that, The method includes: Obtaining trajectory information and behavior information respectively corresponding to multiple persons, where the multiple persons include M store clerks and N customers who are active in the target area within a fixed time, the trajectory information is used to identify the position changes of each person at different times, and the behavior information is used to identify the behavior characteristics of each person at different times; For a target store clerk among the M store clerks and a target customer among the N customers, determining an interaction event between the target store clerk and the target customer according to the trajectory information and the behavior information; Based on multiple interaction events among the multiple persons, determining a reception event in the target area through a pre-constructed classification model.

2. The method according to claim 1, characterized in that, The obtaining trajectory information and behavior information respectively corresponding to multiple persons includes: Obtaining videos of multiple cameras in the target area within the fixed time; Detecting persons in multiple video frames of the video and extracting human body pose key points of the persons in each camera; Through multi-object tracking technology, obtaining trajectory segments corresponding to the persons in each camera; According to the positions of each camera, mapping the human body pose key points and the trajectory segments to a three-dimensional space of the target area to obtain trajectory information and behavior information respectively corresponding to the multiple persons.

3. The method according to claim 2, characterized in that, The mapping the human body pose key points and the trajectory segments to a three-dimensional space of the target area according to the positions of each camera to obtain trajectory information and behavior information respectively corresponding to the multiple persons includes: According to the positions of each camera, mapping the human body pose key points and the trajectory segments to a three-dimensional space of the target area to obtain three-dimensional information of the human body pose key points and the trajectory segments; Based on the information of the human body pose key points, obtaining behavior information respectively corresponding to the multiple persons. Respectively merging the trajectory segments corresponding to each person through a dynamic programming algorithm, and obtaining trajectory information respectively corresponding to the multiple persons based on the three-dimensional information of the trajectory segments.

4. The method according to claim 1, characterized in that, The determining an interaction event between the target store clerk and the target customer according to the trajectory information and the behavior information for a target store clerk among the M store clerks and a target customer among the N customers includes: For a target store clerk among the M store clerks and a target customer among the N customers, obtaining an interaction score of the combination of the target store clerk and the target customer at different times according to the trajectory information and the behavior information based on a preset scoring rule, where the interaction score is used to identify the interaction intensity; According to the interaction score, parsing the interaction event of the combination through a pre-constructed three-pointer algorithm, where the first pointer of the three-pointer algorithm is used to identify the current moment, the second pointer is used to identify the start moment of the interaction event, and the third pointer is used to identify the end moment of the interaction event.

5. The method according to claim 4, wherein The preset scoring rule includes: One or more of a distance-based scoring rule, a region-based scoring rule, and a line-of-sight interaction-based scoring rule.

6. The method according to claim 1, characterized in that, The determining of the reception events of the target area based on the multiple interactive events among the multiple persons by using a pre-built classification model includes: Based on the multiple interaction pending events among the multiple persons, determining multiple scores corresponding to the multiple interaction events respectively through a pre-built classification model; In the same time period, if a clerk has P interactive events with multiple customers who are in the same business relationship, among the scores corresponding to the P interactive events, the P-1 scores that are not the maximum values are suppressed; The interactive events corresponding to the scores greater than the preset threshold among the multiple scores are determined as reception events of the target area.

7. A detection device for reception events, characterized in that, The device comprises an acquisition unit and a determination unit: The acquisition unit is used to acquire trajectory information and behavior information corresponding to a plurality of persons, wherein the plurality of persons include M store employees and N customers active in the target area within a fixed time, the trajectory information is used to identify the position change of each person at different times, and the behavior information is used to identify the behavior characteristics of each person at different times; The determining unit is used to determine, for a target clerk among the M clerks and a target customer among the N customers, an interaction event between the target clerk and the target customer according to the trajectory information and the behavior information; The determining unit is further configured to determine the reception events of the target area based on the multiple interaction events among the multiple persons by using a pre-built classification model.

8. A computer device, characterized in that, The computer device comprises a processor and a memory: The memory is used to store computer programs; The processor is configured to execute the method according to any one of claims 1 to 6 according to the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, and the computer program, when executed by a computer device, implements the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, When the method is executed on a computer device, the computer device is enabled to execute the method according to any one of claims 1 to 6.

Images