Supermarket abnormal behavior detection method and device, storage medium and computer device

Abstract

The application provides a supermarket abnormal behavior detection method and device, a storage medium and computer equipment. The target customer is determined and the multi-path camera video stream is acquired, on the basis of which target detection and tracking are performed, the action track of the customer in the supermarket is generated and the corresponding commodity state change data is associated, so that passive video recording is changed into active perception and tracking. Further, suspicious behavior detection is performed according to the action track and the commodity state change data, and suspicious personnel are marked, and specific interaction action types between the suspicious personnel and the commodities are determined, and the three types of data are combined to jointly determine the type and confidence of the theft and damage behavior. Through the two-level detection mechanism, on the one hand, the calculation overhead of the complex behavior recognition algorithm is reduced, and the real-time response capability in the high customer flow scene is ensured. On the other hand, the suspicious objects are focused on, and specific abnormal actions such as hiding, adjusting the bag, tearing the label and the like can be targeted, so that the occurrence of missed judgment and misjudgment is effectively reduced.

Description

Technical Field

[0001] This application relates to the field of data detection technology, and in particular to a method, device, storage medium and computer equipment for detecting abnormal behavior in supermarkets. Background Technology

[0002] In the operation of the supermarket and retail industry, the loss caused by theft and damage of goods has always been a pain point in the industry. In order to effectively detect abnormal theft and damage by customers, the supermarket sector has gradually adopted a combination of video surveillance and manual identification to carry out loss prevention work. In the existing technology, video streams are collected by cameras deployed in the supermarket and then manually reviewed and analyzed to identify abnormal customer behavior.

[0003] However, relying solely on manual video stream analysis and theft detection lacks the ability to proactively perceive and dynamically track customer behavior. It also lacks effective means to identify complex interactive actions such as hiding goods, switching items, and tearing off labels, resulting in low efficiency and poor accuracy in detecting abnormal behavior in supermarkets and shopping malls, and a high risk of missed or false accusations. Summary of the Invention

[0004] The purpose of this application is to at least address one of the aforementioned technical deficiencies, particularly the fact that in the prior art, relying solely on manual video stream analysis and theft behavior judgment lacks the ability to proactively perceive and dynamically track customer behavior, resulting in low efficiency and poor accuracy in detecting abnormal behavior within supermarkets, and a high risk of missed or incorrect judgments.

[0005] Firstly, this application provides a method for detecting abnormal behavior in supermarkets, the method comprising:

[0006] Identify the target customers to be tested and obtain video streams from multiple cameras within the supermarket;

[0007] Based on the video stream, target detection and tracking are performed to generate the movement trajectory of the target customer within the supermarket, and the corresponding product status change data on the movement trajectory is determined.

[0008] Based on the movement trajectory and the product status change data, suspicious behavior is detected in the target customer, and when suspicious behavior is detected, the target customer is marked as a suspicious person;

[0009] For marked suspicious persons, the type of interaction action between the suspicious person and the interactive product is determined based on the video stream, and the type of theft or damage behavior of the suspicious person and its confidence level are determined by combining the behavior trajectory, the product status change data and the type of interaction action;

[0010] When the confidence level exceeds a preset threshold, an abnormal warning signal of the corresponding level is generated.

[0011] In one embodiment, the step of performing target detection and tracking based on the video stream to generate the movement trajectory of the target customer within the supermarket includes:

[0012] Pedestrian detection is performed on the video stream to extract the visual feature information of the target customer;

[0013] Based on the visual feature information, pedestrian re-identification technology is used to track the target customer across different cameras, forming a continuous movement trajectory of the target customer within the supermarket.

[0014] The movement trajectory includes the areas the target customer passes through, the duration of their stay in each area, and the number of times they go back and forth in front of each shelf.

[0015] In one embodiment, determining the corresponding product status change data on the movement trajectory includes:

[0016] Based on the behavioral trajectory, the target shelves that the target customer passed through and lingered on were determined;

[0017] Acquire sensor data from the target shelf and align the sensor data with the behavior trajectory in time and space. Extract product status change data corresponding to the behavior trajectory from the sensor data.

[0018] In one embodiment, the step of detecting suspicious behavior in the target customer based on the movement trajectory and the product status change data, and marking the target customer as a suspicious person when suspicious behavior is detected, includes:

[0019] Obtain the preset set of suspicious judgment rules;

[0020] Trajectory parameters are extracted from the movement trajectory, and a matching parameter set is generated by combining the trajectory parameters with the product status change data;

[0021] The matching parameter set is matched with each rule in the suspicious determination rule set. When any rule in the suspicious determination rule set matches the matching parameter set, the target customer is marked as a suspicious person.

[0022] In one embodiment, determining the type of interaction action between the suspicious person and the interactive product based on the video stream includes:

[0023] When the interaction between the suspicious person and the product is detected, the video stream is frame extracted, and human posture recognition is performed based on the extracted frame images to obtain the positional change features of the suspicious person's hand feature points.

[0024] Based on the location change characteristics and the product status change data, the type of interaction action between the suspicious person and the interacting product is determined.

[0025] In one embodiment, determining the type of theft or damage behavior of the suspicious person and its confidence level by combining the behavioral trajectory, the product status change data, and the interaction action type includes:

[0026] Determine the pre-trained theft detection model;

[0027] The behavioral trajectory, the product status change data, and the interaction action type are input into the theft and damage identification model, and the theft and damage behavior type and its confidence level of the suspicious person are output.

[0028] In one embodiment, the method further includes:

[0029] When the movement trajectories of multiple customers within a supermarket are determined, heat maps of each area within the supermarket are generated. These heat maps are used to characterize the customer dwell density in each area.

[0030] Secondly, this application provides a device for detecting abnormal behavior in supermarkets, the device comprising:

[0031] The video acquisition module is used to identify the target customer to be detected and to acquire video streams from multiple cameras within the supermarket.

[0032] The trajectory determination module is used to perform target detection and tracking based on the video stream, generate the movement trajectory of the target customer in the supermarket, and determine the corresponding product status change data on the movement trajectory;

[0033] The initial detection module is used to detect suspicious behavior of the target customer based on the movement trajectory and the product status change data, and to mark the target customer as a suspicious person when suspicious behavior is detected.

[0034] The secondary detection module is used to determine the type of interaction action between the marked suspicious person and the interactive product based on the video stream, and to determine the type of theft or damage behavior of the suspicious person and its confidence level by combining the behavior trajectory, the product status change data and the interaction action type.

[0035] An anomaly warning module is used to generate an anomaly warning signal of the corresponding level when the confidence level exceeds a preset threshold.

[0036] Thirdly, this application provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the supermarket abnormal behavior detection method as described in any of the above embodiments.

[0037] Fourthly, this application provides a computer device, including: one or more processors, and a memory;

[0038] The memory stores computer-readable instructions, and when the one or more processors execute the computer-readable instructions, they perform the steps of the supermarket abnormal behavior detection method as described in any of the above embodiments.

[0039] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0040] The supermarket abnormal behavior detection method, device, storage medium, and computer equipment provided in this application identify target customers and acquire video streams from multiple cameras. Based on this, target detection and tracking are performed to generate the customer's movement trajectory within the supermarket and associate it with corresponding product status change data, thus transforming passive recording into active perception and tracking. Furthermore, suspicious behavior is detected and suspicious individuals are marked based on the movement trajectory and product status change data. The specific interaction type between the suspicious individuals and the products is then determined. Combining the movement trajectory, product status changes, and interaction type, the type of theft or damage behavior and its confidence level are jointly determined. This two-level detection mechanism reduces the computational overhead of complex behavior recognition algorithms, ensuring real-time response capabilities in high-traffic scenarios. On the other hand, it focuses on marked suspicious objects, enabling targeted analysis of subtle abnormal actions such as concealment, product swapping, and label removal, thereby effectively reducing missed or false positives and improving the efficiency and accuracy of abnormal behavior detection. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart illustrating a method for detecting abnormal behavior in supermarkets provided in this application embodiment;

[0043] Figure 2 This is a schematic diagram of the structure of a supermarket abnormal behavior detection device provided in an embodiment of this application;

[0044] Figure 3 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] In one embodiment, this application provides a method for detecting abnormal behavior in supermarkets. The following embodiments illustrate the application of this method to a server. It is understood that the method for detecting abnormal behavior in supermarkets can be executed by a single server or a server cluster consisting of multiple servers; this application does not impose specific limitations on this. Furthermore, the customer information collected in this application, such as posture, actions, and clothing information, is only used for detecting abnormal behavior within the supermarket, and a periodic destruction mechanism is implemented. Based on this, as... Figure 1 As shown, this application provides a method for detecting abnormal behavior in supermarkets, the method comprising:

[0047] S101: Identify the target customers to be detected and acquire video streams from multiple cameras within the supermarket.

[0048] Target customers refer to individuals within the video surveillance coverage area of ​​the supermarket / store. Video stream refers to a continuous stream of image data collected in real time by multiple cameras.

[0049] In this step, when someone enters the supermarket, the server can directly identify that person as a target customer, or it can identify individuals with high trust risk as target customers. Trust risk refers to the probability level of an individual's abnormal behavior within the supermarket, such as theft, misconduct, or consumer fraud, based on their historical supermarket behavior records. Higher trust risk indicates a greater likelihood of theft or other abnormal behavior. This allows for differentiated screening and focused control of monitored individuals. The method for determining target customers can be chosen based on actual circumstances and needs; this application does not impose specific restrictions on this.

[0050] After identifying the target customers, the server can acquire video streams from various cameras within the supermarket. Specifically, it can acquire the video streams from each camera in real time after identifying the target customers. Alternatively, after identifying the target customers, it can first extract their visual identity information, then filter out videos from the current video streams captured by each camera that match that visual identity information, and use these filtered videos as the acquired video streams from multiple cameras within the supermarket.

[0051] Specifically, cameras in supermarkets can be installed in various shelf areas, checkout areas, supermarket entrances and exits, employee areas, etc. High-performance cameras or high-density camera distribution can be set up in areas with high-value goods.

[0052] S102: Target detection and tracking based on video stream, generating the movement trajectory of target customers within the supermarket, and determining the corresponding changes in product status along the movement trajectory.

[0053] Among them, the movement trajectory refers to the time-series location sequence formed by continuous spatiotemporal positioning, which includes location coordinates, movement path, dwell area, dwell time, and number of round trips. Product status change data refers to the relevant data on changes in the physical and in-situ status of products that match the target customer's movement trajectory in time and space.

[0054] In this step, the server performs real-time image processing on the acquired multiple video streams, identifies and locks onto the target customer in the frame, continuously tracks their positional changes within the frame, and generates the target customer's movement trajectory within the supermarket. Specifically, the location information of the target customer at different times and under different cameras is sequentially linked to form a continuous and complete activity path, i.e., the movement trajectory. Next, the corresponding product status change data on the movement trajectory is determined. Specifically, the generated movement trajectory is aligned with the spatiotemporal information of the supermarket shelves and product areas, extracting product status changes that occur at the same time and location when the target customer stops and touches the product, thus achieving a correlation between personnel behavior and product status.

[0055] S103: Based on movement trajectory and changes in product status data, detect suspicious behavior of target customers, and mark target customers as suspicious persons when suspicious behavior is detected.

[0056] Suspicious behavior refers to abnormal shopping behavior patterns of target customers in supermarkets and shopping malls, which may indicate potential theft, damage, or other irregularities.

[0057] In this step, the generated movement trajectory and its corresponding product status change data can be compared with the preset normal shopping behavior model or suspicious judgment rules. The abnormal behavior can be judged by dimensions such as trajectory characteristics, dwell time, number of times to and from the area, and product picking and putting back logic. If the comparison finds that there is abnormal behavior, the target customer is marked as a suspicious person.

[0058] Specifically, when using a pre-defined normal shopping behavior model to identify suspicious behavior, this model establishes a baseline of normal behavior based on regular shopping habits, allowing for the detection of suspicious behavior by target customers. When using suspicious behavior determination rules, pre-defined behavioral characteristics of abnormal shopping behaviors can be used, such as customers repeatedly lingering on cosmetic shelves or frequently taking items without adding them to their carts. These behavioral characteristics are then set as suspicious behavior determination rules. The rules are then matched against these rules to detect suspicious behavior in target customers. If suspicious behavior is confirmed in a target customer, that customer is marked as a suspicious person.

[0059] It is understandable that by detecting suspicious behavior and marking suspicious individuals, target customers can be screened in advance, and high-risk individuals can be included in the subsequent in-depth analysis process. This not only achieves accurate identification of potential abnormal individuals, but also avoids the waste of resources caused by indiscriminate analysis of all individuals, and improves the targeting and processing efficiency of abnormal behavior detection.

[0060] S104: For marked suspicious persons, determine the type of interaction action between the suspicious person and the interactive product based on the video stream, and combine the behavior trajectory, product status change data and interaction action type to determine the type of theft or damage behavior of the suspicious person and its confidence level.

[0061] Interaction type refers to the specific action category determined by video stream pose recognition and object tracking, including but not limited to picking up, putting back, hiding, moving, obscuring, tearing off tags, switching, and passing. Theft / damage behavior type refers to the behavior category falling under the category of theft / damage, determined based on a comprehensive assessment of interaction actions, trajectories, and product status. This includes, but is not limited to, hiding goods, switching goods, tearing off anti-theft tags, group passing, and possession of goods without payment. Confidence level is used to quantify the likelihood of suspicious individuals committing the corresponding theft / damage behavior.

[0062] In this step, for identified suspicious individuals, the server first performs human posture analysis and product tracking from the real-time video stream, identifying actions such as contact, movement, and occlusion between the suspicious individual's hands, body, and products, thereby determining the corresponding interaction type. Then, this interaction type is spatiotemporally aligned and jointly analyzed with previously generated product state change data. Feature matching is performed according to a pre-defined theft / damage behavior judgment model to determine the specific theft / damage behavior type. Finally, a weighted calculation is performed based on indicators such as action completeness, trajectory rationality, and state change matching degree, outputting the confidence score corresponding to the theft / damage behavior.

[0063] Specifically, by combining interactive actions, behavioral trajectories, and product status change data for in-depth analysis, the types of theft and damage behaviors can be identified and confidence levels can be quantified. This approach first achieves a detailed breakdown of abnormal actions and then conducts a numerical evaluation of multi-dimensional data. It can not only accurately distinguish between normal shopping and theft and damage behaviors, but also objectively measure the level of behavioral risk with confidence levels, thereby improving the accuracy and reliability of theft and damage behavior identification.

[0064] S105: When the confidence level exceeds the preset threshold, an abnormal warning signal of the corresponding level is generated.

[0065] The abnormal warning signal refers to a warning instruction that includes the warning level, personnel location, type of theft or damage, time information, and fragments of evidence. The preset threshold is an empirical value, and this application does not impose specific restrictions on it.

[0066] In this step, when the confidence level exceeds a preset threshold, a corresponding warning level is assigned based on the confidence level value. Evidence fragments are generated based on video streams, behavioral trajectories, and changes in product status. Finally, the real-time location of suspicious individuals, the type of theft or damage, the warning level, and the evidence fragments are summarized to generate an anomaly warning signal. After generating the anomaly warning signal, it can be pushed to the supermarket manager and security personnel to ensure a timely and efficient warning response.

[0067] In the above embodiments, by identifying target customers and acquiring video streams from multiple cameras, target detection and tracking are performed to generate the customer's movement trajectory within the supermarket and associate it with corresponding product status change data, thus transforming passive recording into active perception and tracking. Furthermore, suspicious behavior is detected and suspicious individuals are marked based on the movement trajectory and product status change data. Then, the specific interaction type between the suspicious individuals and the products is determined. Combining the movement trajectory, product status changes, and interaction type, the type of theft or damage behavior and its confidence level are jointly determined. This two-level detection mechanism reduces the computational overhead of complex behavior recognition algorithms, ensuring real-time response capabilities in high-traffic scenarios. On the other hand, focusing on marked suspicious objects allows for targeted analysis of subtle abnormal actions such as concealment, product swapping, and label removal, effectively reducing missed or false positives and improving the efficiency and accuracy of abnormal behavior detection.

[0068] In one embodiment, target detection and tracking are performed based on the video stream to generate the movement trajectory of the target customer within the supermarket, including:

[0069] S1: Perform pedestrian detection on the video stream and extract visual feature information of target customers.

[0070] S2: Based on visual feature information, pedestrian re-identification technology is used to track target customers across different cameras, forming a continuous movement trajectory of target customers within the supermarket.

[0071] The movement trajectory includes the areas the target customer passes through, the duration of their stay in each area, and the number of times they go back and forth in front of each shelf. Visual feature information refers to the appearance and structural characteristics used to distinguish different customers, including but not limited to clothing color, style, texture, hairstyle, height, and facial features.

[0072] In this embodiment, when performing pedestrian detection on the video stream, the video stream can be analyzed frame by frame using a target detection algorithm to locate the human body area in the image and exclude non-person interference such as shelves, goods, and equipment, thereby locking the location of the target customer. Based on this, a feature extraction network is used to extract visual feature information that can identify the target customer from the locked human body area. Based on the extracted visual feature information, the server can use pedestrian re-identification technology to match and associate the same target customer collected by different cameras and different time segments, realizing continuous tracking across cameras and across regions. Finally, the location segments scattered from the perspectives of various cameras are spliced ​​together according to time sequence and spatial relationship to form a complete and coherent movement trajectory of the target customer in the entire supermarket area.

[0073] Specifically, pedestrian re-identification technology is a computer vision technology that does not rely on a specific camera and achieves cross-camera matching and continuous tracking of people solely through human visual features. By extracting distinguishable features such as clothing, contours, gait, and posture, it automatically identifies and determines whether the same target object exists in camera images from multiple different locations and angles. It can effectively solve the problems of tracking interruption and identity loss caused by camera switching, personnel occlusion, and changes in perspective in complex scenarios such as shopping malls.

[0074] For example, in a real-world supermarket application scenario, when a customer walks from the entrance camera area to the aisle camera area and then to the checkout camera area, the server first detects the customer in each camera view and extracts visual feature information such as clothing and gait. Then, through pedestrian re-identification technology, it determines that the features from these different perspectives belong to the same person, and then connects the three location trajectories into a complete path from entering the store to browsing and then to the checkout. Even if the customer turns around, obstructs the view, or switches the monitoring screen, the tracking can still be maintained without interruption through visual feature information, ensuring the continuity and integrity of the trajectory.

[0075] It is understandable that by using video streams to detect pedestrians and extract visual feature information, and then relying on pedestrian re-identification technology to complete cross-camera tracking under multiple cameras and generate continuous movement trajectories, it is possible to not only stably lock and continuously track target customers in the multi-camera environment of the entire shopping mall, but also effectively avoid tracking interruptions and identity confusion caused by perspective switching and personnel obstruction, providing continuous and complete trajectory data support for subsequent behavior analysis and suspicious judgment.

[0076] In one embodiment, determining the corresponding product status change data along the movement trajectory includes:

[0077] S1: Identify the target shelves that the target customer has passed through and lingered on based on their behavioral trajectory.

[0078] S2: Acquire sensor data from the target shelf and align the sensor data with the behavior trajectory in time and space. Extract the product status change data corresponding to the movement trajectory from the sensor data.

[0079] Among them, sensor data refers to the data collected by sensing devices such as gravity sensors, RFID readers, and pressure sensors deployed on supermarket shelves, which are used to reflect the location, picking up, putting back, weight changes, tag status, and displacement of goods.

[0080] In this embodiment, the target shelves that the target customer has passed through and lingered on are determined based on the behavioral trajectory. That is, according to the generated continuous movement trajectory, the shelf locations where the customer arrives, lingers, and interacts are filtered according to a preset dwell time threshold, and these locations are marked as target shelves. Then, real-time monitoring data collected by various sensors on the corresponding target shelves are retrieved, and the sensor data is matched and aligned with the behavioral trajectory on the time axis and spatial location based on the target customer's dwell time and spatial coordinates on the target shelf. Then, information such as product picking, weight changes, and label status changes corresponding to the customer's dwell and contact time are extracted from the spatiotemporally aligned sensor data, and finally, product status change data corresponding to the movement trajectory is formed.

[0081] Specifically, the system first locates the target shelf where customers interact based on their behavioral trajectory, then acquires the corresponding shelf sensor data and aligns it with the behavioral trajectory in time and space, and finally extracts the matching product status change data. This allows for an objective and accurate correlation between customer behavior and product changes, avoiding errors and delays caused by manual verification, and providing a reliable quantitative basis for subsequent judgment of suspicious behavior and identification of theft and damage.

[0082] In one embodiment, based on movement trajectory and product status change data, suspicious behavior detection is performed on target customers, and when suspicious behavior is detected, the target customer is marked as a suspicious person, including:

[0083] S1: Obtain the preset set of suspicious judgment rules.

[0084] S2: Extract trajectory parameters from the movement trajectory, and combine the trajectory parameters with the product status change data to generate a matching parameter set.

[0085] S3: Match the matching parameter set with each rule in the suspicious judgment rule set. When any rule in the suspicious judgment rule set matches the matching parameter set, mark the target customer as a suspicious person.

[0086] The suspicious behavior determination rule set refers to a set of structured judgment conditions used to determine whether customer behavior poses a risk of theft or damage. Trajectory parameters refer to quantitative feature data extracted from movement trajectories, including but not limited to dwell time, number of trips between areas, movement speed, and the percentage of time spent in high-value areas. The matching parameter set refers to a set of quantitative features formed by fusing trajectory parameters with product status change data within the corresponding spatiotemporal range, used for comparison with the suspicious behavior determination rule set.

[0087] In this embodiment, the server can load various suspicious behavior judgment conditions pre-set by the supermarket's loss prevention logic from local storage or the cloud. These conditions exist in the form of numerical thresholds and behavioral logic. Then, trajectory parameters such as dwell time, number of round trips, and path characteristics are extracted from the generated target customer's movement trajectory. These trajectory parameters are integrated with corresponding product status change data to form a set of matching parameters for comparison. Next, the matching parameter set is compared and calculated against each rule in the suspicious behavior judgment rule set. When any one of the rules is met, suspicious behavior is determined, and the target customer is marked as a suspicious person. This process quickly identifies high-risk individuals in a unified, objective, and quantifiable manner, avoiding the subjectivity and lag of manual judgment, reducing ineffective analysis and wasted computing power, thereby improving the efficiency and accuracy of suspicious behavior identification.

[0088] Furthermore, when determining whether suspicious behavior exists, the condition can be set to a preset value for the number of rules that are met, or other related settings. This application does not impose specific restrictions on this.

[0089] For example, in actual supermarket application scenarios, rules such as "more than 5 trips to the same shelf", "staying in the high-value goods area for more than 10 minutes without a normal checkout path" and "taking goods without putting them back or entering the checkout area" can be loaded. When a customer's trajectory parameters show that they have gone back and forth to the cosmetics shelf 6 times and stayed for 12 minutes, and the product status data shows that the goods have been taken and not put back, the corresponding rule is matched and the customer is immediately marked as a suspicious person.

[0090] In one embodiment, determining the type of interaction between a suspicious person and an interactive product based on the video stream includes:

[0091] S1: When a suspicious person is detected interacting with the product, the video stream is frame extracted, and human posture recognition is performed based on the extracted frame images to obtain the positional change features of the suspicious person's hand feature points.

[0092] S2: Based on location change characteristics and product status change data, determine the type of interaction action between suspicious persons and interacting products.

[0093] Among them, the position change feature refers to the set of quantitative features of the key feature points of a suspicious person's hand, including coordinate offset, direction of movement, speed of movement, trajectory of movement and spatial posture changes between consecutive video frames. It is used to reflect the movement pattern of the hand when it comes into contact with, picks up, hides or covers the goods.

[0094] In this embodiment, when the server detects suspicious interaction between a person and a product, it first extracts frames from the real-time video stream, filtering out continuous frames containing the instant the person comes into contact with the product. Then, it performs human posture recognition on these frames, locating hand-related feature points such as the shoulder, elbow, wrist, and fingers, and calculating the temporal positional offset, movement speed, and trajectory of these feature points to obtain the positional change characteristics of the hand feature points. Subsequently, this positional change characteristic is combined with product state change data in the same time and space, and the specific interaction type between the suspicious person and the interacting product is determined based on the action logic and state correspondence. For example, when a suspicious person touches a product in front of a shelf, the corresponding video frames are extracted, and the continuous positional change characteristics of the hand—from naturally hanging down to reaching towards the product, and then quickly retracting into clothing or a bag—are identified. Combined with the state change data of the product being taken and not put back, or being removed from the shelf, it can be concluded that the suspicious person may be engaging in an interaction type of concealing the product.

[0095] Specifically, the system first uses visual algorithms to accurately capture details of hand movements, and then uses multi-data fusion to determine the type of action. This allows for the objective, precise, and stable identification of complex interactive actions such as taking, putting back, hiding, switching, and tearing off labels, providing a reliable basis for subsequent theft and damage behavior type determination and confidence calculation.

[0096] In one embodiment, by combining behavioral trajectory, product status change data, and interaction action types, the type of theft or damage behavior of suspicious individuals and its confidence level are determined, including:

[0097] S1: Determine the pre-trained theft detection model.

[0098] S2: Input the behavior trajectory, product status change data and interaction action type into the theft and damage identification model, and output the theft and damage behavior type of the suspicious person and its confidence level.

[0099] In this embodiment, a pre-trained theft and damage identification model is determined. This model has the ability to learn abnormal patterns from trajectories, product status, and interactive actions. Subsequently, the obtained behavioral trajectory features, product status change data, and interactive action types are used as input data to the theft and damage identification model for forward inference calculation. The model outputs the theft and damage behavior type corresponding to the suspicious person and the confidence level indicating the probability through internal feature weighting, pattern matching, and probability evaluation.

[0100] Specifically, the training sample set is collected from the supermarket scene, including behavioral trajectories, product status changes, and interaction types of normal shopping behavior and various theft and damage behaviors. The theft and damage behavior types and their confidence levels corresponding to the training sample set are used as label data. The training sample set is used for forward propagation to calculate the predicted behavior type and confidence level output by the model. The loss function is calculated by combining the labeled data and backpropagation is used to update the model parameters. The model's ability to distinguish between normal behavior and theft and damage behaviors is continuously optimized until the model converges, resulting in a theft and damage identification model that can be directly deployed and used.

[0101] Understandably, a pre-trained theft and damage detection model is identified and loaded. Then, three types of information—behavioral trajectory, product status change data, and interaction action type—are input into the model for inference, ultimately outputting the theft and damage behavior type and confidence level. This approach enables the identification of complex theft and damage behaviors in a more efficient and generalizable manner, reducing misjudgments and omissions caused by human experience.

[0102] In one embodiment, the supermarket abnormal behavior detection method further includes:

[0103] When the movement trajectories of multiple customers within a supermarket are determined, heat maps of various areas within the supermarket are generated.

[0104] Among them, the heat map is used to characterize the customer dwell density in each area.

[0105] In this embodiment, when the supermarket abnormal behavior detection method is applied to multiple customers, the movement trajectories of multiple customers can be obtained. Then, the movement trajectories of multiple customers can be summarized and spatially statistically analyzed and visualized to generate heat maps of various areas of the supermarket, thereby realizing an intuitive presentation of customer flow distribution.

[0106] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0107] The following describes the supermarket abnormal behavior detection device provided in the embodiments of this application. The supermarket abnormal behavior detection device described below can be referred to in correspondence with the supermarket abnormal behavior detection method described above.

[0108] like Figure 2 As shown, this application provides a supermarket abnormal behavior detection device 200, the device comprising:

[0109] The video acquisition module 201 is used to identify the target customer to be detected and to acquire video streams collected by multiple cameras within the supermarket.

[0110] The trajectory determination module 202 is used to perform target detection and tracking based on video stream, generate the movement trajectory of the target customer in the supermarket, and determine the corresponding product status change data on the movement trajectory;

[0111] The initial detection module 203 is used to detect suspicious behavior of target customers based on movement trajectory and changes in product status data, and to mark the target customer as a suspicious person when suspicious behavior is detected.

[0112] The secondary detection module 204 is used to determine the type of interaction action between the marked suspicious person and the interactive product based on the video stream, and to determine the type of theft or damage behavior of the suspicious person and its confidence level by combining the behavior trajectory, product status change data and interaction action type.

[0113] The anomaly warning module 205 is used to generate an anomaly warning signal of the corresponding level when the confidence level exceeds a preset threshold.

[0114] In the above embodiments, by identifying target customers and acquiring video streams from multiple cameras, target detection and tracking are performed to generate the customer's movement trajectory within the supermarket and associate it with corresponding product status change data, thus transforming passive recording into active perception and tracking. Furthermore, suspicious behavior is detected and suspicious individuals are marked based on the movement trajectory and product status change data. Then, the specific interaction type between the suspicious individuals and the products is determined. Combining the movement trajectory, product status changes, and interaction type, the type of theft or damage behavior and its confidence level are jointly determined. This two-level detection mechanism reduces the computational overhead of complex behavior recognition algorithms, ensuring real-time response capabilities in high-traffic scenarios. On the other hand, focusing on marked suspicious objects allows for targeted analysis of subtle abnormal actions such as concealment, product swapping, and label removal, effectively reducing missed or false positives and improving the efficiency and accuracy of abnormal behavior detection.

[0115] In one embodiment, the trajectory determination module includes:

[0116] The information extraction submodule is used to perform pedestrian detection on the video stream and extract the visual feature information of the target customers;

[0117] The cross-camera tracking submodule is used to track target customers across different cameras based on visual feature information and pedestrian re-identification technology, forming a continuous movement trajectory of the target customer within the supermarket.

[0118] The movement trajectory includes the areas the target customer passes through, the length of time spent in each area, and the number of times they go back and forth in front of each shelf.

[0119] In one embodiment, the trajectory determination module includes:

[0120] The shelf determination submodule is used to determine the target shelves that the target customer has passed through and stopped on based on the customer's behavior trajectory.

[0121] The data extraction submodule is used to acquire sensor data from the target shelf, align the sensor data with the behavior trajectory in time and space, and extract the product status change data corresponding to the movement trajectory from the sensor data.

[0122] In one embodiment, the initial detection module includes:

[0123] The rule acquisition submodule is used to acquire a preset set of suspicious judgment rules;

[0124] The parameter generation submodule is used to extract trajectory parameters from the movement trajectory and combine the trajectory parameters with the product status change data to generate a matching parameter set;

[0125] The rule matching submodule is used to match the matching parameter set with each rule in the suspicious judgment rule set. When any rule in the suspicious judgment rule set matches the matching parameter set, the target customer is marked as a suspicious person.

[0126] In one embodiment, the secondary detection module includes:

[0127] The posture recognition submodule is used to extract frames from the video stream when an interaction between a suspicious person and a product is detected, and to perform human posture recognition based on the extracted frame images to obtain the positional change features of the suspicious person's hand feature points.

[0128] The Action Determination Submodule is used to determine the type of interaction action between suspicious persons and interacting products based on location change characteristics and product status change data.

[0129] In one embodiment, the secondary detection module includes:

[0130] The model determination submodule is used to determine the pre-trained theft and damage detection model;

[0131] The theft and damage identification submodule is used to input behavioral trajectories, product status change data, and interaction action types into the theft and damage identification model, and output the theft and damage behavior type of suspicious persons and their confidence level.

[0132] In one embodiment, the supermarket abnormal behavior detection device further includes:

[0133] The heat map generation module is used to generate heat maps of various areas within a supermarket when the movement trajectories of multiple customers within the supermarket are determined. The heat maps are used to characterize the customer dwell density in each area.

[0134] The division of modules in the above-described supermarket abnormal behavior detection device is merely illustrative. In other embodiments, the supermarket abnormal behavior detection device can be divided into different modules as needed to complete all or part of its functions. Each module in the above-described supermarket abnormal behavior detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0135] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the supermarket abnormal behavior detection method as described in any of the above embodiments.

[0136] In one embodiment, this application also provides a computer device storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the supermarket abnormal behavior detection method as described in any of the above embodiments.

[0137] Indicatively, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the internal structure of a computer device 300 provided in an embodiment of this application. The computer device 300 can be provided as a server. (Refer to...) Figure 3 The computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by memory 301 for storing instructions, such as application programs, that can be executed by the processing component 302. The application programs stored in memory 301 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 302 is configured to execute instructions to perform the supermarket abnormal behavior detection method of any of the above embodiments.

[0138] The computer device 300 may also include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate on an operating system stored in memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0139] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0140] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this document, the singular forms "a," "an," and "the" may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising / including” or “having” specify the presence of the stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0141] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0142] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting abnormal behavior in supermarkets, characterized in that, The method includes: Identify the target customers to be tested and obtain video streams from multiple cameras within the supermarket; Based on the video stream, target detection and tracking are performed to generate the movement trajectory of the target customer within the supermarket, and the corresponding product status change data on the movement trajectory is determined. Based on the movement trajectory and the product status change data, suspicious behavior is detected in the target customer, and when suspicious behavior is detected, the target customer is marked as a suspicious person; For marked suspicious persons, the type of interaction action between the suspicious person and the interactive product is determined based on the video stream, and the type of theft or damage behavior of the suspicious person and its confidence level are determined by combining the behavior trajectory, the product status change data and the type of interaction action; When the confidence level exceeds a preset threshold, an abnormal warning signal of the corresponding level is generated.

2. The method for detecting abnormal behavior in supermarkets according to claim 1, characterized in that, The step of detecting and tracking the target based on the video stream to generate the movement trajectory of the target customer within the supermarket includes: Pedestrian detection is performed on the video stream to extract the visual feature information of the target customer; Based on the visual feature information, pedestrian re-identification technology is used to track the target customer across different cameras, forming a continuous movement trajectory of the target customer within the supermarket. The movement trajectory includes the areas the target customer passes through, the duration of their stay in each area, and the number of times they go back and forth in front of each shelf.

3. The method for detecting abnormal behavior in supermarkets according to claim 1, characterized in that, The determination of the corresponding product status change data on the movement trajectory includes: Based on the behavioral trajectory, the target shelves that the target customer passed through and lingered on were determined; Acquire sensor data from the target shelf and align the sensor data with the behavior trajectory in time and space. Extract product status change data corresponding to the behavior trajectory from the sensor data.

4. The method for detecting abnormal behavior in supermarkets according to claim 1, characterized in that, The step of detecting suspicious behavior in the target customer based on the movement trajectory and the changes in the product status, and marking the target customer as a suspicious person when suspicious behavior is detected, includes: Obtain the preset set of suspicious judgment rules; Trajectory parameters are extracted from the movement trajectory, and a matching parameter set is generated by combining the trajectory parameters with the product status change data; The matching parameter set is matched with each rule in the suspicious determination rule set. When any rule in the suspicious determination rule set matches the matching parameter set, the target customer is marked as a suspicious person.

5. The method for detecting abnormal behavior in supermarkets according to claim 1, characterized in that, The step of determining the type of interaction action between the suspicious person and the interactive product based on the video stream includes: When the interaction between the suspicious person and the product is detected, the video stream is frame extracted, and human posture recognition is performed based on the extracted frame images to obtain the positional change features of the suspicious person's hand feature points. Based on the location change characteristics and the product status change data, the type of interaction action between the suspicious person and the interacting product is determined.

6. The method for detecting abnormal behavior in supermarkets according to claim 1, characterized in that, The process of determining the type of theft or damage behavior and its confidence level of the suspicious person by combining the behavioral trajectory, the product status change data, and the interaction action type includes: Determine the pre-trained theft detection model; The behavioral trajectory, the product status change data, and the interaction action type are input into the theft and damage identification model, and the theft and damage behavior type and its confidence level of the suspicious person are output.

7. The method for detecting abnormal behavior in supermarkets according to any one of claims 1 to 6, characterized in that, The method further includes: When the movement trajectories of multiple customers within a supermarket are determined, heat maps of each area within the supermarket are generated. These heat maps are used to characterize the customer dwell density in each area.

8. A device for detecting abnormal behavior in supermarkets, characterized in that, The device includes: The video acquisition module is used to identify the target customer to be detected and to acquire video streams from multiple cameras within the supermarket. The trajectory determination module is used to perform target detection and tracking based on the video stream, generate the movement trajectory of the target customer in the supermarket, and determine the corresponding product status change data on the movement trajectory; The initial detection module is used to detect suspicious behavior of the target customer based on the movement trajectory and the product status change data, and to mark the target customer as a suspicious person when suspicious behavior is detected. The secondary detection module is used to determine the type of interaction action between the marked suspicious person and the interactive product based on the video stream, and to determine the type of theft or damage behavior of the suspicious person and its confidence level by combining the behavior trajectory, the product status change data and the interaction action type. An anomaly warning module is used to generate an anomaly warning signal of the corresponding level when the confidence level exceeds a preset threshold.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the supermarket abnormal behavior detection method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the supermarket abnormal behavior detection method as described in any one of claims 1 to 7.

Images