Behavior analysis method applied to self-checkout device, electronic device, and storage medium

Abstract

The application provides a behavior analysis method applied to a self-checkout device, an electronic device and a storage medium. According to an embodiment of the application, a sensor is arranged in a code scanning operation area corresponding to the self-checkout device, the self-checkout device receives user behavior data detected and uploaded by the sensor, and performs behavior analysis on the user behavior data to identify whether a code scanning operation of an article is indicated, in a case where it is determined that the code scanning operation of the article is indicated, a historical code scanning operation having a time correlation with the user behavior data is searched from historical code scanning operations of the self-checkout device, and an abnormal operation behavior label is added to the user behavior data when the correlated historical code scanning operation is not found, active monitoring of commodity missing scanning behavior in a self-checkout scenario is realized, through double verification of sensor action monitoring and historical code scanning operation, the identification accuracy of missing scanning behavior is improved, and the financial loss caused to a merchant due to commodity missing scanning is effectively reduced.

Description

Technical Field

[0001] This application relates to the field of commodity transaction technology, and in particular to a behavior analysis method, electronic device, storage medium, and computer program product applied to self-checkout equipment. Background Technology

[0002] With the large-scale deployment and application of self-checkout machines in offline retail scenarios such as supermarkets and convenience stores, users can complete operations such as scanning, paying, and bagging goods themselves, effectively reducing merchants' manual checkout costs and improving checkout efficiency. However, in self-checkout scenarios where there is no one on duty or only a few assistants, there are various intentional or unintentional instances of missed scanning of goods, such as scanning without actual payment or skipping the scanning of some items, which has become one of the main factors leading to losses for merchants.

[0003] To address the aforementioned risk of missed scans, various loss prevention methods have been developed in existing technologies. For example, video-based loss prevention solutions deploy cameras in self-checkout areas to capture user activity videos and use computer vision technology to identify abnormal behavior, such as placing items in shopping bags without scanning them. However, this solution relies on clear facial and motion images and is significantly affected by factors such as light obstruction, camera blind spots, and users wearing masks, resulting in insufficient accuracy in detecting concealed or unintentional missed scans. Another approach uses the ratio of scan counts to the number of items paid for. However, if a user simply brings their hand near the scanner without carrying any items, or if the barcode is damaged or the scanner malfunctions, requiring multiple scans before successful entry, it means the user hasn't actually missed a scan. This could lead to misjudgments, negatively impacting user experience and increasing manual verification costs.

[0004] Therefore, there is an urgent need to provide a behavior analysis method that can accurately identify missed product scanning behavior in self-checkout scenarios and effectively reduce the occurrence rate of the above-mentioned false alarm scenarios, so as to solve the technical problems of insufficient detection accuracy and high false alarm rate in existing loss prevention technologies. Summary of the Invention

[0005] This application provides a behavior analysis method, electronic device, storage medium, and computer program product for use in self-checkout equipment to solve one or more of the aforementioned technical problems.

[0006] In a first aspect, embodiments of this application provide a behavior analysis method applied to a self-checkout device. The self-checkout device has a sensor installed in its corresponding barcode scanning area. The method includes: receiving user behavior data detected and uploaded by the sensor; performing behavior analysis on the user behavior data to identify whether the user behavior data indicates a barcode scanning operation; if the user behavior data indicates a barcode scanning operation, searching for historical barcode scanning operations that are time-related to the user behavior data from the historical barcode scanning operations of the self-checkout device; if no historical barcode scanning operations that are time-related to the user behavior data are found, adding an abnormal operation tag to the user behavior data.

[0007] Secondly, embodiments of this application provide a data acquisition method applied to a sensor, comprising: detecting user behavior data; uploading the user behavior data to a self-checkout device; wherein the sensor is disposed in the barcode scanning operation area corresponding to the self-checkout device, the self-checkout device is used to perform behavior analysis on the user behavior data to identify whether the user behavior data indicates an item barcode scanning operation, and if it is determined that the user behavior data indicates an item barcode scanning operation, searching for historical barcode scanning operations that are time-related to the user behavior data from the historical barcode scanning operations of the self-checkout device, and if no historical barcode scanning operations that are time-related to the user behavior data are found, adding an abnormal operation behavior tag to the user behavior data.

[0008] Thirdly, this application provides a self-checkout device. The self-checkout device has a sensor installed in its corresponding barcode scanning area. The self-checkout device includes: a data receiving module for receiving user behavior data detected and uploaded by the sensor; a behavior analysis module for performing behavior analysis on the user behavior data to identify whether the user behavior data indicates a barcode scanning operation; an operation lookup module for searching for historical barcode scanning operations that are time-related to the user behavior data from the historical barcode scanning operations of the self-checkout device if the user behavior data indicates a barcode scanning operation; and a tag adding module for adding tags indicating abnormal operation behavior to the user behavior data if no historical barcode scanning operation that is time-related to the user behavior data is found.

[0009] Fourthly, this application provides a sensor, including: a behavior detection module for detecting user behavior data; and a behavior upload module for uploading the user behavior data to a self-service checkout device. The sensor is located in the barcode scanning area of ​​the self-service checkout device. The self-service checkout device performs behavior analysis on the user behavior data to identify whether the user behavior data indicates a barcode scanning operation. If the user behavior data indicates a barcode scanning operation, the device searches for historical barcode scanning operations that are time-related to the user behavior data from its historical barcode scanning operations. If no historical barcode scanning operations that are time-related to the user behavior data are found, the device adds a label indicating abnormal operation behavior to the user behavior data.

[0010] Fifthly, embodiments of this application provide a self-checkout system, including a self-checkout device and a sensor. The sensor is disposed in the scanning operation area corresponding to the self-checkout device. The self-checkout device includes: a data receiving module for receiving user behavior data detected and uploaded by the sensor; a behavior analysis module for performing behavior analysis on the user behavior data to identify whether the user behavior data indicates an item scanning operation; an operation search module for searching for historical scanning operations that are time-related to the user behavior data from the historical scanning operations of the self-checkout device when it is determined that the user behavior data indicates an item scanning operation; and a tag adding module for adding a tag of abnormal operation behavior to the user behavior data when no historical scanning operation that is time-related to the user behavior data is found.

[0011] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0012] In a seventh aspect, embodiments of this application provide a computer program product, wherein the computer program product includes a computer program that, when executed by a processor, implements the above-described method.

[0013] According to the embodiments of this application, sensors are installed in the barcode scanning area of ​​the self-checkout device. The self-checkout device receives user behavior data detected and uploaded by the sensors, and performs behavioral analysis on the user behavior data to identify whether an item barcode scanning operation is indicated. If an item barcode scanning operation is indicated, the device searches for historical barcode scanning operations that are time-related to the user behavior data from its historical barcode scanning operations. If no related historical barcode scanning operation is found, an abnormal operation behavior tag is added to the user behavior data, thereby achieving proactive monitoring of missed item scanning in self-checkout scenarios. Through dual verification of sensor action monitoring and historical barcode scanning operations, the accuracy of missed scanning behavior identification is significantly improved, thereby effectively reducing financial losses to merchants caused by missed item scanning. Compared with traditional video loss prevention solutions, this solution does not require the deployment of high-definition cameras, avoiding the impact of factors such as light obstruction, blind spots, and users wearing masks on detection accuracy. Furthermore, it avoids the collection and storage of biometric information such as facial features, effectively mitigating user privacy compliance risks. Moreover, the sensor solution is far less expensive than video surveillance systems, making deployment more lightweight. Compared to solutions that rely solely on scanning log statistics, this application can effectively distinguish between risky operations that involve scanning but no barcode recognition and false alarm scenarios such as a hand approaching a product without a corresponding item, a non-barcode item approaching, or a successful scan after multiple attempts. By analyzing the correlation between actions and records, it significantly reduces the false alarm rate and avoids impacting user experience and reducing manual review costs due to frequent misjudgments.

[0014] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0015] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0016] Figure 1 This paper shows a schematic diagram of the deployment design of the self-checkout system in an embodiment of this application; Figure 2A-2G This illustration shows a user scanning a code to pay for their bill using a self-service checkout system, as shown in an embodiment of this application. Figure 3 This illustration shows a user's QR code checkout process in one example of an embodiment of this application; Figure 4 A flowchart of a behavior analysis method for self-checkout equipment provided in an embodiment of this application is shown; Figure 5 A flowchart of a data acquisition method for a sensor provided in an embodiment of this application is shown; Figure 6 This paper shows a structural block diagram of a self-checkout device provided in an embodiment of this application; Figure 7 A structural block diagram of a sensor provided in an embodiment of this application is shown; Figure 8 This application illustrates a structural block diagram of a self-checkout system provided in an embodiment of the present application; and Figure 9 A block diagram of an electronic device used to implement embodiments of this application is shown. Detailed Implementation

[0017] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0018] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0019] Self-checkout (SQ) devices are intelligent terminal devices deployed in retail stores such as supermarkets and convenience stores, allowing users to complete the checkout process themselves, including scanning, paying, and bagging goods. These devices typically integrate hardware modules such as a barcode scanner (for reading product barcodes), a touchscreen display (for displaying product information and an interactive interface), a payment terminal (supporting multiple payment methods such as card swiping, barcode scanning, and facial recognition), and a receipt printer. Unlike traditional cashier counters, SQ devices do not require cashier intervention; users can complete the entire checkout process independently. Each SQ device has a corresponding barcode scanning area, which is the physical space on or in front of the device where the user holds the product barcode close to allow the scanner to successfully read the barcode information. This area usually corresponds spatially to the integrated scanning window (such as a laser or red light scanner) and is the interactive area where the user performs the scanning action. In this application, a sensor is installed in the barcode scanning area of ​​the self-checkout device to detect the user's operation behavior. The sensor detects changes in the distance between itself and the item to determine whether the item has entered the barcode scanning area, thereby determining whether the user is performing or about to perform a barcode scanning operation.

[0020] In one embodiment, multiple sensors are arranged in the barcode scanning area, with at least two sensors positioned opposite each other. The design aims to address the fact that the detection range of a single sensor (such as an infrared photodiode) is typically a cone or fan-shaped area. When a sensor detects an object approaching, it's necessary to further distinguish whether the object is within the scanning area in front of the self-checkout machine's scanning window, at another location on the self-checkout machine, or even simply passing through a more distant area. If the detection ranges of multiple sensors are in one or similar directions, it's impossible to accurately identify whether user behavior occurred within the barcode scanning area. Therefore, by positioning at least two sensors opposite each other (e.g., deployed on the left and right sides, or above and below, of the operating area), a valid action is only considered when both sensors detect the object (i.e., the object must simultaneously be within the overlapping detection areas of both sensors).

[0021] In one embodiment, one side of the scanning operation area is a self-checkout device, the opposite side is the user, and the other two sides are perpendicular to the self-checkout device. The sensor can be set in a direction perpendicular to the self-checkout device. The purpose of setting the sensor in a direction perpendicular to the self-checkout device is that the distance between the user and the sensor usually changes in a time sequence from far to near and then from near to far during the scanning process. The distance change pattern can be used to determine whether the user's behavior conforms to the scanning pattern.

[0022] Figure 1 A schematic diagram of the deployment design of the self-checkout system in an embodiment of this application is shown. The sensors deployed on both sides can also be considered as sensor modules belonging to the same sensing device and sharing a single chip.

[0023] The sensors can specifically be infrared reflective sensors or distance sensors (such as ultrasonic distance sensors). Each sensor can include one or more sensor modules, each of which includes a transmitting unit and a receiving unit. The transmitting unit is used to transmit detection signals (such as infrared light or ultrasound), and the receiving unit is used to receive the echo signals reflected back from the user's hand or objects, and convert the intensity or time difference of the reflected signals into electrical signals to be output to the chip. The self-checkout device communicates with the sensor via a wired USB interface or via the sensor's built-in wireless communication chip. Taking USB interface communication as an example, after the sensor establishes a physical connection with the self-checkout device via the USB interface and completes the device driver installation and serial port parameter configuration, the main control host of the self-checkout device will authorize and verify the sensor (e.g., by confirming the connection legitimacy through device ID or handshake protocol). After authorization, the main control host starts a serial port listening thread to continuously listen for user behavior data returned by the sensor through the virtual serial port. The sensor chip is a USB-to-serial bridge chip that performs analog-to-digital conversion and filtering on the electrical signals uploaded from the two sensors, and then reports the data to the self-checkout device via the USB interface for subsequent behavior analysis.

[0024] The self-checkout device receives and uploads user behavior data detected by sensors, performs behavioral analysis on the user behavior data to identify whether the user behavior data indicates an item scanning operation, that is, to determine whether it meets the characteristics of an item scanning operation. For example, if the distance between the item and the sensor follows a distance change pattern from far to near and then far again, it is identified as the user performing a scanning operation by bringing the item close to the scanning window. Through this identification step, every suspected scanning behavior event can be accurately captured from the continuous sensor data stream.

[0025] After recognizing that a user has performed an item scanning operation, the system further searches the historical scanning records stored by the self-checkout device for a successful scan record that is temporally associated with that scanning action. For example, if the sensor detects that an item approaches the scanning window at time T1, lasting for 0.5 seconds, the system searches the historical scanning records to see if the time difference between the successful scan time and T1 is less than a preset threshold (e.g., 1 second). If a historical scanning record is found where the successful scan time is within 1 second of T1, the item scanning operation is considered a successful barcode recognition; if no temporally matching successful scan record is found, the item scanning operation is considered to have not produced a valid barcode recognition result, and is considered a suspected missed scan. Through this time-related search mechanism, this application achieves the correlation verification between scanning actions and scanning results.

[0026] If no successful barcode scanning record is found in the historical scanning records that is temporally associated with the item scanning operation, it can be determined that the current scanning operation did not yield a corresponding valid barcode recognition result, and an abnormal operation behavior tag can be added to this user behavior data. For example, if the sensor detects that the user moves the product close to the scanning window, but there are no successful scanning records in the historical scanning records within a preset time window before and after the action (e.g., within 1 second before and after), it is determined that although the user made a scanning action, the product's barcode was not actually entered, which is a suspected missed scan behavior. Therefore, this behavior data is tagged as "abnormal operation" or "suspected missed scan" for subsequent loss prevention processing, such as prompting the user to rescan the barcode on the device interface or uploading the record to the abnormal behavior log for subsequent analysis. Through this step, this application realizes the automatic identification and marking of risky operations that involve scanning actions but no scanning results.

[0027] During the sensing process of an operation, the sensing device transmits data multiple times. Therefore, a time window can be set. If no new data is available within this time window, the previous multiple data transmissions are packaged and uploaded together for subsequent analysis. The user behavior data detected and uploaded by the aforementioned sensors can include a sequence of sensor data within a set time window, starting from the time when the object is detected within the sensor's detection range. In other words, the sensor data sequence collected within a preset time window (e.g., from 500 milliseconds before to 1000 milliseconds after the start point) is considered user behavior data, starting from the time the sensor first detects the object entering its detection range. The sensor data sequence is marked with the distance data between the object and the sensor, as well as the corresponding timestamp, thus completely recording the complete dynamic trajectory of the object moving from the sensor to the sensor during a single scanning operation. This provides a refined data foundation for subsequent behavior analysis (such as determining whether the object's approach distance is less than a threshold or whether the approach duration is within a reasonable range). The content format of the sensor data transmission can include the following fields: fixed identifier (marking the start of data), left / right side identifier (distinguishing between the left and right sensors), distance value (distance between the object and the sensor), and timestamp.

[0028] Typically, when an item approaches a sensor, it's determined that the user is about to scan a product or other barcode; when the item moves away from the sensor, it's determined that the user has completed the scanning action. Abstracting this physical scenario into a data model, each valid scanning action follows a "from far to near, then from near to far" pattern in terms of distance changes between the item and the sensor. That is, the item approaches the sensor from a distance, reaches its closest point, and then moves away from the sensor from various directions. Based on this pattern, when analyzing user behavior data to identify whether it indicates a barcode scanning operation, for each sensor, the distance data between the item and the sensor, marked according to the sensor data sequence, and the corresponding timestamp, can be analyzed to determine if the distance change between the item and the sensor follows a chronological pattern of "from far to near, then from near to far." If both patterns are present, then the user behavior data indicates a barcode scanning operation.

[0029] When the item moves away from the sensing device, it is determined that the user has completed the scanning action. Figure 2A , Figure 2B , Figure 2C , Figure 2D , Figure 2E , Figure 2F and Figure 2G The illustrations show schematic diagrams of user scanning for payment in a self-checkout system according to embodiments of this application. Sensors are located on the left and right sides of the scanning operation area. When the physical scenario is broken down, possible operational trajectories include, but are not limited to: an item entering from the right sensor and exiting from the left sensor; an item entering from the right sensor and exiting from the right sensor; an item entering from the left sensor and exiting from the left sensor; an item entering from the front and exiting from the front; an item entering from the front and exiting from the left or right; and an item entering from the side and exiting from the front. By analyzing the order of entry and exit of the item relative to the left and right sensors and the changes in distance, the granularity of scanning behavior recognition can be further refined, improving the accuracy of behavior analysis.

[0030] Before analyzing whether the distance changes between the item and the sensor follow a chronological pattern of moving from far to near and then from near to far, it's possible to determine whether the item has corresponding distance data in the sensor data sequences of multiple sensors at the same time point, based on the distance data and timestamps corresponding to the marked items in the sensor data sequences. If corresponding distance data is present in all sequences, a valid user action is detected within the overlapping detection area of ​​the sensors. In other words, based on the distance data and timestamps in the data sequences reported by each sensor, it's determined whether multiple sensors detected the item at the same time point (or adjacent time points with a time difference within a preset allowable range). Specifically, one data point in each sensor's data sequence has the same or similar timestamp as another data point in another sensor's data sequence, and both data points contain valid distance values. If multiple sensors have corresponding distance data at the same time point, the item is confirmed to be within the scanning operation area covered by multiple sensors. If only one side detects the item and the other side does not, the item is determined not to have actually entered the scanning operation area (e.g., the item only passed by one side of the device without entering the scanning window), failing to meet the prerequisites for subsequent entry / exit action analysis. Through the verification mechanism of simultaneous detection by multiple sensors, this application ensures that subsequent scanning action recognition only targets valid behaviors of items actually entering the scanning operation area, effectively eliminating false detections caused by unilateral false triggers (such as other parts of the user's body passing by, non-scanned items approaching, etc.), and improving the accuracy of behavior analysis.

[0031] When searching for historical barcode scanning operations that are time-related to user behavior data from the historical scanning operations of self-checkout devices, the process can begin by checking if a corresponding historical barcode scanning operation exists within a first time range corresponding to the user behavior data. If no corresponding historical barcode scanning operation is found within the first time range, the search continues within a second time range extended from the first time range. The first time range can be the time point when the distance between the item and the sensor is measured; that is, the timestamp of the minimum distance is taken as the timestamp of the successful scan, and the search is conducted to see if a corresponding historical barcode scanning operation exists at that time point. If no operation is found within the first time range, the search range is further expanded to the second time range (e.g., extending the time window to within 1 second before and after the timestamp) to search again for a corresponding historical barcode scanning operation. This two-level time window search mechanism can quickly and accurately match scanning actions and records under normal circumstances, while also being compatible with scenarios where scanning records are delayed due to device response latency or barcode recognition lag, thus reducing the false negative rate while maintaining detection accuracy. For example, when the sensor detects a barcode scanning action, if the barcode fails to be recognized on the first attempt due to dirt and only succeeds on the second attempt, the successful recording time may be later than the time of sensor detection. In this case, the second time window can capture the delayed record and avoid misjudging it as a missed scan.

[0032] Accordingly, this application also provides a data acquisition scheme applied to sensors. The sensors detect user behavior data and upload the user behavior data to the self-checkout device. The sensors are set in the corresponding barcode scanning area of ​​the self-checkout device. The self-checkout device is used to perform behavior analysis on the user behavior data to identify whether the user behavior data indicates a barcode scanning operation. If it is determined that the user behavior data indicates a barcode scanning operation, the device searches for historical barcode scanning operations that are time-related to the user behavior data from the historical barcode scanning operations of the self-checkout device. If no historical barcode scanning operations that are time-related to the user behavior data are found, the device adds an abnormal operation tag to the user behavior data.

[0033] When the sensor detects user behavior data, it continuously detects whether there are any objects within its detection range. If an object is detected within the sensor's detection range, it collects a sequence of sensor data within a set time window, starting from the time point when the detected object is within the sensor's detection range, as one instance of user behavior data.

[0034] In addition, sensors can add sensor identifiers to user behavior data, including sensor data sequences, distance data between objects and sensors, and corresponding timestamps.

[0035] Figure 3 This illustration shows a schematic diagram of a user's QR code checkout process in an example of an embodiment of this application. It can be used in conjunction with a video loss prevention solution. When a suspected missed scan event is detected through video loss prevention, it first determines whether sensor data was reported at the time of the event. If sensor data exists, its validity is verified. After successful verification, the nearest distance data and the corresponding time point of the scan action are parsed out. Subsequently, historical scan records matching the current scan action are searched from the scan logs of the autonomous scanning device. This search can specifically include two operations: one directly checks for the existence of a corresponding historical scan record, and the other searches again after a 1-second delay to cover potential delays in scan entry due to device response latency. If no matching historical scan record is found in either search, the scan action is deemed to have no valid recognition result. After the process ends, the relevant data of this missed scan event (including sensor data, scan action time, search results, etc.) is reported to the backend server or data platform for subsequent loss prevention analysis and processing. Through the above process, this application realizes the linkage between video loss prevention and sensor loss prevention, and uses sensor data to verify whether the suspected missed scan event detected by the video has actually occurred, thereby reducing the false alarm rate.

[0036] The execution entity in this application embodiment can be an application, service, instance, functional module in software form, virtual machine (VM), container, or cloud server, or hardware device with data processing capabilities (such as server or terminal device) or hardware chip (such as CPU, GPU, FPGA, NPU, AI accelerator card, or DPU). The device for providing the service can be deployed on the computing device of the application providing the corresponding service or on a cloud computing platform providing computing power, storage, and network resources. The cloud computing platform can provide services in the following modes: IaaS (Infrastructure as a Service), PaaS (Platform as a Service), SaaS (Software as a Service), or DaaS (Data as a Service). Taking the platform providing SaaS (Software as a Service) as an example, the cloud computing platform can utilize its own computing resources to provide one or more of the above steps, and the specific application architecture can be built according to service requirements.

[0037] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0038] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0039] This application provides a behavior analysis method for self-checkout equipment, such as... Figure 4 The diagram shows a flowchart of a behavior analysis method 400 applied to a self-checkout device according to an embodiment of this application. The self-checkout device has a sensor installed in its corresponding barcode scanning area. The method 400 may include: in step S401, receiving user behavior data detected and uploaded by the sensor; in step S402, performing behavior analysis on the user behavior data to identify whether the user behavior data indicates a barcode scanning operation; in step S403, if it is determined that the user behavior data indicates a barcode scanning operation, searching for historical barcode scanning operations that are time-related to the user behavior data from the historical barcode scanning operations of the self-checkout device; and in step S404, if no historical barcode scanning operation that is time-related to the user behavior data is found, adding a tag for abnormal operation behavior to the user behavior data.

[0040] In one embodiment, each sensor corresponds to one or more sensor modules, and the self-checkout device communicates with the sensor via a USB interface via wired communication or via a wireless communication chip built into the sensor.

[0041] In one embodiment, the sensor is a distance sensor, and the user behavior data includes a sequence of sensor data within a set time window, starting from the time point at which the object is detected to be within the detection range of the sensor. The sensor data sequence is marked with distance data between the object and the sensor and a corresponding timestamp.

[0042] In one embodiment, the sensors include multiple sensors arranged in a direction perpendicular to the self-checkout device, with at least two sensors arranged opposite each other. The detection areas of the oppositely arranged sensors overlap the scanning operation area of ​​the self-checkout device. The step of performing behavioral analysis on the user behavior data to identify whether the user behavior data indicates an item scanning operation includes: for each sensor, analyzing whether the change in distance between the item and the sensor follows a chronological pattern of moving from far to near and then from near to far, based on the distance data between the marked item and the sensor corresponding to the sensor data sequence and the corresponding timestamp; if both patterns exist, then it is determined that the user behavior data indicates an item scanning operation.

[0043] In one embodiment, before analyzing whether the change in distance between the item and the sensor follows a chronological pattern of moving from far to near and then from near to far, the step of performing behavioral analysis on the user behavior data to identify whether the user behavior data indicates an item scanning operation further includes: determining, based on the distance data between the item and the sensor corresponding to the marked data sequence of the sensor data and the corresponding timestamp, whether the item has corresponding distance data in the sensor data sequences corresponding to multiple sensors at the same time point; if corresponding distance data exists in all sequences, determining that a valid user behavior was detected within the detection area where the sensors overlap.

[0044] In one embodiment, the plurality of sensors are respectively deployed on the left and right sides of the operating area, or the plurality of sensors are respectively deployed on the upper and lower sides of the operating area.

[0045] In one embodiment, the step of searching for historical scanning operations that are time-related to the user behavior data from the historical scanning operations of the self-checkout device includes: searching for whether there is a corresponding historical scanning operation within a first time range corresponding to the user behavior data from the historical scanning operations of the self-checkout device; if no corresponding historical scanning operation is found within the time range, searching for whether there is a corresponding historical scanning operation within a second time range extended from the first time range.

[0046] This application provides a data acquisition method applied to sensors, such as... Figure 5The diagram shows a flowchart of a data acquisition method 500 applied to a sensor according to an embodiment of this application. The method 500 may include: in step S501, detecting user behavior data; in step S502, uploading the user behavior data to a self-checkout device; wherein the sensor is disposed in the scanning operation area corresponding to the self-checkout device, and the self-checkout device is used to perform behavior analysis on the user behavior data to identify whether the user behavior data indicates an item scanning operation; if it is determined that the user behavior data indicates an item scanning operation, searching for historical scanning operations with a time correlation to the user behavior data from the historical scanning operations of the self-checkout device; and if no historical scanning operations with a time correlation to the user behavior data are found, adding an abnormal operation behavior tag to the user behavior data.

[0047] In one embodiment, the detection of user behavior data includes: continuously detecting whether an item is within the detection range of the sensor; and when an item is detected within the detection range of the sensor, collecting a sequence of sensor data within a set time window starting from the time point when the detected item is within the detection range of the sensor, as one instance of user behavior data.

[0048] In one embodiment, the method further includes: adding sensor identifiers to the user behavior data; and marking the distance data between the item and the sensor and the corresponding timestamp for the sensor data sequence included in the user behavior data.

[0049] According to the embodiments of this application, sensors are installed in the barcode scanning area of ​​the self-checkout device. The self-checkout device receives user behavior data detected and uploaded by the sensors, and performs behavioral analysis on the user behavior data to identify whether an item barcode scanning operation is indicated. If an item barcode scanning operation is indicated, the device searches for historical barcode scanning operations that are time-related to the user behavior data from its historical barcode scanning operations. If no related historical barcode scanning operation is found, an abnormal operation behavior tag is added to the user behavior data, thereby achieving proactive monitoring of missed item scanning in self-checkout scenarios. Through dual verification of sensor action monitoring and historical barcode scanning operations, the accuracy of missed scanning behavior identification is significantly improved, thereby effectively reducing financial losses to merchants caused by missed item scanning. Compared with traditional video loss prevention solutions, this solution does not require the deployment of high-definition cameras, avoiding the impact of factors such as light obstruction, blind spots, and users wearing masks on detection accuracy. Furthermore, it avoids the collection and storage of biometric information such as facial features, effectively mitigating user privacy compliance risks. Moreover, the sensor solution is far less expensive than video surveillance systems, making deployment more lightweight. Compared to solutions that rely solely on scanning log statistics, this application can effectively distinguish between risky operations that involve scanning but no barcode recognition and false alarm scenarios such as a hand approaching a product without a corresponding item, a non-barcode item approaching, or a successful scan after multiple attempts. By analyzing the correlation between actions and records, it significantly reduces the false alarm rate and avoids impacting user experience and reducing manual review costs due to frequent misjudgments.

[0050] Corresponding to the examples and method embodiments provided in this application, this application also provides a self-checkout device. For example... Figure 6 The diagram shows a structural block diagram of a self-checkout device 600 according to an embodiment of this application. The self-checkout device has a sensor installed in the corresponding barcode scanning area. The self-checkout device includes: a data receiving module 601 for receiving user behavior data detected and uploaded by the sensor; a behavior analysis module 602 for performing behavior analysis on the user behavior data to identify whether the user behavior data indicates a barcode scanning operation; an operation search module 603 for searching for historical barcode scanning operations that are time-related to the user behavior data from the historical barcode scanning operations of the self-checkout device if it is determined that the user behavior data indicates a barcode scanning operation; and a tag adding module 604 for adding a tag indicating abnormal operation behavior to the user behavior data if no historical barcode scanning operation that is time-related to the user behavior data is found.

[0051] In one embodiment, each sensor corresponds to one or more sensor modules, and the self-checkout device communicates with the sensor via a USB interface via wired communication or via a wireless communication chip built into the sensor.

[0052] In one embodiment, the sensor is a distance sensor, and the user behavior data includes a sequence of sensor data within a set time window, starting from the time point at which the object is detected to be within the detection range of the sensor. The sensor data sequence is marked with distance data between the object and the sensor and a corresponding timestamp.

[0053] In one embodiment, the sensors include multiple sensors arranged in a direction perpendicular to the self-checkout device, with at least two sensors arranged opposite each other. The detection areas of the oppositely arranged sensors overlap the scanning operation area of ​​the self-checkout device. The behavior analysis module includes an action analysis submodule, used to analyze, for each sensor, whether there is a chronological pattern of distance changes between the item and the sensor, based on the distance data between the marked item and the sensor corresponding to the sensor data sequence and the corresponding timestamp; if both patterns exist, then the user behavior data indicates an item scanning operation.

[0054] In one embodiment, the behavior analysis module further includes a data determination submodule, used to determine, before analyzing whether the change in distance between the item and the sensor follows a chronological pattern of moving from far to near and then from near to far, whether the item has corresponding distance data in the sensor data sequences corresponding to multiple sensors at the same time point, based on the distance data between the item and the sensor marked in the sensor data sequence and the corresponding timestamp, and if corresponding distance data is present in all of them, to determine that a valid user behavior was detected in the detection area where the sensors overlap.

[0055] In one embodiment, the plurality of sensors are respectively deployed on the left and right sides of the operating area, or the plurality of sensors are respectively deployed on the upper and lower sides of the operating area.

[0056] In one embodiment, the operation search module is specifically used to search for whether there is a corresponding historical scan operation within a first time range corresponding to the user behavior data from the historical scan operations of the self-service checkout device; if no corresponding historical scan operation is found within the time range, it searches for whether there is a corresponding historical scan operation within a second time range extended from the first time range.

[0057] Corresponding to the examples and method embodiments provided in this application, this application also provides a sensor. For example... Figure 7The diagram shown is a structural block diagram of a sensor 700 according to an embodiment of this application. The sensor 700 may include: a behavior detection module 701 for detecting user behavior data; and a behavior upload module 702 for uploading the user behavior data to a self-service checkout device. The sensor is located in the barcode scanning area of ​​the self-service checkout device. The self-service checkout device performs behavior analysis on the user behavior data to identify whether the user behavior data indicates a barcode scanning operation. If the user behavior data indicates a barcode scanning operation, the device searches for historical barcode scanning operations that are time-related to the user behavior data from its historical barcode scanning operations. If no historical barcode scanning operation is found that is time-related to the user behavior data, the device adds an abnormal operation tag to the user behavior data.

[0058] In one embodiment, the behavior detection module package is specifically used to continuously detect whether there is an object within the detection range of the sensor; when an object is detected within the detection range of the sensor, it collects a sequence of sensor data within a set time window starting from the time point when the detected object is within the detection range of the sensor, as a user behavior data.

[0059] In one embodiment, the sensor further includes: a data adding module, used to add sensor identifiers to the user behavior data; and to mark the distance data between the item and the sensor and the corresponding timestamp for the sensor data sequence included in the user behavior data.

[0060] Corresponding to the examples and method embodiments provided in this application, this application also provides a self-checkout system. For example... Figure 8 The diagram shows a structural block diagram of a self-checkout system 800 according to an embodiment of this application. It includes a self-checkout device 801 and a sensor 802. The sensor is located in the scanning operation area corresponding to the self-checkout device. The self-checkout device includes: a data receiving module for receiving user behavior data detected and uploaded by the sensor; a behavior analysis module for performing behavior analysis on the user behavior data to identify whether the user behavior data indicates a barcode scanning operation; an operation lookup module for searching for historical barcode scanning operations that are time-related to the user behavior data from the historical barcode scanning operations of the self-checkout device if it is determined that the user behavior data indicates a barcode scanning operation; and a tag adding module for adding a tag indicating abnormal operation behavior to the user behavior data if no historical barcode scanning operation that is time-related to the user behavior data is found.

[0061] According to the embodiments of this application, sensors are installed in the barcode scanning area of ​​the self-checkout device. The self-checkout device receives user behavior data detected and uploaded by the sensors, and performs behavioral analysis on the user behavior data to identify whether an item barcode scanning operation is indicated. If an item barcode scanning operation is indicated, the device searches for historical barcode scanning operations that are time-related to the user behavior data from its historical barcode scanning operations. If no related historical barcode scanning operation is found, an abnormal operation behavior tag is added to the user behavior data, thereby achieving proactive monitoring of missed item scanning in self-checkout scenarios. Through dual verification of sensor action monitoring and historical barcode scanning operations, the accuracy of missed scanning behavior identification is significantly improved, thereby effectively reducing financial losses to merchants caused by missed item scanning. Compared with traditional video loss prevention solutions, this solution does not require the deployment of high-definition cameras, avoiding the impact of factors such as light obstruction, blind spots, and users wearing masks on detection accuracy. Furthermore, it avoids the collection and storage of biometric information such as facial features, effectively mitigating user privacy compliance risks. Moreover, the sensor solution is far less expensive than video surveillance systems, making deployment more lightweight. Compared to solutions that rely solely on scanning log statistics, this application can effectively distinguish between risky operations that involve scanning but no barcode recognition and false alarm scenarios such as a hand approaching a product without a corresponding item, a non-barcode item approaching, or a successful scan after multiple attempts. By analyzing the correlation between actions and records, it significantly reduces the false alarm rate and avoids impacting user experience and reducing manual review costs due to frequent misjudgments.

[0062] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0063] Figure 9 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 9 As shown, the electronic device includes a memory 901 and a processor 902. The memory 901 stores a computer program that can run on the processor 902. When the processor 902 executes the computer program, it implements the method described in the above embodiments. The number of memories 901 and processors 902 can be one or more.

[0064] The electronic device also includes: The communication interface 903 is used to communicate with external devices and exchange and transmit data.

[0065] If the memory 901, processor 902, and communication interface 903 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0066] Optionally, in a specific implementation, if the memory 901, processor 902, and communication interface 903 are integrated on a single chip, then the memory 901, processor 902, and communication interface 903 can communicate with each other through an internal interface.

[0067] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0068] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the methods provided in any embodiment of this application.

[0069] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0070] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in this application.

[0071] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0072] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0073] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0074] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0075] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0076] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0077] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0078] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0079] Furthermore, the functional units in the various embodiments of this application can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0080] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A behavior analysis method for self-checkout equipment, wherein a sensor is installed in the barcode scanning area of ​​the self-checkout equipment, the method comprising: Receive user behavior data detected and uploaded by the sensor; Behavioral analysis is performed on the user behavior data to identify whether the user behavior data indicates an item scanning operation; If the user behavior data indicates a barcode scanning operation, search for historical barcode scanning operations that are time-related to the user behavior data from the historical barcode scanning operations of the self-checkout device. If no historical QR code scanning operations with a time correlation to the user behavior data are found, an abnormal operation behavior label is added to the user behavior data.

2. The method according to claim 1, wherein, Each sensor includes one or more sensor modules. The self-checkout device communicates with the sensor via a USB interface or via a wireless communication chip built into the sensor.

3. The method according to claim 1, wherein, The sensor is a distance sensor, and the user behavior data includes a sequence of sensor data within a set time window, starting from the time point when the object is detected to be within the detection range of the sensor. The sensor data sequence is marked with distance data between the object and the sensor and a corresponding timestamp.

4. The method according to claim 3, wherein, The step of performing behavioral analysis on the user behavior data to identify whether the user behavior data indicates a barcode scanning operation includes: For each sensor, based on the distance data between the marked item and the sensor and the corresponding timestamp in the sensor data sequence, analyze whether the change in distance between the item and the sensor follows a chronological pattern of moving from far to near and then from near to far. If both exist, then the user behavior data indicates an item scanning operation.

5. The method according to claim 4, wherein, The sensors include multiple sensors, which are arranged in a direction perpendicular to the self-checkout device, and at least two of the multiple sensors are arranged opposite each other, with the detection areas of the oppositely arranged sensors overlapping the scanning operation area of ​​the self-checkout device. Before analyzing whether the change in distance between the item and the sensor follows a chronological pattern of moving from far to near and then from near to far, the step of performing behavioral analysis on the user behavior data to identify whether the user behavior data indicates an item scanning operation further includes: Based on the distance data between the marked item and the sensor and the corresponding timestamp in the sensor data sequence, determine whether the item has corresponding distance data in the sensor data sequences corresponding to multiple sensors at the same time point; Given the existence of corresponding distance data, it is determined that a valid user action was detected within the detection area where the sensors overlap.

6. The method according to claim 5, wherein, The multiple sensors are respectively deployed on the left and right sides of the operating area, or the multiple sensors are respectively deployed on the upper and lower sides of the operating area.

7. The method according to claim 1, wherein, The step of searching for historical QR code scanning operations that are time-related to the user behavior data from the historical QR code scanning operations of the self-checkout device includes: From the historical scanning operations of the self-checkout device, search for whether there is a corresponding historical scanning operation within the first time range corresponding to the user behavior data; If no corresponding historical scanning operation is found within the specified time range, the search continues to look for a corresponding historical scanning operation within a second time range extended from the first time range.

8. A data acquisition method for a sensor, comprising: Detect user behavior data; Upload the user behavior data to the self-service checkout device; The sensor is located in the scanning area of ​​the self-checkout device. The self-checkout device performs behavioral analysis on the user behavior data to identify whether the user behavior data indicates an item scanning operation. If the user behavior data indicates an item scanning operation, the device searches for historical scanning operations that are time-related to the user behavior data from its historical scanning operations. If no historical scanning operations that are time-related to the user behavior data are found, the device adds an abnormal operation tag to the user behavior data.

9. The method according to claim 8, wherein, The detected user behavior data includes: Continuously detect whether there are any objects within the detection range of the sensor; When an object is detected within the detection range of the sensor, a sequence of sensor data within a set time window, starting from the time point when the object was detected within the detection range of the sensor, is collected as a user behavior data.

10. The method according to claim 8, wherein, Also includes: Add sensor identifiers to the user behavior data; The user behavior data includes sensor data sequences, distance data between marked items and sensors, and corresponding timestamps.

11. A self-checkout device, wherein a sensor is provided in the barcode scanning area corresponding to the self-checkout device, the self-checkout device comprising: The data receiving module is used to receive user behavior data detected and uploaded by the sensor; The behavior analysis module is used to perform behavior analysis on the user behavior data to identify whether the user behavior data indicates an item scanning operation; The operation search module is used to search for historical barcode scanning operations that are time-related to the user behavior data from the historical barcode scanning operations of the self-checkout device when it is determined that the user behavior data indicates a barcode scanning operation. The tag adding module is used to add tags for abnormal operation behavior to the user behavior data when no historical QR code scanning operation with time correlation with the user behavior data is found.

12. A sensor, comprising: The behavior detection module is used to detect user behavior data; The behavior upload module is used to upload the user behavior data to the self-service checkout device; The sensor is located in the scanning area of ​​the self-checkout device. The self-checkout device performs behavioral analysis on the user behavior data to identify whether the user behavior data indicates an item scanning operation. If the user behavior data indicates an item scanning operation, the device searches for historical scanning operations that are time-related to the user behavior data from its historical scanning operations. If no historical scanning operations that are time-related to the user behavior data are found, the device adds an abnormal operation tag to the user behavior data.

13. A self-checkout system, comprising a self-checkout device and a sensor, wherein the sensor is disposed in the barcode scanning area corresponding to the self-checkout device; The self-checkout equipment includes: The data receiving module is used to receive user behavior data detected and uploaded by the sensor; The behavior analysis module is used to perform behavior analysis on the user behavior data to identify whether the user behavior data indicates an item scanning operation; The operation search module is used to search for historical barcode scanning operations that are time-related to the user behavior data from the historical barcode scanning operations of the self-checkout device when it is determined that the user behavior data indicates a barcode scanning operation. The tag adding module is used to add tags for abnormal operation behavior to the user behavior data when no historical QR code scanning operation with time correlation with the user behavior data is found.

14. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1-10.

15. A computer program product, wherein, The computer program product includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-10.

Images