Artificial cashier loss prevention method, device and equipment
Through the analysis of real-time video streams and order operation information, the risk behavior in manual cashier is identified and output, which solves the problems of high labor costs and low identification efficiency in the existing technology, improves the discovery rate and identification accuracy of risk behaviors, and reduces the The supermarket lost revenue.
Patent Information
- Application Number
- CN202411506510.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing manual cashier and loss prevention methods have high labor costs and low identification efficiency, low risk behavior discovery rate, and difficult to effectively identify and prevent theft in supermarkets.
By obtaining real-time video streams and order operation information, we identify the interaction behavior between the target user and the target item, and determine the risk identification results based on the matching relationship between the interaction behavior and the order operation information, output abnormal risk behavior prompt information, and based on the multiple within the preset period The risk identification results determine the target risk user.
It reduces the cost of manual supervision, improves the identification efficiency and discovery rate of risk behaviors, realizes accurate identification and timely reminders of risk behaviors in supermarkets, and reduces income losses.
Smart Images

Figure CN119026924A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a manual cashier loss prevention method, device and equipment. Background Art
[0002] In offline retail scenarios such as supermarkets, manual cashiering is still an indispensable cashiering method. However, there may be some risky behaviors in the manual cashiering process, such as consumers hiding goods and cashiers forging transactions, which has caused great income losses to supermarkets.
[0003] In related technologies, risky behaviors in the manual checkout process are mainly confirmed and checked through on-site supervision by internal supermarket personnel combined with manual review of video recordings. This manual checkout loss prevention method has high labor costs, low overall recognition efficiency, and a low detection rate for risky behaviors. Summary of the invention
[0004] Multiple aspects of the present application provide a manual checkout loss prevention method, device and equipment, which can reduce the manpower cost of manual checkout loss prevention and improve recognition efficiency and the detection rate of risky behaviors.
[0005] In a first aspect, an embodiment of the present application provides a manual cashier loss prevention method, comprising:
[0006] Acquire the real-time video stream sent by the shooting device, and acquire the order operation information sent by the target cash register device corresponding to the shooting device;
[0007] According to the real-time video stream, identifying the interaction behavior between the target user and the target object in the real-time video stream;
[0008] Determine and store a risk identification result corresponding to the real-time video stream according to a matching relationship between the interaction behavior and the order operation information;
[0009] If there is abnormal risk behavior in the risk identification result, output target prompt information;
[0010] Determine target risk users based on multiple risk identification results within a preset period.
[0011] In a possible implementation, the step of identifying, according to the real-time video stream, an interactive behavior between a target user and a target object in the real-time video stream includes:
[0012] Detecting a target user and a target object in the real-time video stream, and extracting human feature information corresponding to the target user and object feature information corresponding to the target object;
[0013] Determine, according to the human feature information and the object feature information, a human movement trajectory corresponding to the target user and an object movement trajectory corresponding to the target object;
[0014] The interaction behavior between the target user and the target object is determined according to the human body movement trajectory and the object movement trajectory.
[0015] In a possible implementation, the interactive behavior includes a code scanning behavior; and determining the interactive behavior between the target user and the target object according to the human body movement trajectory and the object movement trajectory includes:
[0016] Determine a first user and a second user among the target users according to an intersection relationship between the human body movement trajectory and the target channel area;
[0017] If the moving track of the object passes through the target code scanning area, it is determined that the first user has a code scanning behavior with the target object;
[0018] If the distance between the moving track of the object and the moving track of the target scanning device is less than a preset distance threshold, it is determined that the first user has a scanning behavior with the target object;
[0019] If the movement track of the object moves from the first area corresponding to the target cash register device to the second area corresponding to the target cash register device, it is determined that the first user has a code scanning behavior with the target object.
[0020] In a possible implementation, determining and storing a risk identification result corresponding to the real-time video stream according to a matching relationship between the interactive behavior and the order operation information includes:
[0021] Determine the order operation status corresponding to the target cash register device according to the order operation information; the order operation status includes a non-transaction status, a transaction status, and a transaction end time;
[0022] For each of the order operation states, according to the matching relationship between the interactive behavior and the order operation information, a risk identification result corresponding to the real-time video stream is determined and stored.
[0023] In a possible implementation, the method further includes:
[0024] In each of the order operation states, if it is detected that the cashier cash box corresponding to the target cashier device is in an open state, it is determined that the interaction behavior does not match the order operation information;
[0025] In a non-transaction state, if the body movement trajectory of the second user and the commodity movement trajectory of the target commodity pass through the target channel area, it is determined that the interaction behavior does not match the order operation information.
[0026] In a possible implementation, the method further includes:
[0027] In the transaction state, if there is transaction order information in the order operation information, and the stay time of the second user and the target product in the target channel area is less than a preset time threshold, it is determined that the interaction behavior does not match the order operation information;
[0028] In the transaction state, if the stay time of the second user and the target product in the target channel area is equal to or greater than the preset time threshold, and there is a code scanning behavior of the first user for the target product in the interaction behavior, but the order operation information does not include the code scanning information corresponding to the target product during the duration of the code scanning behavior, it is determined that the interaction behavior does not match the order operation information;
[0029] In the transaction state, if the order operation information includes modification operation information of the first user, it is determined that the interaction behavior does not match the order operation information.
[0030] In a possible implementation, the method further includes:
[0031] If the order operation information includes the transaction order information in the transaction state, but the order operation information does not include the transaction order information at the transaction end time, it is determined that the interaction behavior does not match the order operation information;
[0032] If, at the end of the transaction, the quantity of goods in the order operation information does not match the target quantity of the target goods corresponding to the second user in the real-time video stream, it is determined that the interaction behavior does not match the order operation information.
[0033] In a possible implementation manner, determining a target risk user according to a plurality of risk identification results within a preset period includes:
[0034] Obtain risk identification results within a preset period and transaction order information corresponding to the risk identification results;
[0035] Determine risk characteristic data corresponding to each first user according to the risk identification result and the transaction order information; the risk characteristic data includes risk behavior time and risk behavior quantity;
[0036] The target risk user is determined from the first users according to the risk characteristic data and a preset risk user identification method.
[0037] In a second aspect, an embodiment of the present application provides a manual cashier loss prevention device, comprising:
[0038] An acquisition module, used to acquire a real-time video stream sent by a shooting device, and to acquire order operation information sent by a target cash register device corresponding to the shooting device;
[0039] An identification module, used to identify the interaction behavior between the target user and the target object in the real-time video stream according to the real-time video stream;
[0040] A first determination module, configured to determine and store a risk identification result corresponding to the real-time video stream according to a matching relationship between the interaction behavior and the order operation information;
[0041] An output module, used for outputting target prompt information if there is abnormal risk behavior in the risk identification result;
[0042] The second determination module is used to determine the target risk user according to a plurality of risk identification results within a preset period.
[0043] In a possible implementation manner, the identification module is specifically used to:
[0044] Detecting a target user and a target object in the real-time video stream, and extracting human feature information corresponding to the target user and object feature information corresponding to the target object;
[0045] Determine, according to the human feature information and the object feature information, a human movement trajectory corresponding to the target user and an object movement trajectory corresponding to the target object;
[0046] The interaction behavior between the target user and the target object is determined according to the human body movement trajectory and the object movement trajectory.
[0047] In a possible implementation, the interactive behavior includes a code scanning behavior; and the recognition module is specifically used to:
[0048] Determine a first user and a second user among the target users according to an intersection relationship between the human body movement trajectory and the target channel area;
[0049] If the moving track of the object passes through the target code scanning area, it is determined that the first user has a code scanning behavior with the target object;
[0050] If the distance between the moving track of the object and the moving track of the target scanning device is less than a preset distance threshold, it is determined that the first user has a scanning behavior with the target object;
[0051] If the movement track of the object moves from the first area corresponding to the target cash register device to the second area corresponding to the target cash register device, it is determined that the first user has a code scanning behavior with the target object.
[0052] In a possible implementation manner, the first determining module is specifically configured to:
[0053] Determine the order operation status corresponding to the target cash register device according to the order operation information; the order operation status includes a non-transaction status, a transaction status, and a transaction end time;
[0054] For each of the order operation states, according to the matching relationship between the interactive behavior and the order operation information, a risk identification result corresponding to the real-time video stream is determined and stored.
[0055] In a possible implementation manner, the device is further used for:
[0056] In each of the order operation states, if it is detected that the cashier cash box corresponding to the target cashier device is in an open state, it is determined that the interaction behavior does not match the order operation information;
[0057] In a non-transaction state, if the body movement trajectory of the second user and the commodity movement trajectory of the target commodity pass through the target channel area, it is determined that the interaction behavior does not match the order operation information.
[0058] In a possible implementation manner, the device is further used for:
[0059] In the transaction state, if there is transaction order information in the order operation information, and the stay time of the second user and the target product in the target channel area is less than a preset time threshold, it is determined that the interaction behavior does not match the order operation information;
[0060] In the transaction state, if the stay time of the second user and the target product in the target channel area is equal to or greater than the preset time threshold, and there is a code scanning behavior of the first user for the target product in the interaction behavior, but the order operation information does not include the code scanning information corresponding to the target product during the duration of the code scanning behavior, it is determined that the interaction behavior does not match the order operation information;
[0061] In the transaction state, if the order operation information includes modification operation information of the first user, it is determined that the interaction behavior does not match the order operation information.
[0062] In a possible implementation manner, the device is further used for:
[0063] If the order operation information includes the transaction order information in the transaction state, but the order operation information does not include the transaction order information at the transaction end time, it is determined that the interaction behavior does not match the order operation information;
[0064] If, at the end of the transaction, the quantity of goods in the order operation information does not match the target quantity of the target goods corresponding to the second user in the real-time video stream, it is determined that the interaction behavior does not match the order operation information.
[0065] In a possible implementation manner, the second determining module is specifically configured to:
[0066] Obtain risk identification results within a preset period and transaction order information corresponding to the risk identification results;
[0067] Determine risk characteristic data corresponding to each first user according to the risk identification result and the transaction order information; the risk characteristic data includes risk behavior time and risk behavior quantity;
[0068] The target risk user is determined from the first users according to the risk characteristic data and a preset risk user identification method.
[0069] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor;
[0070] The memory stores computer-executable instructions;
[0071] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the manual checkout loss prevention method described in any one of the first aspects.
[0072] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, which, when executed by a processor, are used to implement the manual checkout loss prevention method described in any one of the first aspects.
[0073] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the manual checkout loss prevention method shown in any one of the first aspects.
[0074] In an embodiment of the present application, a real-time video stream sent by a shooting device is obtained, and order operation information sent by a target cash register device corresponding to the shooting device is obtained; according to the real-time video stream, the interaction behavior between the target user and the target item in the real-time video stream is identified; according to the matching relationship between the interaction behavior and the order operation information, the risk identification result corresponding to the real-time video stream is determined and stored; if there is an abnormal risk behavior in the risk identification result, the target prompt information is output; according to multiple risk identification results within a preset period, the target risk user is determined. In the present application, the electronic device obtains the real-time video stream and the order operation information, identifies the interaction behavior between the target user and the target item according to the real-time video stream, and then determines the risk identification result corresponding to the real-time video stream according to the matching relationship between the interaction behavior and the order operation information. If there is an abnormal risk behavior, the electronic device can output the target prompt information in real time; the electronic device can determine the target risk user inside the supermarket according to multiple risk identification results within a preset period. In this way, the electronic device can accurately identify the risk behavior of manual cashier based on the real-time video stream and the order operation information, without the need for manual supervision and manual investigation, which can reduce labor costs, improve overall identification efficiency, and also improve the detection rate of risk behavior. BRIEF DESCRIPTION OF THE DRAWINGS
[0075] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0076] Figure 1 A schematic diagram of an application scenario provided for an exemplary embodiment of the present application;
[0077] Figure 2 A flowchart of a manual cashier loss prevention method provided for an exemplary embodiment of the present application;
[0078] Figure 3 A flow chart of another manual cashier loss prevention method provided for an exemplary embodiment of the present application;
[0079] Figure 4 A schematic diagram of identifying interactive behaviors in a real-time video stream provided by an exemplary embodiment of the present application;
[0080] Figure 5 A schematic diagram of identifying risky behavior provided for an exemplary embodiment of the present application;
[0081] Figure 6 A schematic diagram for identifying a target risk user provided by an exemplary embodiment of the present application;
[0082] Figure 7A schematic diagram of a technical link for manual cashier loss prevention provided for an exemplary embodiment of the present application;
[0083] Figure 8 A schematic diagram of the structure of a manual cashier loss prevention device provided for an exemplary embodiment of the present application;
[0084] Fig. 9 A schematic structural diagram of an electronic device provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION
[0085] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0086] At present, manual checkout is still an indispensable cashier method in major supermarkets. At the same time, there may be risky behaviors such as internal theft, external theft, and collusion between internal and external theft during manual checkout, which have caused serious income losses to supermarkets. Among them, external theft refers to external personnel such as consumers stealing goods sold inside supermarkets, such as deliberately leaving goods in shopping carts and hiding goods; internal theft refers to internal employees of supermarkets stealing goods sold inside supermarkets, such as deliberately canceling customer orders and keeping the payment for themselves, creating unrecorded transactions and keeping the payment for themselves, and directly stealing cash; collusion between internal and external personnel can refer to internal employees of supermarkets colluding with external personnel to steal goods sold inside supermarkets, such as cashiers deliberately omitting or pretending to scan a certain item of external personnel, deleting goods, replacing the barcodes of cheaper goods, not settling orders for all goods, and privately discounting.
[0087] Since there are many types of risk behaviors in supermarkets, some risk behaviors have obvious behavioral characteristics, while some risk behaviors are extremely hidden. In addition, some risk behaviors need to reach a certain amount before they can be determined. On this basis, the identification of risk behaviors for manual cashiers in related technologies is mainly through on-site supervision by internal supermarket personnel and manual review of video recordings. This manual cashier loss prevention method has high labor costs, low overall identification efficiency, and a low detection rate for risk behaviors.
[0088] In order to solve the above problems, the present application provides a method, device and equipment for preventing loss at manual cashier. The electronic device obtains the real-time video stream and the order operation information of the target cashier device; identifies the interaction between the target user and the target item according to the real-time video stream, and then determines the risk identification result corresponding to the real-time video stream according to the matching relationship between the interaction behavior and the order operation information; if there is an abnormal risk behavior in the risk identification result, the electronic device can output the target prompt information in real time; the electronic device can determine the target risk user inside the supermarket according to multiple risk identification results within a preset period. In this way, the electronic device can accurately identify the risk behavior of manual cashier based on the real-time video stream and the order operation information, without the need for manual supervision and manual investigation, which can reduce labor costs and improve the overall identification efficiency; it can not only provide real-time risk reminders and recover losses in a timely manner, but also improve the efficiency of risk behavior discovery in post-investigation, and also improve the risk behavior discovery rate.
[0089] Figure 1 A schematic diagram of an application scenario provided for an exemplary embodiment of the present application. Figure 1 The first user 101, the second user 102, the target cash register device 103 and the target commodity 104 are included. Figure 1 As shown, in an offline retail scenario such as a supermarket, the second user 102 (consumer) selects a target product 104 for settlement, and the first user 101 (cashier) scans the target product 104 through the target cash register device 103 to enter the code and calculate the transaction order information. Finally, the second user 102 leaves with the target product 104 after payment is completed. Figure 1 The manual checkout process shown in the figure is used to identify risky behaviors in order to reduce the revenue loss of supermarkets.
[0090] The technical solution shown in the present application is described in detail below through specific embodiments. It should be noted that the following embodiments can exist independently or in combination with each other, and the same or similar contents will not be described repeatedly in different embodiments.
[0091] Figure 2A flowchart of a manual cashier loss prevention method provided by an exemplary embodiment of the present application. Figure 2 , the manual cashier loss prevention method may include:
[0092] S201. Acquire a real-time video stream sent by a shooting device, and acquire order operation information sent by a target cash register device corresponding to the shooting device.
[0093] The execution subject of the embodiment of the present application may be an electronic device, or a manual cashier loss prevention device set in the electronic device. The manual cashier loss prevention device may be implemented by software, or by a combination of software and hardware. For ease of understanding, the following description will be made by taking the execution subject as an electronic device as an example. The electronic device may be a mobile phone, a computer, a cloud server, or a wearable device, etc., which is not limited in the embodiment of the present application.
[0094] In the embodiment of the present application, the shooting device may refer to a camera used to shoot the manual checkout screen, which may be an ordinary camera or a camera with a computing chip that can perform video analysis. The real-time video stream may refer to a video stream including the manual checkout screen shot by the shooting device. The target cashier device may refer to a hardware device such as a cashier (table) machine corresponding to the shooting device. The order operation information may refer to various order operation information generated by the target cashier device in response to the operations of the cashier and the consumer, which may specifically include order start information, order end information, order cancellation information, product scanning information, product manual input information, product deletion information, order transaction information, order discount information, order payment information, and order operator information. The order operation information may also include other information, which is not limited in the embodiment of the present application.
[0095] In the embodiment of the present application, the photographing device and the target cash register device can be in a one-to-one correspondence, that is, one photographing device can shoot the manual cash register screen corresponding to one target cash register device to form a real-time video stream. At this time, the photographing device can be deployed directly above or above the side of the target cash register device to ensure that the photographing device can capture the entire operation process of manual cash register, and there is no large area of blockage on the cashier's upper body, the cash register counter, and the customer channel. Of course, in order to reduce hardware costs, the photographing device and the target cash register device can also be in a one-to-many relationship, that is, one photographing device can shoot the real-time video streams of two or more target cash register devices, and the embodiment of the present application does not limit this.
[0096] In this step, in offline retail scenarios such as supermarkets, the electronic device can obtain the real-time video stream sent by the shooting device, which includes the real-time screen of the manual cashier of the target cashier device corresponding to the shooting device, and the electronic device can obtain the order operation information sent by the target cashier device corresponding to the shooting device. The electronic device can then identify risky behaviors in the real-time video stream based on the real-time video stream and the order operation information.
[0097] S202: Identify, according to the real-time video stream, the interaction behavior between the target user and the target object in the real-time video stream.
[0098] In the embodiment of the present application, the target user may refer to the user appearing in the real-time video stream, and may specifically include the first user and the second user, wherein the first user may refer to the internal staff of the supermarket such as the cashier, and the second user may refer to the external personnel such as the consumer and the customer. The target item may refer to various items appearing in the real-time video stream, and may specifically include the target product, the shopping cart, and the code scanning device. The interactive behavior may refer to the interactive behavior between the target user and the target item, and may specifically include the target user holding the product, the target user putting down the product, and the first user scanning the code.
[0099] In this step, after acquiring the real-time video stream, the electronic device can detect and analyze each video frame of the real-time video stream. Specifically, it can first determine the target user and the target object in the video frame based on the target detection model, and then determine the body movement trajectory of the target user and the object movement trajectory of the target object; thereafter, the electronic device can determine the interaction behavior between the target user and the target object based on the body movement trajectory and the object movement trajectory, so as to achieve accurate and comprehensive recognition of the interaction between people and objects in the real-time video stream.
[0100] S203: Determine and store the risk identification result corresponding to the real-time video stream according to the matching relationship between the interactive behavior and the order operation information.
[0101] In an embodiment of the present application, the matching relationship may refer to the corresponding relationship between the interactive behavior and the specific order information in the order operation information. The risk identification result may refer to the judgment result of the specific risk behavior corresponding to the real-time video stream. Specifically, after the electronic device identifies the interactive behavior in the real-time video stream, it can compare and match the interactive behavior with the order operation information. If the interactive behavior does not match the order operation information, the electronic device can determine that there is a risk behavior at this time. The electronic device can add the risk behavior to the risk identification result. The risk behavior may specifically include the type of risk behavior, the time of the risk behavior, the first user identification information corresponding to the risk behavior, the interactive behavior corresponding to the risk behavior, and the order operation information corresponding to the risk behavior, etc. It can also include the video clip corresponding to the risk behavior, etc., and the embodiment of the present application does not limit this. After determining the risk identification result corresponding to the real-time video stream, the electronic device can store the risk identification result corresponding to the real-time video stream, which is convenient for subsequent comprehensive analysis to determine the target risk user.
[0102] Exemplarily, if in the transaction state, there is a scanning behavior in the interactive behavior in the real-time video stream, but during the duration of the scanning behavior, there is no scanning information corresponding to the scanning behavior in the order operation information sent by the target cash register device, then the electronic device can determine that the interactive behavior does not match the order operation information, and the electronic device can determine that the first user has a fake scanning behavior, and can add the risk behavior corresponding to the process to the risk identification result of the real-time video stream. The type of risk behavior is fake scanning behavior. At the same time, the risk behavior may include the specific moment of the fake scanning behavior, the first user identification information corresponding to the fake scanning behavior, the scanning behavior corresponding to the fake scanning behavior, and the order operation information. In addition, it can also include a video clip corresponding to the fake scanning behavior, and the video clip can be a video clip captured by the electronic device from the real-time video stream, from the beginning to the end of the fake scanning behavior.
[0103] In an embodiment of the present application, the electronic device compares and matches the interactive behavior with the order operation information. According to the different matching relationships between the interactive behavior and the order operation information, the electronic device can identify the risky behavior in the real-time video stream, and then generate and store the risk identification results of the real-time video stream, thereby achieving comprehensive and accurate identification of risky behavior in the real-time video stream.
[0104] S204: If there is abnormal risk behavior in the risk identification result, output target prompt information.
[0105] In the embodiment of the present application, abnormal risk behavior may refer to risk behavior included in the risk identification result and requiring real-time warning, which may specifically include the second user's unsettled behavior (hiding goods or leaving goods in the shopping cart, etc.), the second user's unpaid behavior, the first user's abnormal scanning behavior (missed scanning or false scanning, etc.), the first user's cancellation of the transaction and the first user's false transaction behavior, etc. Of course, the abnormal risk behavior may also include other types of risk behavior, which are not limited in the embodiment of the present application. The target prompt information may refer to the alarm prompt information corresponding to the abnormal risk behavior, and the specific type of the target prompt information may be text prompt information, voice prompt information, image prompt information or video (animation) prompt information, etc.; the specific output method of the target prompt information may be by changing the display state of the indicator light (color and / or flashing display, etc.), or the electronic device may send the target prompt information to the electronic device used by the internal staff of the supermarket, or the target prompt information may be played through a display or other playback device, etc. The embodiment of the present application does not limit the specific type and specific output method of the target prompt information.
[0106] In this step, after obtaining the risk identification results corresponding to the real-time video stream, the electronic device can determine the abnormal risk behaviors in the risk identification results according to a preset abnormal risk behavior list (or name list, etc.), and then output the target prompt information in real time, so as to realize real-time reminders and real-time warnings to the internal personnel of the supermarket, which can avoid the loss of income of the supermarket to a certain extent.
[0107] S205. Determine target risk users based on multiple risk identification results within a preset period.
[0108] In the embodiment of the present application, the preset period may refer to a preset time period, which may specifically be a week, a month, or a quarter, etc. The target risk user may refer to a cashier who has committed theft in a supermarket.
[0109] Specifically, after determining the risk identification result of the real-time video stream, the electronic device can store the risk identification result of the real-time video stream. Afterwards, the electronic device can obtain multiple risk identification results within a preset period, such as obtaining risk identification results for one month, and then mine and analyze the multiple risk identification results to determine the risk characteristic data corresponding to each first user in the supermarket. The risk characteristic data may include the risk behavior type, risk behavior time, and risk behavior frequency, etc.; the electronic device can determine the target risk user among the first users according to the risk characteristic data through a preset risk user identification method. In this way, by obtaining multiple risk identification results within a preset period and performing data mining and analysis on a large number of risk identification results, the electronic device can achieve comprehensive and accurate judgment of the target risk user, improve the detection rate of risk behavior, and thus avoid revenue loss of the supermarket.
[0110] In an embodiment of the present application, an electronic device obtains a real-time video stream sent by a shooting device, and obtains order operation information sent by a target cash register device corresponding to the shooting device; identifies the interaction behavior between the target user and the target item in the real-time video stream according to the real-time video stream; determines and stores the risk identification result corresponding to the real-time video stream according to the matching relationship between the interaction behavior and the order operation information; if there is an abnormal risk behavior in the risk identification result, outputs the target prompt information; determines the target risk user according to multiple risk identification results within a preset period. In the present application, the electronic device obtains the real-time video stream and the order operation information, identifies the interaction behavior between the target user and the target item according to the real-time video stream, and then determines the risk identification result corresponding to the real-time video stream according to the matching relationship between the interaction behavior and the order operation information. If there is an abnormal risk behavior, the electronic device can output the target prompt information in real time; the electronic device can determine the target risk user inside the supermarket according to multiple risk identification results within a preset period. In this way, the electronic device can accurately identify the risk behavior of manual cash register based on the real-time video stream and the order operation information, without the need for manual supervision and manual investigation, which can reduce labor costs, improve overall identification efficiency, and also improve the discovery rate of risk behavior.
[0111] Based on the above embodiments, Figure 3 A flowchart of another manual cashier loss prevention method provided by an exemplary embodiment of the present application. Figure 3 , the manual cashier loss prevention method may include:
[0112] S301. Acquire a real-time video stream sent by a shooting device, and acquire order operation information sent by a target cash register device corresponding to the shooting device.
[0113] S302, detecting a target user and a target object in a real-time video stream, and extracting human feature information corresponding to the target user and object feature information corresponding to the target object; determining a human movement trajectory corresponding to the target user and an object movement trajectory corresponding to the target object based on the human feature information and the object feature information.
[0114] In the embodiment of the present application, human feature information may refer to the user feature information of the target user appearing in the real-time video stream. The human feature information may specifically include human features such as body shape, clothing, and accessories, as well as human head features such as face and hair. Item feature information may refer to feature information corresponding to the target item, which may specifically include the shape, color, volume, etc. of the item. Of course, human feature information and item feature information may also specifically include other types of features, which are not limited in the embodiment of the present application. Human movement trajectory may refer to the movement trajectory of the target user, which may include the movement trajectory of the target user's feet and the movement trajectory of the hands, etc. The item movement trajectory may refer to the moving path of the target item. The human movement trajectory and the item movement trajectory may include a series of trajectory point information, and each trajectory point information includes the coordinates and time of the current trajectory point.
[0115] In this step, after acquiring the real-time video stream, the electronic device can detect and identify the target user and the target object in each image frame of the real-time video stream through a target detection model based on deep learning, such as a target detection algorithm based on a convolutional neural network (You Only Look Once, YOLO) model, etc., to determine the specific location of the first user and the second user in each image frame, and at the same time determine the specific location of target items such as the target commodity, the shopping cart, and the barcode scanning device (such as a barcode scanning gun); then the electronic device can extract the human feature information of the target user and the item feature information of the target item through a feature extraction model based on deep learning, such as a residual network (Residual Network, ResNet) model, etc. It should be noted that the human feature information and the item feature information can respectively include the location information of the target user and the target item. Then the electronic device can generate the human movement trajectory of the target user according to the human feature information in each image frame; and at the same time, it can generate the item movement trajectory of the target item according to the item feature information in each image frame.
[0116] S303: Determine the interaction behavior between the target user and the target object according to the movement trajectory of the human body and the movement trajectory of the object.
[0117] In an embodiment of the present application, after determining the human body movement trajectory and the object movement trajectory in the real-time video stream, the electronic device can determine the interaction behavior between the target user and the target object based on the intersection of the human body movement trajectory and the object movement trajectory, thereby achieving comprehensive and accurate identification of the interaction behavior in the real-time video stream.
[0118] In the embodiment of the present application, the interactive behavior may specifically include a code scanning behavior, which is an important interactive behavior in the manual checkout process. The electronic device may determine the code scanning behavior in the real-time video stream based on the movement trajectory of the target object and the movement trajectory of the target user. In a possible implementation, the code scanning behavior may be specifically identified through the following steps (1) to (4):
[0119] (1) According to the intersection relationship between the human body movement trajectory and the target channel area, the first user and the second user among the target users are determined.
[0120] In the embodiment of the present application, the target channel area may refer to a preset image area in a real-time video stream, corresponding to the customer channel in the actual scene. In the manual checkout process, the code scanning behavior is mainly performed by the first user, so the electronic device first needs to determine the first user among the target users. In the actual scene, the second user usually needs to check out and leave through the customer channel, while the first user does not need to pass through the customer channel. On this basis, the electronic device can determine the target channel area corresponding to the customer channel in the image frame of the real-time video stream. If the human body movement trajectory of the target user passes through the target channel area, that is, the human body movement trajectory intersects with the target channel area, then the electronic device can determine that the target user is the second user; if the human body movement trajectory of the target user does not pass through the target channel area, that is, the human body movement trajectory does not intersect with the target channel area, then the electronic device can determine that the target user is the first user. In this way, the electronic device determines the specific type of the target user based on the intersection relationship between the human body movement trajectory and the target channel area, and can accurately identify the identity of the target user.
[0121] (2) If the moving trajectory of the object passes through the target scanning area, it is determined that the first user has a scanning behavior with the target object.
[0122] In the embodiment of the present application, the target code scanning area may refer to a fixed code scanning area corresponding to the target cash register device in the real-time video stream. The target cash register device may be a device that integrates the code scanning device and the cash register, that is, the code scanning device is fixed to the cash register and cannot be moved. At this time, the electronic device can calibrate the target code scanning area corresponding to the target cash register device in the real-time video stream. When the movement trajectory of the object passes through (or enters and exits) the target code scanning area, that is, the movement trajectory of the object enters the target code scanning area and leaves the target code scanning area, the electronic device can determine that there is a code scanning behavior between the first user and the target item.
[0123] (3) If the distance between the moving trajectory of the object and the moving trajectory of the target scanning device is less than a preset distance threshold, it is determined that the first user has a scanning behavior with the target object.
[0124] In the embodiment of the present application, the target code scanning device may refer to a code scanning device such as a code scanning gun corresponding to the target cash register device. The target code scanning device may be movable, and the first user may hold the target code scanning device to scan the target item. The preset distance threshold may refer to a preset distance critical threshold. When the distance between the moving trajectory of the target item and the moving trajectory of the target code scanning device is less than the preset distance threshold, the electronic device may determine that there is a close-range code scanning process between the target code scanning device and the target item, and the target code scanning gun and the target item overlap in position. At this time, the electronic device may determine that the first user and the target item have a code scanning behavior.
[0125] (4) If the object movement trajectory moves from the first area corresponding to the target cash register device to the second area corresponding to the target cash register device, it is determined that the first user has a scanning behavior with the target object.
[0126] In an embodiment of the present application, the first area may refer to an unsettled area corresponding to the target cash register device; the second area may refer to a settled area corresponding to the target cash register device. The target cash register device may include a cash register and a desktop device corresponding to the cash register. In the manual cash register process in an actual scenario, the second user places the target item in the first area of the target cash register device, the first user takes the target item from the first area and scans the code, and then places the target item in the second area of the target cash register device. On this basis, when the object movement trajectory of the target item moves from the first area corresponding to the target cash register device to the second area corresponding to the target cash register device, the electronic device can determine that the first user has a scanning behavior with the target item.
[0127] It should be noted that the electronic device may also use other methods to identify interactive behaviors in real-time video streams. For example, the electronic device may determine the hand movement trajectory of the first user based on a human key point detection algorithm, and determine the interactive behavior based on the hand movement trajectory, the object movement trajectory, and the movement trajectory of the target scanning device. Of course, other methods may also be used for identification, and the embodiments of the present application are not limited to this.
[0128] In an embodiment of the present application, the electronic device first determines the first user and the second user among the target users based on the human body movement trajectory, and then determines the scanning behavior in the real-time video stream based on the object movement trajectory, the human body movement trajectory, etc., thereby achieving accurate and comprehensive recognition of the scanning behavior, and can improve the accuracy of scanning behavior recognition.
[0129] For example, Figure 4 The following is a schematic diagram of identifying interactive behaviors in a real-time video stream provided by an exemplary embodiment of the present application. Figure 4As shown, after acquiring the real-time video stream, the electronic device can first detect the target user and the target object in each image frame of the real-time video stream based on the target detection algorithm; then the electronic device can extract the human feature information of the target user, and track and detect the target user to determine the human movement trajectory corresponding to the target user; and the electronic device can extract the object feature information of the target object, and track and detect the target object to determine the object movement trajectory corresponding to the target object. The electronic device can determine the first user and the second user among the target users based on the human movement trajectory corresponding to the target user, and then can determine the interactive behavior in the real-time video stream based on the human movement trajectory and the object movement trajectory.
[0130] S304. Determine the order operation status corresponding to the target cash register device according to the order operation information; the order operation status includes a non-transaction status, a transaction status, and a transaction end time.
[0131] In the embodiment of the present application, the order operation status may refer to different transaction order statuses of the target cash register device during the manual cash register process, which may specifically include a non-transaction status, a transaction status, and a transaction end time. Specifically, when the order operation information includes transaction start information, the electronic device may determine that the target cash register device enters the transaction state from the time corresponding to the transaction start information; when the order operation information includes payment success information, the electronic device may determine that the time corresponding to the payment success information is the transaction end time, and the target cash register device enters the non-transaction state from the transaction end time.
[0132] S305. For each order operation status, determine and store the risk identification result corresponding to the real-time video stream according to the matching relationship between the interactive behavior and the order operation information.
[0133] In an embodiment of the present application, after determining different order operation states corresponding to the order operation information, the electronic device can determine the risk identification result corresponding to the real-time video stream under different order operation states based on the matching relationship between the interactive behavior and the order operation information, thereby achieving accurate identification of various risk behaviors in the manual checkout process. The electronic device can store the risk identification result to facilitate subsequent mining and analysis of large amounts of data.
[0134] In a possible implementation, the matching relationship between the interactive behavior and the order operation information can be specifically identified and determined through the following steps (5) to (11):
[0135] (5) In each order operation state, if it is detected that the cash drawer corresponding to the target cash register device is in an open state, it is determined that the interaction behavior does not match the order operation information.
[0136] In the embodiment of the present application, during the manual cashiering process, the first user may need to open the cashier cash box of the target cashiering device to complete the manual cashiering process, but frequent opening of the cashier cash box indicates that the first user may have certain risky behaviors. On this basis, the electronic device can record the opening behaviors of each first user for the cashier cash box of the target cashiering device, and subsequently identify risky users according to the number of cashier cash box openings corresponding to each first user within a preset period.
[0137] Specifically, the electronic device can detect the switch state of the cashier cashbox corresponding to the target cashier device through image detection and analysis in the real-time video stream, or can determine the switch state of the cashier cashbox according to the switch signal of the cashier cashbox in the order operation information; when it is detected that the cashier cashbox is in the open state, the electronic device can determine that the interactive behavior does not match the order operation information. At this time, the electronic device can add the cashier cashbox opening behavior to the risk identification result, and record the cashier cashbox opening time and the first user identification information corresponding to the cashier cashbox opening behavior. Afterwards, the electronic device can identify and determine the risk user in combination with the number of times the cashier cashbox of each first user is opened within a preset period. Exemplarily, if the difference between the number of times the cashier cashbox of the first user is opened within the preset period and the average number of times the cashier cashbox is opened is greater than the preset number threshold, the electronic device can determine that the first user belongs to the target risk user. Of course, the electronic device can also use or combine other judgment conditions to determine the target risk user, and the embodiment of the present application is not limited to this.
[0138] (6) In a non-transaction state, if the body movement trajectory of the second user and the product movement trajectory of the target product pass through the target channel area, it is determined that the interaction behavior does not match the order operation information.
[0139] In an embodiment of the present application, when the target checkout device is in a non-transaction state, if the body movement trajectory of the second user and the product movement trajectory of the target product pass through (or enter and exit) the target channel area, that is, the second user carries the target product into the target channel area and leaves the target channel area, since the transaction has not been started in the order operation information in the non-transaction state, the electronic device can determine that the second user left directly without settling the goods. The electronic device can determine that the interactive behavior in the process does not match the order operation information, and can add the corresponding risk behavior of the customer leaving directly without settling the goods in the risk identification results.
[0140] (7) In the transaction state, if there is transaction order information in the order operation information, and the stay time of the second user and the target product in the target channel area is less than the preset time threshold, it is determined that the interaction behavior does not match the order operation information.
[0141] In an embodiment of the present application, the transaction order information may refer to specific order information in the order operation information, which may include information such as the type, quantity and amount to be paid of the target product. The preset time threshold may refer to a preset critical threshold of the residence time. When the target cash register device is in a transaction state, if there is transaction order information in the order operation information, but the residence time of the second user and the target product in the target channel area is less than the preset time threshold, the electronic device may determine that there is transaction order information in the process, but the second user and the target product do not appear in the time period from the start of the transaction state to the end of the transaction (the residence time less than the preset time threshold can be determined as not appearing), and the interactive behavior order operation information in the process does not match, and the electronic device may add risk behaviors with transaction order information but no corresponding customers in the risk identification results.
[0142] (8) In the transaction state, if the stay time of the second user and the target product in the target channel area is equal to or greater than the preset time threshold, and there is a code scanning behavior of the first user for the target product in the interaction behavior, but the order operation information does not include the code scanning information corresponding to the target product during the duration of the code scanning behavior, then it is determined that the interaction behavior does not match the order operation information.
[0143] In an embodiment of the present application, under the transaction state, if the residence time of the second user and the target product in the target channel area is not less than the preset time threshold, that is, the second user and the target product appear at this time, and there is a scanning behavior of the first user for the target product in the interactive behavior of the real-time video stream, but during the duration of the scanning behavior, the order operation information did not receive the scanning information corresponding to the target product, then the electronic device can determine that the target product information of the target product does not exist in the order operation information, the scanning behavior of the first user as an interactive behavior does not match the order operation information, and the first user has missed scanning behavior or false scanning behavior, and the electronic device can add the missed scanning behavior or false scanning behavior as a risk behavior in the risk identification result of the real-time video stream.
[0144] (9) In the transaction state, if the order operation information includes the modification operation information of the first user, it is determined that the interaction behavior does not match the order operation information.
[0145] In the embodiment of the present application, the modification operation information may refer to the manual modification operation information of the first user for the transaction order information, which may specifically include manual addition operation of the product barcode, product deletion operation, and use operation of preferential information (such as discount, etc.). In the manual checkout process of the actual scenario, the first user can usually complete the entire manual checkout process based on the code scanning behavior and simple key operation. If the order operation information obtained by the electronic device in the transaction state includes the modification operation information of the first user, the electronic device can determine that the interaction behavior does not match the order operation information, and the electronic device can record the risk behavior corresponding to the modification operation information in the risk identification result. Since the modification operation information of the first user may also exist in the normal manual revenue process, the electronic device can subsequently count the modification operation information of each first user in the risk identification result within a preset period, thereby realizing the mining and identification of target risk users.
[0146] (10) If the order operation information in the transaction state includes the transaction order information, but the order operation information at the transaction end time does not include the transaction order information, it is determined that the interaction behavior does not match the order operation information.
[0147] In an embodiment of the present application, if the target cash register device is in a transaction state, the order operation information includes the transaction order information, but the first user cancels the transaction order information at the end of the transaction, that is, the order operation information at the end of the transaction does not include the transaction order information, then the electronic device can determine that the first user's interactive behavior does not match the order operation information, and the electronic device can add the risk behavior of the first user canceling the transaction order information in the risk identification result. If the target cash register device is in a transaction state, the order operation information includes the transaction order information, and the order operation information at the end of the transaction includes the transaction order information, and the second user makes a normal payment, then the electronic device can determine that the interactive behavior matches the order operation information.
[0148] (11) If, at the end of the transaction, the quantity of goods in the order operation information does not match the target quantity of the target goods corresponding to the second user in the real-time video stream, it is determined that the interaction behavior does not match the order operation information.
[0149] In an embodiment of the present application, the quantity of goods may refer to the total number of goods in the transaction order information. The target quantity may refer to the total number of target goods of the second user in the real-time video stream. Specifically, if at the time of the transaction, the number of goods in the transaction order information of the completed payment in the target cash register device does not match the target number of target goods of the second user in the real-time video stream, the electronic device may determine that the interactive behavior does not match the order operation information, and the electronic device may add the risk behavior of the existence of unsettled goods to the risk identification result. Of course, if at the end of the transaction, the electronic device detects that the shopping cart corresponding to the second user or the second user himself carries the target goods for which the code scanning behavior has not been executed, the electronic device may also determine that the interactive behavior of the process does not match the order operation information, and the electronic device may add the risk behavior of the existence of unsettled goods to the risk identification result.
[0150] In the embodiment of the present application, the electronic device determines the matching relationship between the interactive behavior and the order operation information under different order operation states, which can accurately identify various risk behaviors in the manual checkout process, thereby improving the comprehensiveness and accuracy of risk behavior identification. It should be noted that the matching relationship between the interactive behavior and the order operation information can also be determined by other means, thereby identifying more risk behaviors, which is not limited in the embodiment of the present application. For example, Figure 5 A schematic diagram of identifying risky behaviors provided by an exemplary embodiment of the present application. Figure 5 As shown, after obtaining the interactive behavior and order operation information, the electronic device can determine the matching relationship between the interactive behavior and the order operation information under different order operation states. In the non-transaction state, if the second user carries the target product through the target channel area, the electronic device can determine that there is a risk behavior of the customer leaving directly without settlement at this time. In the transaction state, the electronic device can identify risk behaviors such as transaction order information but no customer, false scanning behavior or missed scanning behavior, manual modification operation, etc. At the end of the transaction, the electronic device can identify risk behaviors such as cancellation of the transaction and the existence of unsettled goods by the second user. In the non-transaction state, transaction state and transaction end time, the electronic device can also detect the opening behavior of the cashier cash box of the first user. If it is detected that the cashier cash box is in an open state, the electronic device can add the risk behavior of opening the cashier cash box in the risk identification result. The specific identification method of the above-mentioned risk behavior can refer to the above content, and the embodiments of the present application will not be repeated here.
[0151] S306: If there is abnormal risk behavior in the risk identification result, output target prompt information.
[0152] S307, obtaining risk identification results within a preset period and transaction order information corresponding to the risk identification results; determining risk characteristic data corresponding to each first user according to the risk identification results and the transaction order information; the risk characteristic data includes risk behavior time and risk behavior quantity.
[0153] In the embodiment of the present application, the transaction order information may refer to the order information corresponding to each risk behavior in the risk identification result. The risk characteristic data may refer to the risk behavior data of the first user, which may include the risk behavior time corresponding to each risk behavior of the first user, the number of risk behaviors of each type, and the total number of risk behaviors.
[0154] In this step, after the electronic device obtains the risk identification result of the real-time video stream, if there is an abnormal risk behavior in the risk identification result that needs to be reported in real time, the electronic device can directly output the target prompt information corresponding to the abnormal risk behavior. For other risk behaviors besides abnormal risk behaviors, the electronic device can conduct a comprehensive analysis and mining based on the risk identification results over a period of time to determine the target risk users within the supermarket. Specifically, the electronic device can obtain multiple risk identification results within a preset period, as well as the transaction order information corresponding to each risk identification result. Afterwards, the electronic device can cluster the various risk behaviors in the risk identification results according to the first user identification information, obtain the risk behavior set of each first user, and then extract the risk feature data of each first user.
[0155] S308: Determine a target risk user from the first user according to the risk characteristic data and a preset risk user identification method.
[0156] In the embodiment of the present application, the preset risk user identification method may refer to a preset risk user determination method, which may specifically include a statistical data analysis method, a machine learning analysis method, and a preset threshold screening method, etc. Specifically, the statistical data analysis method refers to the identification of target risk users based on statistical data. The electronic device may calculate statistical data such as the average value and standard deviation of each dimension in the risk characteristic data of the first user, and determine the first user whose risk characteristic data is obviously abnormal compared to the statistical data as a target risk user. For example, the first user whose total number of risk behaviors is more than 1.5 times the average value of the total number of risk behaviors may be determined as a target risk user.
[0157] The machine learning analysis method may refer to predicting target risk users based on clustering algorithms such as machine learning. The clustering algorithm may specifically include unsupervised clustering algorithms such as K-Nearest Neighbor (KNN) and isolation forest algorithms, and may also include supervised algorithms such as neural networks. The electronic device may train a preset clustering model based on sample risk data to obtain a target clustering model, and then input the risk identification results within a preset period into the target clustering model. The target clustering model may analyze the risk feature data of the first user, and then output the target risk users among the first users.
[0158] The preset threshold screening method may refer to determining a target risk user among the first users based on a preset threshold, and the preset threshold may be set based on the operating rules of the supermarket. The specific value of the preset threshold is not limited in the embodiment of the present application. Preset thresholds corresponding to various dimensions in the risk feature data may be pre-set in the electronic device. When at least one dimension in the risk feature data of the first user is greater than the preset threshold, the electronic device may determine that the first user is a target risk user.
[0159] For example, Figure 6 A schematic diagram of identifying a target risk user provided by an exemplary embodiment of the present application. Figure 6 As shown, the electronic device obtains the risk identification results within a preset period and the transaction order information of each risk identification result, and then performs statistical analysis on the risk identification results to determine the risk characteristic data of each first user; then the electronic device can determine the target risk user among the first users based on the risk characteristic data and the preset risk user identification method, and the preset risk user identification method can include statistical data analysis method, machine learning analysis method, and preset threshold screening method. In this way, the electronic device can discover risk behaviors with stronger concealment by conducting comprehensive mining and analysis on the risk identification results within a period of time, and can improve the accuracy and comprehensiveness of risk behavior identification.
[0160] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0161] Based on any one of the above embodiments, Figure 7 A schematic diagram of a manual cashier loss prevention technology link provided by an exemplary embodiment of the present application. Figure 7As shown, the electronic device obtains the real-time video stream and the order operation information corresponding to the real-time video stream; then the electronic device can analyze the real-time video stream frame by frame to determine the interactive behavior in the real-time video stream, and then can determine the risk identification result in the real-time video stream according to the matching relationship between the interactive behavior and the order operation information, so as to realize the identification of different types of risk behaviors. If there is an abnormal risk behavior in the risk identification result, the electronic device can output the target prompt information in real time for real-time warning. The electronic device can store the risk identification result, and then can mine and analyze multiple risk identification results within a preset period, and determine the target risk user from the first user. In this way, based on the real-time video stream and order operation information, the electronic device can not only give a real-time warning for abnormal risk behavior, but also screen and confirm internal employees with risky behaviors that are highly concealed, which can improve the comprehensiveness and accuracy of risk behavior identification.
[0162] The hardware devices involved in the manual cashier loss prevention method in the embodiment of the present application may include a shooting device (camera, etc.), an algorithm calculation unit, a data analysis unit, and a video storage unit, wherein the algorithm calculation unit, the data analysis unit, and the video storage unit may be integrated in the electronic device, or may be implemented by a computing power server, a data analysis server, and a video storage server, respectively. In addition, the shooting device may also be a camera with video analysis computing power, so that the hardware devices in the embodiment of the present application may include the camera with video analysis computing power and an electronic device (data analysis and storage). Of course, other hardware devices may also be used in actual scenarios, and the embodiment of the present application does not limit this.
[0163] In an embodiment of the present application, the electronic device analyzes the key targets of the real-time video stream, determines the interactive behaviors of the target users and the target items, and then generates risk identification results based on the matching relationship between the interactive behaviors and the order operation information. The categories of risk behavior identification are richer and more comprehensive, and abnormal risk behaviors occurring in real time are reported immediately. At the same time, the risk identification results are accumulated in the database, and the target risk users within the supermarket are found through comprehensive analysis of data across time periods and multiple personnel. This can not only achieve real-time risk reminders and recover losses in a timely manner, but also improve the detection rate of target risk users in subsequent data mining analysis, thereby saving labor costs and improving the accuracy and comprehensiveness of risk behavior identification.
[0164] Figure 8 For a structural diagram of a manual cashier loss prevention device provided by an exemplary embodiment of the present application, see Figure 8 , the manual cashier loss prevention device 80 comprises:
[0165] The acquisition module 81 is used to acquire the real-time video stream sent by the shooting device, and acquire the order operation information sent by the target cash register device corresponding to the shooting device;
[0166] The recognition module 82 is used to recognize the interaction behavior between the target user and the target object in the real-time video stream according to the real-time video stream;
[0167] A first determination module 83, used to determine and store a risk identification result corresponding to the real-time video stream according to a matching relationship between the interaction behavior and the order operation information;
[0168] The output module 84 is used to output target prompt information if there is abnormal risk behavior in the risk identification result;
[0169] The second determination module 85 is used to determine the target risk user according to a plurality of risk identification results within a preset period.
[0170] In a possible implementation, the identification module 82 is specifically configured to:
[0171] Detect the target user and the target object in the real-time video stream, and extract the human feature information corresponding to the target user and the object feature information corresponding to the target object;
[0172] Determine the movement trajectory of the target user and the movement trajectory of the target object according to the human feature information and the object feature information;
[0173] According to the movement trajectory of the human body and the object, the interaction behavior between the target user and the target object is determined.
[0174] In a possible implementation, the interactive behavior includes a code scanning behavior; the recognition module 82 is specifically used to:
[0175] Determine a first user and a second user among the target users according to the intersection relationship between the human body movement trajectory and the target channel area;
[0176] If the moving track of the object passes through the target code scanning area, it is determined that the first user has a code scanning behavior with the target object;
[0177] If the distance between the moving track of the object and the moving track of the target scanning device is less than a preset distance threshold, it is determined that the first user has a scanning behavior with the target object;
[0178] If the object movement track moves from the first area corresponding to the target cash register device to the second area corresponding to the target cash register device, it is determined that the first user has a code scanning behavior with the target object.
[0179] In a possible implementation manner, the first determining module 83 is specifically configured to:
[0180] According to the order operation information, determine the order operation status corresponding to the target cash register device; the order operation status includes non-transaction status, transaction status and transaction end time;
[0181] For each order operation status, the risk identification result corresponding to the real-time video stream is determined and stored according to the matching relationship between the interactive behavior and the order operation information.
[0182] In a possible implementation manner, the device 80 is further used for:
[0183] In each order operation state, if it is detected that the cashier box corresponding to the target cashier device is in an open state, it is determined that the interaction behavior does not match the order operation information;
[0184] In a non-transaction state, if the body movement trajectory of the second user and the commodity movement trajectory of the target commodity pass through the target channel area, it is determined that the interaction behavior does not match the order operation information.
[0185] In a possible implementation manner, the device 80 is further used for:
[0186] In the transaction state, if there is transaction order information in the order operation information, and the stay time of the second user and the target product in the target channel area is less than the preset time threshold, it is determined that the interaction behavior does not match the order operation information;
[0187] In the transaction state, if the stay time of the second user and the target product in the target channel area is equal to or greater than the preset time threshold, and there is a code scanning behavior of the first user for the target product in the interaction behavior, but the order operation information does not include the code scanning information corresponding to the target product during the duration of the code scanning behavior, it is determined that the interaction behavior does not match the order operation information;
[0188] In the transaction state, if the order operation information includes the modification operation information of the first user, it is determined that the interaction behavior does not match the order operation information.
[0189] In a possible implementation manner, the device 80 is further used for:
[0190] If the order operation information in the transaction state includes the transaction order information, but the order operation information at the transaction end time does not include the transaction order information, it is determined that the interaction behavior does not match the order operation information;
[0191] If, at the end of the transaction, the quantity of goods in the order operation information does not match the target quantity of the target goods corresponding to the second user in the real-time video stream, it is determined that the interaction behavior does not match the order operation information.
[0192] In a possible implementation manner, the second determining module 85 is specifically configured to:
[0193] Obtain risk identification results within a preset period and transaction order information corresponding to the risk identification results;
[0194] Determine the risk characteristic data corresponding to each first user according to the risk identification result and the transaction order information; the risk characteristic data includes the time of risk behavior and the number of risk behaviors;
[0195] According to the risk characteristic data and a preset risk user identification method, a target risk user is determined from the first users.
[0196] The manual checkout loss prevention device 80 provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar and will not be repeated here.
[0197] Fig. 9 For a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application, see Fig. 9 The electronic device 90 may include a processor 91 and a memory 92. Exemplarily, the processor 91 and the memory 92 are interconnected via a bus 93.
[0198] Memory 92 stores computer executable instructions;
[0199] The processor 91 executes the computer execution instructions stored in the memory 92, so that the processor 91 executes the manual checkout loss prevention method as shown in the above method embodiment.
[0200] Accordingly, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the manual checkout loss prevention method of the above method embodiment.
[0201] Accordingly, an embodiment of the present application may also provide a computer program product, including a computer program, which, when executed by a processor, can implement the manual checkout loss prevention method shown in the above method embodiment.
[0202] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0203] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0204] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0205] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0206] In a typical configuration, a computing device includes one or more processors, input / output interfaces, network interfaces, and memory.
[0207] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
[0208] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0209] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0210] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A manual cashier loss prevention method, characterized in that: include: Acquire the real-time video stream sent by the shooting device, and acquire the order operation information sent by the target cash register device corresponding to the shooting device; According to the real-time video stream, identifying the interaction behavior between the target user and the target object in the real-time video stream; Determine and store a risk identification result corresponding to the real-time video stream according to a matching relationship between the interaction behavior and the order operation information; If there is abnormal risk behavior in the risk identification result, output target prompt information; Determine target risk users based on multiple risk identification results within a preset period.
2. The method according to claim 1, characterized in that The step of identifying, according to the real-time video stream, an interaction behavior between a target user and a target object in the real-time video stream includes: Detecting a target user and a target object in the real-time video stream, and extracting human feature information corresponding to the target user and object feature information corresponding to the target object; Determine, according to the human feature information and the object feature information, a human movement trajectory corresponding to the target user and an object movement trajectory corresponding to the target object; The interaction behavior between the target user and the target object is determined according to the human body movement trajectory and the object movement trajectory.
3. The method according to claim 2, characterized in that The interactive behavior includes a code scanning behavior; and determining the interactive behavior between the target user and the target object according to the human body movement trajectory and the object movement trajectory includes: Determine a first user and a second user among the target users according to an intersection relationship between the human body movement trajectory and the target channel area; If the moving track of the object passes through the target code scanning area, it is determined that the first user has a code scanning behavior with the target object; If the distance between the moving track of the object and the moving track of the target scanning device is less than a preset distance threshold, it is determined that the first user has a scanning behavior with the target object; If the movement track of the object moves from the first area corresponding to the target cash register device to the second area corresponding to the target cash register device, it is determined that the first user has a code scanning behavior with the target object.
4. The method according to claim 1, characterized in that: The determining and storing the risk identification result corresponding to the real-time video stream according to the matching relationship between the interactive behavior and the order operation information includes: Determine the order operation status corresponding to the target cash register device according to the order operation information; the order operation status includes a non-transaction status, a transaction status, and a transaction end time; For each of the order operation states, according to the matching relationship between the interactive behavior and the order operation information, a risk identification result corresponding to the real-time video stream is determined and stored.
5. The method according to claim 4, characterized in that The method further comprises: In each of the order operation states, if it is detected that the cashier cash box corresponding to the target cashier device is in an open state, it is determined that the interaction behavior does not match the order operation information; In a non-transaction state, if the body movement trajectory of the second user and the commodity movement trajectory of the target commodity pass through the target channel area, it is determined that the interaction behavior does not match the order operation information.
6. The method according to claim 4, characterized in that The method further comprises: In the transaction state, if there is transaction order information in the order operation information, and the stay time of the second user and the target product in the target channel area is less than a preset time threshold, it is determined that the interaction behavior does not match the order operation information; In the transaction state, if the stay time of the second user and the target product in the target channel area is equal to or greater than the preset time threshold, and there is a code scanning behavior of the first user for the target product in the interaction behavior, but the order operation information does not include the code scanning information corresponding to the target product during the duration of the code scanning behavior, it is determined that the interaction behavior does not match the order operation information; In the transaction state, if the order operation information includes modification operation information of the first user, it is determined that the interaction behavior does not match the order operation information.
7. The method according to claim 4, characterized in that The method further comprises: If the order operation information includes the transaction order information in the transaction state, but the order operation information does not include the transaction order information at the transaction end time, it is determined that the interaction behavior does not match the order operation information; If, at the end of the transaction, the quantity of goods in the order operation information does not match the target quantity of the target goods corresponding to the second user in the real-time video stream, it is determined that the interaction behavior does not match the order operation information.
8. The method according to any one of claims 1 to 7, characterized in that: The step of determining target risk users according to multiple risk identification results within a preset period includes: Obtain risk identification results within a preset period and transaction order information corresponding to the risk identification results; Determine risk characteristic data corresponding to each first user according to the risk identification result and the transaction order information; the risk characteristic data includes risk behavior time and risk behavior quantity; The target risk user is determined from the first users according to the risk characteristic data and a preset risk user identification method.
9. A manual cashier loss prevention device, characterized in that: include: An acquisition module, used to acquire a real-time video stream sent by a shooting device, and to acquire order operation information sent by a target cash register device corresponding to the shooting device; An identification module, used to identify the interaction behavior between the target user and the target object in the real-time video stream according to the real-time video stream; A first determination module, configured to determine and store a risk identification result corresponding to the real-time video stream according to a matching relationship between the interaction behavior and the order operation information; An output module, used for outputting target prompt information if there is abnormal risk behavior in the risk identification result; The second determination module is used to determine the target risk user according to a plurality of risk identification results within a preset period.
10. An electronic device, characterized in that: include: Memory and processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the manual checkout loss prevention method according to any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the manual checkout loss prevention method according to any one of claims 1 to 8.
12. A computer program product, characterized in that It includes a computer program, which, when executed by a computer, implements the manual checkout loss prevention method as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Cashier loss prevention method and system
CN109961457A
Video analysis method, device and system, storage medium and computer equipment
CN110309801A
Commodity missing code scanning behavior identification method, device and system
CN112115745A
Cashier loss prevention method and system
CN113537034A
Information processing method and device, server and storage medium
CN117172784A
Cited By
Abnormal transaction behavior identification method and device
CN121708533A
