Method of monitoring anomalies in self-shopping and self-shopping anomaly monitoring device

Abstract

The present disclosure relates to a method of monitoring an anomaly in self-shopping and a self-shopping anomaly monitoring device. The method of monitoring an anomaly in self-shopping includes: detecting a monitoring video by using a detection model to output a detection result of each frame in the monitoring video including a detected action of a customer and a location of a product; determining, for each frame in the monitoring video, a product box representing the location of the product when the customer interacts with each product based on the detection result; generating a trajectory set including trajectories of the products based on the product boxes; determining whether each trajectory in the trajectory set is a real trajectory based on at least one of a length and a duration of the trajectory and a direction of the trajectory; determining a number of product scans based on the real trajectories and the detected action of the customer; determining a number of products based on the real trajectories; and determining whether an anomaly exists based on the number of product scans and the number of products.

Description

Technical Field

[0001] This disclosure relates to monitoring anomalies in the self-service shopping process, and more specifically to methods and devices for monitoring anomalies in self-service shopping. Background Technology

[0002] With the development of technology, self-service shopping in supermarkets is becoming increasingly accepted. While self-checkout saves on labor costs, it can also lead to irregularities, such as unintentional missed checks or intentional theft.

[0003] To prevent this from happening, existing technologies propose a detection model based on a DNN network and a counting method using tracking techniques to determine the presence of anomalies. The detection model detects whether customers and products have interacted and outputs the bounding boxes of the interacting customers and products, along with the interaction actions. The counting method uses tracking techniques to count the number of products and the number of scans based on the output of the detection model.

[0004] Counting methods are typically based on tracking technology. Tracking technology includes traditional tracking algorithms and deep learning-based tracking algorithms. Deep learning-based tracking algorithms require extracting product feature information and involve significant computation, making them difficult to implement in real-time. While traditional tracking algorithms are faster and require less complex computation, they lack product status information (e.g., whether the product interacts with the customer, and the nature of that interaction) and require predefined information, such as the start and end checkout areas.

[0005] Therefore, a real-time, fast, and accurate technology is needed to monitor anomalies in self-service shopping. Summary of the Invention

[0006] A brief overview of this disclosure is provided below to provide a basic understanding of certain aspects of it. It should be understood that this overview is not an exhaustive summary of the disclosure. It is not intended to identify key or essential parts of the disclosure, nor is it intended to limit its scope. Its purpose is merely to present certain concepts in a simplified form as a prelude to the more detailed description that follows.

[0007] According to one aspect of the present invention, a method for monitoring anomalies during a self-service shopping process is provided, comprising: detecting a monitoring video using a detection model and outputting detection results for each frame of the monitoring video, including detected customer actions and product positions; for each frame of the monitoring video, determining, based on the detection results, a product frame representing the position of each product when the customer interacts with it; generating a trajectory set including the trajectories of the products based on the product frames; determining whether a trajectory is a real trajectory based on at least one of the length and duration of each trajectory in the trajectory set and the direction of the trajectory; determining the number of product scans based on the real trajectories and the detected customer actions; determining the number of products based on the real trajectories; and determining whether an anomaly exists based on the number of product scans and the number of products.

[0008] According to one aspect of the present invention, a self-service shopping anomaly detection device is provided, comprising: a detection unit configured to detect a monitoring video using a detection model and output detection results for each frame of the monitoring video, including the detected actions of customers and the positions of goods; an extraction unit configured to determine, based on the detection results, a product frame representing the position of a product when a customer interacts with each product in each frame of the monitoring video; a trajectory generation unit configured to generate a trajectory set including the trajectories of the products based on the product frames; a true trajectory determination unit configured to determine whether a trajectory is a true trajectory based on at least one of the length and duration of each trajectory in the trajectory set and the direction of the trajectory; a calculation unit configured to determine the number of product scans based on the true trajectories and the detected actions of customers, and to determine the number of products based on the true trajectories; and an anomaly determination unit configured to determine whether an anomaly exists based on the number of product scans and the number of products.

[0009] According to one aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon. When executed by a computer, the computer program implements a method for monitoring anomalies during a self-service shopping process. The method includes: detecting a monitoring video using a detection model and outputting detection results for each frame of the monitoring video, including detected customer actions and product positions; for each frame of the monitoring video, determining, based on the detection results, a product frame representing the position of each product when the customer interacts with it; generating a trajectory set including the trajectories of the products based on the product frames; determining whether a trajectory is a true trajectory based on at least one of the length and duration of each trajectory in the trajectory set and the direction of the trajectory; determining the number of product scans based on the true trajectories and the detected customer actions; determining the number of products based on the true trajectories; and determining whether an anomaly exists based on the number of product scans and the number of products.

[0010] These and other advantages of the invention will become more apparent from the following detailed description of preferred embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0011] To further illustrate the above and other advantages and features of this disclosure, the specific embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. These drawings, together with the following detailed description, are included in and form a part of this specification. Elements having the same function and structure are indicated by the same reference numerals. It should be understood that these drawings only depict typical examples of this disclosure and should not be considered as limiting the scope of this disclosure. In the drawings:

[0012] Figure 1 A flowchart illustrating a method for monitoring anomalies during a self-service shopping process according to an embodiment of the present invention;

[0013] Figure 2 schematically shown Figure 1 The process of generating a trajectory based on the detection results of the detection model in steps S103-S104;

[0014] Figures 3A-3C To illustrate Figure 1 In step S105, how to obtain a diagram of the movement direction of the goods;

[0015] Figure 4 A schematic diagram of the structure of a self-service shopping anomaly monitoring device according to an embodiment of the present invention is shown; and

[0016] Figure 5 This is a block diagram of an exemplary structure of a general-purpose personal computer in which methods and / or networks according to embodiments of the present invention can be implemented. Detailed Implementation

[0017] This disclosure will now be described more fully with reference to the accompanying drawings. The following description is merely exemplary and is not intended to limit the disclosure, its application, or its uses.

[0018] Example embodiments are provided so that this disclosure will become exhaustive and will fully convey its scope to those skilled in the art. Numerous specific details, such as examples of particular components, apparatus, and methods, are set forth to provide a detailed understanding of embodiments of this disclosure. It will be apparent to those skilled in the art that the specific details are not required, and that the example embodiments may be implemented in many different forms, none of which should be construed as limiting the scope of this disclosure. In some example embodiments, well-known processes, well-known structures, and well-known techniques are not described in detail.

[0019] It should also be noted that, in order to avoid obscuring this disclosure with unnecessary details, only the equipment structure and / or processing steps closely related to the solution according to this disclosure are shown in the accompanying drawings, while other details that are not closely related to this disclosure are omitted.

[0020] Figure 1 A flowchart illustrating a method for monitoring anomalies during a self-service shopping process according to an embodiment of the present invention is shown.

[0021] In step S101, method 100 begins.

[0022] In step S102, the detection model is used to detect the input monitoring video and output the detection results of each frame in the video. The detection results include the actions of the customer interacting with the product, as well as the product frame and customer frame when the customer interacts with the product. The product frame reflects the position of the product, and the customer frame reflects the position of the customer. It should be understood that the detected actions of the customer interacting with the product can include actions such as picking up and scanning.

[0023] In step S103, based on the detection results of step S102, for each frame in the monitoring video, the product frame is extracted when the customer interacts with the product by picking it up or scanning it.

[0024] In step S104, a trajectory set is generated based on the extracted product frames. The trajectory set is a collection of product trajectories. Specifically, all product frames in the monitoring video are sorted in chronological order, grouped based on whether the product frames of two adjacent frames in the monitoring video overlap, and a trajectory is generated by connecting the centers of the product frames in each group.

[0025] In step S105, it is determined whether a trajectory in the trajectory set is a real trajectory. Specifically, based on the trajectory set, the movement direction of the goods is obtained using clustering methods, etc. Then, it is determined whether the trajectory is a real trajectory based on whether the trajectory in the trajectory set satisfies at least one of a length condition and a duration condition, and whether the direction of the trajectory is consistent with the movement direction of the goods. The length condition is that the length of the trajectory is greater than or equal to a length threshold, and the duration condition is that the duration of the trajectory is greater than or equal to a time threshold. When a trajectory satisfies at least one of the length condition and the duration condition, and the direction of the trajectory is consistent with the movement direction of the goods, the trajectory is determined to be a real trajectory.

[0026] In step S106, the scan count is calculated, where the initial value of the scan count is zero. For each real trajectory, based on the detection results of the detection model, it is determined whether there is a scan action of customer interaction with product within the duration of the trajectory. If there is a scan action of customer interaction with product within the duration of the trajectory, the scan count is incremented by one, thus obtaining the final scan count.

[0027] In step S107, the number of items is calculated, where the number of items is equal to the number of actual trajectories. Although step S106 is executed before step S107 in the above description, it should be understood that the present invention is not limited to the above embodiments. According to other embodiments of the present invention, step S106 may be executed after step S107 or step S106 and step S107 may be executed simultaneously.

[0028] In step S108, it is determined whether the scan count is the same as the number of items. If the scan count is different from the number of items (i.e., inconsistent), an anomaly is determined to exist in the self-service shopping process, and the method proceeds to step S109, where the method ends. According to an embodiment of the present invention, when the scan count is less than the number of items, theft is determined to have occurred. If the scan count is the same as the number of items, the self-service shopping process is determined to be normal. The method then proceeds to step S110, where the method ends.

[0029] Figure 2 schematically shown Figure 1 The process of generating a trajectory based on the detection results of the detection model in steps S103-S104.

[0030] like Figure 2 As shown, for the detection results of the input detection model, the product boxes when the customer interacts with the product are extracted from each frame of the video, thus obtaining a sequence of product boxes.

[0031] Then, all product frames are sorted in chronological order and grouped based on whether product frames in adjacent frames of the monitoring video overlap.

[0032] Finally, a trajectory is generated by connecting the centers of the product boxes in each group.

[0033] Figures 3A-3C To illustrate Figure 1 How to obtain the diagram of the movement direction of the goods in step S105?

[0034] like Figure 3A As shown, circular icons represent the starting point of each trajectory in the trajectory set, and triangle icons represent the ending point of each trajectory in the trajectory set.

[0035] like Figure 3B As shown, clustering methods are used to classify the start and end points of each trajectory in the trajectory set. For example, in... Figure 3B In the diagram, circles are used to indicate that the starting point or ending point within the circle is grouped together. It can be seen that the starting point is divided into three starting point groups, and the ending point is divided into two ending point groups.

[0036] like Figure 3CAs shown, the average starting point is obtained by using the group with the most starting points. It can be assumed that the group with the most starting points has a higher density of starting points. For example, in... Figure 3C In the diagram, the circle on the right contains two starting points. This group represents the starting point group with the most starting points among the three starting point groups, and therefore, this starting point group is used to obtain the starting point for the mean. Furthermore, in... Figure 3C In the diagram, the circle on the left contains three endpoints. This endpoint group has the most endpoints among the two-endpoint groups, and is therefore used to obtain the mean endpoint. The direction of product movement is determined by connecting the mean starting point and the mean ending point. Figure 3C In the diagram, the direction of movement of the obtained goods is indicated by line segments with arrows.

[0037] According to an embodiment of the present invention, if a trajectory in the trajectory set is determined not to be a true trajectory, then the trajectory is regarded as a candidate trajectory. Usually, these candidate trajectories are short and have random directions. These candidate trajectories in the trajectory set can be merged into a new trajectory in the trajectory set according to certain conditions.

[0038] According to an embodiment of the present invention, when the length of the candidate trajectory is less than the length threshold and the direction of the candidate trajectory is consistent with the direction of movement of the goods, the candidate trajectory is merged with other candidate trajectories that are closest in time and whose direction is consistent with the direction of movement of the goods. In addition, according to another embodiment of the present invention, the candidate trajectory can also be merged with other candidate trajectories that are closest in time and whose direction is in the approximate extension direction of the candidate trajectory and consistent with the direction of movement of the goods.

[0039] It should be understood that although the above discussion pertains to the situation where a customer takes a single item at checkout, the present invention can also be applied to situations where a customer is holding different items in both hands. The following description focuses only on the differences between handling different items in the case of a customer holding a single item and handling different items in both hands. The remaining procedures are similar to those for the case of a customer holding a single item and will not be repeated here.

[0040] When holding different items in both hands, the detection model extracts the product bounding boxes when the customer picks up or scans the items. Based on the extracted product bounding boxes, a trajectory set including the product trajectory is generated. Since both hands interact with different items simultaneously, two trajectories are generated at the same time. The product bounding box in the next frame is determined to be on which of the existing trajectories the product bounding box belongs to based on the degree of overlap and distance between the product bounding box in the next frame and the product bounding box in the previous frame.

[0041] Figure 4 A schematic diagram of the structure of a self-service shopping anomaly monitoring device 400 according to an embodiment of the present invention is shown, as follows: Figure 4As shown, the self-service shopping anomaly monitoring device 400 may include a detection unit 410, an extraction unit 420, a trajectory generation unit 430, a real trajectory determination unit 440, a calculation unit 450, and an anomaly determination unit 460.

[0042] The detection unit 410, extraction unit 420, trajectory generation unit 430, and true trajectory determination unit 440 are respectively configured to achieve the above-mentioned functions. Figure 1 In steps S102, S103, S104, and S105, the computing unit 450 is configured to perform the above-mentioned processing. Figure 1 In steps S106-S107, the anomaly determination unit 460 is configured to perform the above-mentioned processing. Figure 1 The processing in step S108 is described in detail below. Figure 1 The details are given in the description, so I will not repeat them here.

[0043] Each component and unit in the aforementioned self-service shopping anomaly monitoring device can be configured via software, firmware, hardware, or a combination thereof. Specific means or methods of configuration are well known to those skilled in the art and will not be elaborated upon here. In the case of software or firmware implementation, data is transmitted from a storage medium or network to a computer with a dedicated hardware architecture (e.g., ...). Figure 5 The general-purpose computer 500 shown is equipped with the programs that constitute the software, and when various programs are installed, the computer is able to perform various functions, etc.

[0044] Figure 5 This is a block diagram of an exemplary structure of a general-purpose personal computer in which methods and / or networks according to embodiments of the present invention can be implemented. For example... Figure 5 As shown, the Central Processing Unit (CPU) 501 performs various processes based on programs stored in the Read-Only Memory (ROM) 502 or programs loaded into the Random Access Memory (RAM) 503 from the storage section 508. The RAM 503 also stores data required as needed when the CPU 501 performs various processes, etc. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0045] The following components are connected to the input / output interface 505: input section 506 (including keyboard, mouse, etc.), output section 507 (including display, such as cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.), storage section 508 (including hard disk, etc.), and communication section 509 (including network interface card, such as LAN card, modem, etc.). The communication section 509 performs communication processing via a network, such as the Internet. If necessary, a drive 510 may also be connected to the input / output interface 505. Removable media 511, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive 510 as needed, so that computer programs read from them can be installed into the storage section 508 as needed.

[0046] When the above series of processes are implemented through software, the program constituting the software is installed from a network such as the Internet or a storage medium such as removable media 511.

[0047] Those skilled in the art will understand that such storage media are not limited to Figure 5 The illustration shows a removable medium 511 containing a program, distributed separately from the device to provide the program to the user. Examples of removable media 511 include disks (including floppy disks (registered trademark)), optical disks (including optical disc read-only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including mini-discs (MD) (registered trademark)), and semiconductor memory. Alternatively, the storage medium may be ROM 502, a hard disk included in storage section 508, etc., containing programs and distributed to the user along with the device containing them.

[0048] The present invention also provides corresponding computer program code and a computer program product storing machine-readable instruction code. When the instruction code is read and executed by a machine, the method described above according to the embodiments of the present invention can be performed.

[0049] Accordingly, storage media configured to carry the aforementioned program product storing machine-readable instruction code are also included in the disclosure of this invention. These storage media include, but are not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, etc.

[0050] Based on the above description, the embodiments of this disclosure provide the following technical solutions, but are not limited thereto.

[0051] Appendix 1. A method for monitoring anomalies during self-service shopping, comprising:

[0052] The detection model is used to detect the monitoring video, and the detection results of each frame in the monitoring video, including the detected customer actions and the position of the goods, are output.

[0053] For each frame in the monitoring video, a product frame representing the position of the product when the customer interacts with each product is determined based on the detection results;

[0054] Generate a trajectory set including the trajectory of the product based on the product frame;

[0055] Whether a trajectory is a real trajectory is determined based on at least one of the length and duration of each trajectory in the trajectory set, as well as the direction of the trajectory.

[0056] The number of items scanned is determined based on the actual trajectory and the detected customer movements;

[0057] The number of goods is determined based on the actual trajectory; and

[0058] The presence of anomalies is determined based on the number of scanned items and the total number of items.

[0059] Appendix 2. According to the method described in Appendix 1, generating the trajectory set based on the product frame includes:

[0060] All product frames in the monitored video are sorted in chronological order;

[0061] Grouping is based on whether the product frames of two adjacent frames in the monitoring video overlap;

[0062] The trajectory is generated by connecting the centers of the product frames in each group.

[0063] Appendix 3. According to the method described in Appendix 1, determining whether a trajectory is a true trajectory based on at least one of the length and duration of each trajectory in the trajectory set and the direction of the trajectory includes:

[0064] Clustering methods are used to classify the start and end points of each trajectory in the trajectory set;

[0065] The mean starting point and mean ending point are obtained based on the density of the starting or ending points in the classification.

[0066] The direction of movement of the goods is obtained based on the mean starting point and the mean ending point;

[0067] For each trajectory in the trajectory set, it is determined whether the trajectory is a real trajectory based on whether it meets at least one of the length condition and the duration condition, and whether the direction of the trajectory is consistent with the direction of movement of the goods.

[0068] Note 4. According to the method described in Note 3, the length condition is that the length of the trajectory is greater than or equal to a length threshold.

[0069] Note 5. According to the method described in Note 4, the duration condition is that the duration of the trajectory is greater than or equal to a time threshold.

[0070] Note 6. According to the method described in Note 5, wherein when the length of the trajectory is less than the length threshold and the direction of the trajectory is consistent with the direction of movement of the product, the trajectory is merged with the trajectory that is closest in time and whose direction is consistent with the direction of movement of the product.

[0071] Note 7. According to the method described in Note 1, an anomaly is determined when the number of scanned items is inconsistent with the number of items, and theft is determined when the number of scanned items is less than the number of items.

[0072] Note 8. The method according to Note 1, wherein the customer's actions include at least the customer taking the goods and scanning them.

[0073] Appendix 9. According to the method described in Appendix 2, when there is more than one product frame in each frame of the monitoring video, the product frames in the later frame are grouped based on the degree of overlap and distance between the product frames in the previous frame and the product frames in the later frame.

[0074] Note 10. The method according to Note 1, wherein the detection model is based on a deep neural network.

[0075] Appendix 11. A self-service shopping anomaly monitoring device, the self-service shopping anomaly monitoring device comprising:

[0076] The detection unit is configured to use a detection model to detect the monitoring video and output the detection results for each frame of the monitoring video, including the detected customer actions and the position of the goods.

[0077] The extraction unit is configured to determine, for each frame in the monitoring video, a product box representing the position of the product when the customer interacts with each product, based on the detection results.

[0078] The trajectory generation unit is configured to generate a trajectory set including the trajectory of the product based on the product frame;

[0079] The real trajectory determination unit is configured to determine whether a trajectory is a real trajectory based on at least one of the length and duration of each trajectory in the trajectory set and the direction of the trajectory.

[0080] The calculation unit is configured to determine the number of product scans based on the actual trajectory and the detected customer movements, and to determine the total number of products based on the actual trajectory; and

[0081] The anomaly determination unit is configured to determine whether an anomaly exists based on the number of scanned products and the number of products.

[0082] Appendix 12. According to the shopping assistance anomaly monitoring device described in Appendix 11, the trajectory generation unit is configured to generate a trajectory set based on the product frame, including:

[0083] All product frames in the monitored video are sorted in chronological order;

[0084] Grouping is based on whether the product frames of two adjacent frames in the monitoring video overlap;

[0085] The trajectory is generated by connecting the centers of the product frames in each group.

[0086] Appendix 13. According to the shopping assistance anomaly monitoring device described in Appendix 11, the real trajectory determination unit is configured to determine whether a trajectory is a real trajectory based on at least one of the length and duration of each trajectory in the trajectory set and the direction of the trajectory, including:

[0087] Clustering methods are used to classify the start and end points of each trajectory in the trajectory set;

[0088] The mean starting point and mean ending point are obtained based on the density of the starting or ending points in the classification.

[0089] The direction of movement of the goods is obtained based on the mean starting point and the mean ending point;

[0090] For each trajectory in the trajectory set, it is determined whether the trajectory is a real trajectory based on whether it meets at least one of the length condition and the duration condition, and whether the direction of the trajectory is consistent with the direction of movement of the goods.

[0091] Note 14. In the self-service shopping anomaly monitoring device according to Note 13, the length condition is that the length of the trajectory is greater than or equal to a length threshold.

[0092] Note 15. In the self-service shopping anomaly monitoring device according to Note 14, the duration condition is that the duration of the trajectory is greater than or equal to a time threshold.

[0093] Note 16. According to the self-service shopping anomaly monitoring device described in Note 15, when the length of the trajectory is less than the length threshold and the direction of the trajectory is consistent with the direction of movement of the goods, the trajectory is merged with the trajectory that is closest in time and whose direction is consistent with the direction of movement of the goods.

[0094] Note 17. According to the self-service shopping anomaly monitoring device described in Note 11, the anomaly determination unit is further configured to: determine that an anomaly exists when the number of scanned goods is inconsistent with the number of goods, wherein the theft is determined when the number of scanned goods is less than the number of goods.

[0095] Note 18. The self-service shopping anomaly monitoring device according to Note 11, wherein the customer's actions include at least the customer taking and scanning goods.

[0096] Note 19. According to the self-service shopping anomaly monitoring device described in Note 12, when there is more than one product frame in each frame of the monitoring video, the product frames in the later frame are grouped based on the degree of overlap and distance between the product frames in the previous frame and the product frames in the later frame.

[0097] Appendix 20. A computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a computer, enabling a method for monitoring anomalies during a self-service shopping process, the method comprising:

[0098] The detection model is used to detect the monitoring video, and the detection results of each frame in the monitoring video, including the detected customer actions and the position of the goods, are output.

[0099] For each frame in the monitoring video, a product frame representing the position of the product when the customer interacts with each product is determined based on the detection results;

[0100] Generate a trajectory set including the trajectory of the product based on the product frame;

[0101] Whether a trajectory is a real trajectory is determined based on at least one of the length and duration of each trajectory in the trajectory set, as well as the direction of the trajectory.

[0102] The number of items scanned is determined based on the actual trajectory and the detected customer movements;

[0103] The number of goods is determined based on the actual trajectory; and

[0104] The presence of anomalies is determined based on the number of scanned items and the total number of items.

[0105] Finally, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Furthermore, unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0106] While embodiments of the present invention have been described in detail above with reference to the accompanying drawings, it should be understood that the embodiments described above are merely illustrative of the invention and do not constitute a limitation thereof. Those skilled in the art can make various modifications and alterations to the above embodiments without departing from the spirit and scope of the invention. Therefore, the scope of the invention is defined only by the appended claims and their equivalents.

Claims

1.A method for monitoring anomalies in a self-shopping process, comprising: detecting a monitoring video using a detection model, and outputting detection results of each frame in the monitoring video including detected actions of a customer and locations of goods; for each frame in the monitoring video, determining, based on the detection results, a goods box representing a location of a good when the customer interacts with each good; generating, based on the goods boxes, a trajectory set including trajectories of goods; judging whether each trajectory in the trajectory set is a real trajectory based on at least one of a length and a duration of the trajectory and a direction of the trajectory; determining a number of good scans based on real trajectories and the customer’s scans of goods in the detected actions of the customer; determining a number of goods based on the real trajectories; and judging whether there is an anomaly based on the number of good scans and the number of goods. 2.The method of claim 1, wherein generating a trajectory set based on the goods boxes comprises: sorting all the goods boxes in the monitoring video in a time sequence; grouping based on whether the goods boxes of adjacent two frames in the monitoring video overlap; generating the trajectories by connecting centers of the goods boxes in each group. 3.The method of claim 1, wherein judging whether each trajectory in the trajectory set is a real trajectory based on at least one of a length and a duration of the trajectory and a direction of the trajectory comprises: classifying start points and end points of each trajectory in the trajectory set using a clustering method; obtaining a mean start point and a mean end point based on a density of the start points or the end points in the classification; obtaining a moving direction of the goods based on the mean start point and the mean end point; judging whether each trajectory in the trajectory set is a real trajectory based on whether at least one of a length condition, a duration condition and the direction of the trajectory being consistent with the moving direction of the goods is satisfied. 4.The method of claim 3, wherein the length condition is that the length of the trajectory is greater than or equal to a length threshold. 5.The method of claim 4, wherein the duration condition is that the duration of the trajectory is greater than or equal to a time threshold. 6.The method of claim 5, wherein when the length of the trajectory is less than the length threshold and the direction of the trajectory is consistent with the moving direction of the goods, the trajectory is merged with a trajectory that is closest in time and has a direction consistent with the moving direction of the goods. 7.The method of claim 1, wherein an anomaly is judged to exist when the number of good scans is inconsistent with the number of goods, and wherein a theft is judged to exist when the number of good scans is less than the number of goods. 8.The method of claim 1, wherein the actions of the customer further include the customer’s taking of goods. 9.A self-shopping anomaly monitoring apparatus, comprising: a detection unit configured to detect a monitoring video using a detection model and output detection results of each frame in the monitoring video including detected actions of a customer and locations of goods; ​ an extraction unit configured to determine, for each frame in the monitoring video, a product box representing a position of a product when the customer interacts with each product based on the detection result; a trajectory generation unit configured to generate a trajectory set including trajectories of products based on the product boxes; a real trajectory determination unit configured to determine whether each trajectory in the trajectory set is a real trajectory based on at least one of a length and a duration of the trajectory and a direction of the trajectory; a calculation unit configured to determine a number of product scans based on real trajectories and the customer's scans on products in the detected customer's actions, and determine a number of products based on the real trajectories; and an anomaly determination unit configured to determine whether there is an anomaly based on the number of product scans and the number of products. 10.A computer readable storage medium having stored thereon a computer program, the computer program being capable of implementing a method for monitoring an anomaly in a self-service shopping process when executed by a computer, the method comprising: detecting a monitoring video using a detection model to output a detection result of each frame in the monitoring video including detected customer's actions and positions of products; determining, for each frame in the monitoring video, a product box representing a position of a product when the customer interacts with each product based on the detection result; generating a trajectory set including trajectories of products based on the product boxes; determining whether each trajectory in the trajectory set is a real trajectory based on at least one of a length and a duration of the trajectory and a direction of the trajectory; determining a number of product scans based on real trajectories and the customer's scans on products in the detected customer's actions; determining a number of products based on the real trajectories; and determining whether there is an anomaly based on the number of product scans and the number of products.

Images