Methods, devices and systems for identifying abnormal order issuance

Abstract

The present application discloses a method, device and system for identifying abnormal order issuance, including: determining the transaction time based on the transaction event; obtaining multiple frames of target images within a time period associated with the transaction time, the associated time period being a preset time period before the transaction time and / or a preset time period after the transaction time, and the target image is obtained by an image acquisition device photographing a target environment area; determining a first detection image area in the target image, identifying whether there is a customer target in the first detection image area, the first detection image area being an area within a preset range around the cash register device; determining the number of image frames where there is no customer target, and if the image frame number is greater than a preset first frame number threshold, outputting an alarm instruction; or determining the number of image frames where there is a customer target, and if the image frame number is less than a preset second frame number threshold, outputting an alarm instruction. The present application can automatically and effectively identify the illegal behavior of having transaction orders but no customers in a store, thereby improving the operation and management of the store.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device and system for identifying abnormal order issuance. Background Art

[0002] In many chain supermarkets, there are problems with stores placing orders abnormally, including but not limited to the following behaviors: store clerks personally purchase goods in the store and use discounts to earn profit margins. However, currently, stores circumvent abnormal orders through corporate management methods, which increases costs, management difficulties, and low efficiency. Summary of the invention

[0003] The purpose of this application is to provide a method and device for identifying abnormal order issuance, which can automatically and effectively identify the illegal behavior of having transaction orders without customers in a store, thereby improving the store's business management.

[0004] In a first aspect, the present application provides a method for identifying abnormal order issuance, the method comprising:

[0005] Acquire a transaction event, and determine the transaction time based on the transaction event, wherein the transaction event is generated by the cash register device according to the transaction instruction;

[0006] Acquire multiple frames of target images within a time period associated with the transaction time, wherein the associated time period is a preset time period before and / or a preset time period after the transaction time, and the target image is captured by an image acquisition device when monitoring the target environment area;

[0007] Determine a first detection image area in the target image, and identify whether there is a customer target in the first detection image area, wherein the first detection image area is an area within a preset range around the cash register device;

[0008] Determine the number of image frames in which the customer target does not exist, and if the image frame number is greater than a preset first frame number threshold, output an alarm instruction; or determine the number of image frames in which the customer target exists, and if the image frame number is less than a preset second frame number threshold, output an alarm instruction.

[0009] In one embodiment, the method further comprises:

[0010] Determine the number of image frames in which the customer target does not exist, and if the number of image frames is less than a first frame number threshold, determine the number of goods sold based on the transaction event; or determine the number of image frames in which the customer target exists, and if the number of image frames is greater than a second frame number threshold, determine the number of goods sold based on the transaction event;

[0011] Determine the number of items actually acquired by the customer target;

[0012] If the quantity of goods sold is greater than the actual quantity of goods obtained by the customer target, an alarm instruction is output.

[0013] In one embodiment, determining the number of commodities actually acquired by the target customer includes:

[0014] Determine a second detection image area in the target image, and identify a first quantity placed on the cash register based on the second detection image area, wherein the second detection image area includes the cash register;

[0015] Determine a third detection image area in the target image, and identify a second quantity of commodities held by the target customer based on the third detection image area; wherein the third image detection area is an area within a preset range surrounding the cash register device, and the third detection image area is the same as or different from the first detection image area;

[0016] If the first quantity is the same as the second quantity, it is determined that the first quantity or the second quantity is the quantity of the goods actually obtained by the target customer.

[0017] In one embodiment, the method further comprises:

[0018] Determine the quantity of goods sold according to the transaction order in the transaction event;

[0019] If the first quantity is different from the second quantity, comparing the larger of the first quantity and the second quantity with the quantity of goods sold, and if the larger of the first quantity and the second quantity is less than the quantity of goods sold, outputting an alarm instruction;

[0020] or,

[0021] Determine the quantity of goods sold according to the transaction order in the transaction event;

[0022] If the first quantity is different from the second quantity, the first quantity and the second quantity are compared with the quantity of goods sold respectively, and if any one of them is less than the quantity of goods sold, an alarm instruction is output.

[0023] In one embodiment, identifying a first quantity of commodities placed on a checkout counter based on the second detection image area includes:

[0024] Obtaining an identification frame of each commodity in the second detection image area;

[0025] Get the identification frame of the cash register;

[0026] If the identification frame of the product intersects with the identification frame of the checkout counter, or intersects with the identification frames of other products, it is counted into the first quantity.

[0027] In one embodiment, identifying the second quantity of commodities held by the target customer based on the third detection image area includes:

[0028] Identify at least one outer packaging detection frame in the third detection image area, determine the size and / or pattern of the outer packaging detection frame, and determine the size type of the outer packaging based on the size and / or pattern of the outer packaging detection frame;

[0029] Based on a preset mapping relationship between the size type of the outer packaging and the quantity of the unit packaging, determining the quantity of the unit packaging placed in the outer packaging based on the size type of the outer packaging;

[0030] Based on a preset mapping relationship between unit packages and commodity quantities, the second quantity is determined according to the quantity of unit packages placed in the outer package.

[0031] In a second aspect, the present application provides a device for identifying abnormal order issuance, the device comprising:

[0032] A first acquisition module is used to acquire a transaction event and determine a transaction time based on the transaction event, wherein the transaction event is generated by a cash register device according to a transaction instruction;

[0033] A second acquisition module is used to acquire multiple frames of target images within a time period associated with the transaction time, wherein the associated time period is a preset time period before and / or a preset time period after the transaction time, and the target image is obtained by taking pictures when the image acquisition device monitors the target environment area;

[0034] A recognition module, used to determine a first detection image area in the target image, and to identify whether a customer target exists in the first detection image area, wherein the first detection image area is an area within a preset range around the cash register device;

[0035] The alarm module is used to determine the number of image frames in which the customer target does not exist, and output an alarm instruction if the image frame number is greater than a preset first frame number threshold; or to determine the number of image frames in which the customer target exists, and output an alarm instruction if the image frame number is less than a preset second frame number threshold.

[0036] In a third aspect, the present application provides a system for identifying abnormal order issuance, the system comprising:

[0037] An image acquisition device, used for acquiring multiple frames of target images of a target environment area;

[0038] The server is used to obtain transaction events, determine the transaction time based on the transaction events, obtain multiple frames of target images within a time period associated with the transaction time, determine a first detection image area in the target image, identify whether there is a customer target in the first detection image area, determine the number of image frames where no customer target exists, and output an alarm instruction if the number of image frames is greater than a preset first frame number threshold; or determine the number of image frames where a customer target exists, and output an alarm instruction if the number of image frames is less than a preset second frame number threshold, wherein the transaction event is generated by a cash register device according to a transaction instruction, the associated time period is a preset time period before and / or a preset time period after the transaction time, and the first detection image area is an area within a preset range around the cash register device.

[0039] In a fourth aspect, the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for identifying abnormal order issuance as described above are implemented.

[0040] In a fifth aspect, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for identifying abnormal order issuance as described above.

[0041] The present application obtains a target image in a time period before and / or after the transaction time, identifies a first detection image area in the target image, and determines whether there are customers in the area. If the number of image frames without customers is greater than a first frame number threshold, or if the number of image frames with customers is less than a second frame number threshold, it is determined that a transaction has occurred in which there is a transaction order but no customer, and that a staff member has violated the regulations by purchasing goods in the store without permission. The application can accurately determine whether there are any violations by staff in the store, can automatically and effectively identify violations by staff, thereby improving the efficiency of store management and reducing the difficulty and cost of managing staff. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of a first application scenario provided in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of a second application scenario provided in an embodiment of the present application;

[0044] Figure 3 A first flow chart of a method for identifying abnormal order placement provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of area division provided in an embodiment of the present application;

[0046] Figure 5 A second flow chart of the method for identifying abnormal order placement provided in an embodiment of the present application;

[0047] Figure 6 A third flow chart of the method for identifying abnormal order placement provided in an embodiment of the present application;

[0048] Figure 7 A fourth flow chart of a method for identifying abnormal order placement provided in an embodiment of the present application;

[0049] Figure 8 A fifth flow chart of a method for identifying abnormal order placement provided in an embodiment of the present application;

[0050] Fig. 9 A system block diagram of a device for identifying abnormal order placement provided in an embodiment of the present application;

[0051] Fig.10 A system block diagram of a system for identifying abnormal order placement provided in an embodiment of the present application;

[0052] Fig.11 A system block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The present application will be described in detail below in conjunction with the specific implementation modes shown in the accompanying drawings, but these implementation modes do not limit the present application. Structural, methodological, or functional changes made by ordinary technicians in the field based on these implementation modes are included in the protection scope of the present application.

[0054] The embodiment of the present application provides a method for identifying abnormal bill issuance. The execution subject of the method can be a camera, the camera can be an IPC (Internet Protocol Camera, a camera with an interconnection protocol between networks), and the camera can be connected to a cash register, such as Figure 1 One of the application scenarios of the embodiment of the present application shown in the figure. The camera may be provided with a processor, a memory, a transceiver and a camera device, wherein the camera device may be used to capture images, that is, to obtain target images, the transceiver may be used to receive and send data, that is, to receive transaction events sent by a cash register, and the memory may be used to process the target image.

[0055] The execution subject of the method for identifying abnormal order placement in the embodiment of the present application may be an electronic device, such as Figure 2 One of the application scenarios of the embodiment of the present application shown. A camera is connected to an electronic device, and a cash register is connected to the electronic device. The camera is used to capture images. The electronic device obtains transaction events of the cash register and performs image processing based on the target image captured by the camera.

[0056] Exemplarily, the electronic device may be a terminal or other device, including but not limited to a mobile terminal and a fixed terminal. For example, a mobile terminal includes but is not limited to a smart phone, a smart watch, a tablet computer, a laptop computer, a smart vehicle, a smart car-mounted device, etc., wherein a fixed terminal includes but is not limited to a desktop computer, a smart TV, etc.

[0057] Exemplarily, the electronic device may also be a device such as a server, which may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms, but is not limited to these.

[0058] like Figure 3 As shown, an embodiment of the present application provides a method for identifying abnormal order issuance, which includes steps S301-S304.

[0059] Step S301, obtaining a transaction event, and determining a transaction time based on the transaction event, wherein the transaction event is generated by a cash register device according to a transaction instruction.

[0060] Based on the transaction instruction input by the cashier, the cash register device outputs a transaction event, obtains the transaction event, and determines the transaction time according to the transaction event. The transaction event may include transaction order information, cash register device identification information (such as POS machine number), and store information. The transaction order information may include transaction time, transaction commodity quantity, amount, etc.

[0061] Step 302, obtaining multiple frames of target images within a time period associated with the transaction time, wherein the associated time period is a preset time period before the transaction time and / or a preset time period after the transaction time, and the target image is captured by an image acquisition device when monitoring the target environment area.

[0062] Exemplarily, the first target environmental area may be an environmental area within a store such as a chain store.

[0063] For example, the image corresponding to the first target environment area is collected and acquired by a camera installed near the cashier counter. In actual application, the camera can be mounted above the cashier counter area to fully capture the cashier, cash register, and customers in the transaction. The installation location of the camera can be determined according to the site conditions of the place where the camera is used, and this application does not limit this.

[0064] For example, the preset time period before the transaction time and the preset time period after the transaction time can be set to different durations, or the same duration, which is not limited here. The preset time period before the transaction time and the preset time period after the transaction time can be set according to the actual needs of the user.

[0065] For example, in an actual transaction process, the customer usually pays for the goods first, the cashier collects the payment, the cashier enters the transaction instruction, the cash register outputs the transaction event based on the transaction instruction, and the customer takes the purchased goods and leaves directly. Therefore, the preset time period before the transaction time can be set to be longer than the preset time period after the transaction time, so as to avoid partial overlap of the image associated with the next transaction time and the image associated with the current transaction time, so as to accurately obtain the target image associated with the current transaction time.

[0066] Exemplarily, a video stream within a time period associated with a transaction time is obtained, and frame extraction is performed on video frames in the video stream to obtain multiple frames of target images within the time period associated with the transaction time, that is, multiple frames of target images within a preset time period before the transaction time are obtained, and multiple frames of target images within a preset time period after the transaction time are obtained, and the number of frames of the target image is related to the length of the preset time period.

[0067] Step S303, determining a first detection image area in the target image, and identifying whether there is a customer target in the first detection image area, wherein the first detection image area is an area within a preset range surrounding the cash register device.

[0068] Exemplarily, the first target environment area may be an environment area within a store such as a chain store, and the first detection image area may be a cash register area within the store, and the first detection image area includes a cash register device.

[0069] Exemplarily, based on the environment layout in the store, the target scene can be divided into regions, as shown in the regional division schematic diagram 4, in which the triangle represents the cashier 401, the rectangle represents the cash register 402, and the circle represents the customer 403. The upper rectangular box in the figure is divided into the customer area, and the lower rectangular box is divided into the cash register area. With the cash register as the center, the transaction range radius R is set to delineate the transaction area, that is, the dotted circle area is the transaction area. The customer area, the cash register area and the transaction area are artificially divided in advance on the image according to the actual scenes of different stores. The divided areas can be regular shapes or irregular shapes. The camera is generally installed in a fixed position. Based on the pre-divided area, the first detection image area can be directly determined in the acquired target image, that is, the first detection image area corresponds to the transaction area. Specifically, the coordinates of each pre-generated area are obtained, and the first detection image area is determined in the target image according to the coordinates of each area. The various areas under the camera's viewing angle can be determined by image segmentation, edge detection, etc., and each area can also be defined in the camera by manual annotation to generate the coordinates of each area. It should be noted that if the layout of the camera or the target environment area changes, the coordinates of each area can be updated and adjusted accordingly. By pre-generating the coordinates of each area, the detection area in the image can be simply and efficiently located, which is conducive to effectively and accurately identifying the target object in the first detection image area.

[0070] The first detection image area is an area within a preset range surrounding the cash register device. The target object in this area may be a customer, a staff member, or a cash register device. Target object recognition is performed on the first detection image area to determine whether there is a customer target in the first detection image area.

[0071] For example, staff members generally wear uniforms specified by the store, or wear store badges, etc. Store transaction scene images are collected as training sample sets, and the transaction scene images include staff members, customers, and cash registers. The staff members, customers, and cash registers in the images are marked with categories, and the marked training sets are input into the neural network model for training, and the categories of each target object are output to obtain a trained target detection network model. Based on the target detection network model, the first detection image area is identified to determine whether there is a customer target in the first detection image area. The target detection network model can be a neural network model, which is not limited here.

[0072] For example, the staff in the store are generally fixed, so a face library of the staff can be set in advance, and the staff can enter the face in advance and recognize it. When performing object recognition on the first detection image area, the face target in the area is obtained, the face features of the face target are extracted, and the face features are compared with the face library one by one. If the comparison is unsuccessful, it is determined that the face target is a customer. Face recognition can be achieved through a pre-trained face detection network. The face detection network model can be a neural network model, which is not limited here.

[0073] Step S304, determine the number of image frames in which the customer target does not exist, and if the image frame number is greater than a preset first frame number threshold, output an alarm instruction; or determine the number of image frames in which the customer target exists, and if the image frame number is less than a preset second frame number threshold, output an alarm instruction.

[0074] Exemplarily, when it is determined that there is no customer target in the first detection image area, the number of image frames without the customer target is determined based on the first detection image area in each frame of the target image, and if the number of image frames is greater than the preset first frame number threshold, an alarm instruction is output. The first frame number threshold can determine the number of image frames according to the interval time of image acquisition. If the number of image frames without the customer target is greater than the preset first frame number threshold, it indicates that within the preset time period before the transaction time and / or the preset time period after the transaction time, the cash register device outputs a transaction event, that is, a commodity transaction has occurred, but there is no customer in the designated detection area, and a violation of the transaction order without the customer has occurred, so it is determined that a violation of the staff's private purchase of the goods in the store has occurred. This alarm instruction can be to send an alarm message to the management personnel, or to display a warning on the monitoring system to remind the relevant management personnel to take corresponding measures to control the private purchase behavior.

[0075] Exemplarily, when it is determined that there is a customer target in the first detection image area, the number of image frames in which the customer target exists is determined based on the first detection image area in each frame of the target image. If the number of image frames is greater than a preset second frame number threshold, an alarm instruction is output. The second frame number threshold can determine the number of image frames according to the interval time of image acquisition. The second frame number threshold is much smaller than the first frame number threshold. If the number of image frames in which the customer target exists is greater than the preset second frame number threshold, it means that in the acquired multiple frames of target images, the number of frames in which the customer target does not exist is much greater than the number of frames in which the customer target exists, indicating that in the preset time period before the transaction time and / or the preset time period after the transaction time, a violation of the transaction order without a customer has occurred. Therefore, it is determined that a violation of the staff member's private purchase of in-store goods has occurred, and an alarm instruction is output.

[0076] This embodiment obtains the target image in the time period before / or after the transaction time, identifies the first detection image area in the target image, determines whether there are customers in the area, and if the number of image frames without customers is greater than the first frame number threshold, or if the number of image frames with customers is less than the second frame number threshold, it is determined that there is a transaction behavior with a transaction order but no customers, and it is determined that there is a violation of the staff's private purchase of goods in the store. For example, when the staff knows that the store has a discount, they take advantage of the loopholes in the rules to purchase goods themselves and take advantage of the discount to obtain the interest rate difference. By determining the number of image frames without customers or the number of image frames without customers, it is possible to accurately determine whether there are customers in the specified area (first image detection area) in the time period before / or after the transaction time. For example, if there are passing customers who only stay in the first detection image area for a relatively short time, the influence of these customers on the judgment of the violation of having orders but no customers can be eliminated, so that it is possible to accurately determine whether there is a violation of the staff, and it is possible to automatically and effectively identify the violation of the staff, thereby improving the efficiency of store management and reducing the management difficulty and management cost of the staff.

[0077] In actual applications, staff members use the store's discount rules to purchase goods with customers, thereby taking advantage of the discount to obtain the interest rate difference, so it is necessary to judge this violation. In some embodiments, such as Figure 5 As shown, the method for identifying abnormal orders also includes the steps of:

[0078] Step S501, determine the number of image frames in which no customer target exists, and if the image frame number is less than the first frame number threshold, determine the number of goods sold based on the transaction event; or determine the number of image frames in which a customer target exists, and if the image frame number is greater than the frame number threshold, determine the number of goods sold based on the transaction event.

[0079] Exemplarily, when it is determined that there is no customer target in the first detection image area, if the image frame number is less than the first frame number threshold, and the number of goods sold is determined based on the transaction event, it indicates that there is a transaction behavior of a customer purchasing goods in the time period before and / or after the transaction time, and it is necessary to further determine whether the transaction behavior is that the customer purchased the goods by himself or the staff and the customer purchased the goods together.

[0080] Exemplarily, when it is determined that there is a customer target in the first detection image area, if the image frame number is greater than the second frame number threshold, and the number of goods sold is determined based on the transaction event, it indicates that there is a transaction behavior of a customer purchasing goods in the time period before / or after the transaction time, and it is necessary to further determine whether the transaction behavior is the customer purchasing the goods by himself or the staff and the customer purchasing the goods together.

[0081] Exemplarily, the transaction order information in the transaction event is obtained, and the quantity of transaction commodities, that is, the quantity of sold commodities, can be determined according to the transaction order information.

[0082] Step S502, determining the quantity of commodities actually acquired by the target customer.

[0083] The first detection image area in the target image is identified to obtain the number of commodities actually obtained by the target customer.

[0084] For example, a convolutional neural network for target detection may be used to detect the number of commodities actually acquired by the customer target in the first detection image area. The specific detection method is not limited.

[0085] Step S503: If the quantity of goods sold is greater than the quantity of goods actually obtained by the customer target, an alarm instruction is output.

[0086] If the number of goods sold is greater than the number of goods actually obtained by the customer, it indicates that the staff and the customer have placed an order together, and an alarm instruction is output based on this behavior.

[0087] In this embodiment, in this transaction event, by judging whether the number of goods sold is greater than the number of goods actually obtained by the customer, if so, it indicates that there are staff members who take advantage of the store's discount rules to purchase goods together with customers and use the discount to obtain the interest rate difference, thereby accurately judging whether there are any violations by staff members in the store, and automatically and effectively identifying any violations by staff members, thereby improving the efficiency of store management and reducing the difficulty and cost of managing staff members.

[0088] In some embodiments, Figure 6 As shown, the steps for determining the number of commodities actually obtained by the customer target include:

[0089] Step S601, determining a second detection image area in a target image, and identifying a first quantity placed on a cash register based on the second detection image area, wherein the second detection image area includes the cash register.

[0090] Exemplarily, target recognition is performed on the first detection image area in the target image to identify the cash register target in the first detection image area, and the area near the cash register is used as the second detection image area. The second detection image area is recognized to identify the number of goods placed on the cash register and record it as the first quantity.

[0091] Exemplarily, a convolutional neural network for target detection, such as a YOLO network, can be used to detect items in the second detection image area to obtain an item target rectangular frame, each item corresponds to an item target rectangular frame, and the first quantity placed on the cashier counter can be determined based on the number of item target rectangular frames. Alternatively, other target detection algorithms can be used to detect the first quantity placed on the cashier counter in the second detection image area. The specific detection method is not limited.

[0092] Step S602, determine a third detection image area in the target image, and identify a second quantity of goods held by the target customer based on the third detection image area, wherein the third image detection area is an area within a preset range surrounding the cash register device, and the third detection image area is the same as or different from the first detection image area.

[0093] The target image also includes a third detection image area, which is an area within a preset range around the cash register device, and the third detection image area is the same as or different from the first detection image area. The third detection image area can be determined based on a pre-divided area method. Please refer to the above description for details, which will not be repeated here.

[0094] Target recognition is performed on the third detection image area to identify the second number of commodities held by the target customer. Exemplarily, image recognition is performed on the third detection image area to identify the hand object of the target customer, and the number of commodities held in the hand, i.e., the second number of commodities held by the target customer, is identified. For example, a key point detection algorithm of a convolutional neural network may be used to detect the key points of the hand in the third detection image area. Alternatively, other target detection algorithms may be used to detect the customer's hands in the third detection image area. The specific detection method is not limited. To detect the number of commodities held by the customer, a convolutional neural network for target detection may be used to detect the items in the image, obtain the target rectangular frame of the items, and determine the number held by the target customer's hand, i.e., the second number, based on the number of target rectangular frames.

[0095] Step S603: If the first quantity is the same as the second quantity, determine that the first quantity or the second quantity is the quantity of goods actually obtained by the target customer.

[0096] If the first quantity obtained is consistent with the second quantity, the first quantity or the second quantity is used as the quantity of goods actually obtained by the customer target.

[0097] In this embodiment, the first quantity placed on the cash register is identified through the second detection image area, and the second quantity of the goods held by the customer target is identified through the third detection image area. The first quantity and the second quantity are compared. If the two are consistent, the first quantity or the second quantity is used as the number of goods actually obtained by the customer target. The number of goods actually obtained by the customer target is obtained by two methods, and the obtained number of goods is compared, so as to accurately obtain the number of goods actually purchased by the customer, and try to avoid the situation where image recognition errors lead to errors in subsequent judgments, so as to accurately judge whether the transaction behavior is the customer purchasing the goods himself, or the staff and the customer purchase the goods together.

[0098] In some embodiments, the steps of the method for identifying abnormal orders include:

[0099] Determine the quantity of goods sold according to the transaction order in the transaction event;

[0100] If the first quantity is different from the second quantity, the larger one between the first quantity and the second quantity is compared with the quantity of goods sold. If it is less than the quantity of goods sold, an alarm instruction is output.

[0101] In this embodiment, the number of goods sold is determined through the transaction order, and the first number and the second number actually obtained by the customer are determined based on the above two methods. When the first number and the second number are different, it means that when the first number or the second number is identified above, there is an image recognition error, resulting in inconsistency between the two quantities. Therefore, the larger of the first number and the second number is compared with the number of goods sold, and the larger one is taken as the number of goods actually obtained by the customer. If the larger one is less than the number of goods sold, an alarm instruction is output, that is, the number of goods actually obtained by the customer is less than the number of goods sold, indicating that there is a violation of the rules of store discounts, and the staff takes advantage of the store's discount rules to purchase goods with customers, thereby taking advantage of the discount to obtain the interest rate difference, and an alarm message is output for this violation.

[0102] In some embodiments, the steps of the method for identifying abnormal orders include:

[0103] Determine the quantity of goods sold according to the transaction order in the transaction event;

[0104] If the first quantity is different from the second quantity, the first quantity, the second quantity and the quantity of goods sold are compared respectively, and if any one of them is less than the quantity of goods sold, an alarm instruction is output.

[0105] In this embodiment, the number of goods sold is determined through the transaction order, and the first quantity and the second quantity actually obtained by the customer are determined based on the above two methods. When the first quantity and the second quantity are different, it means that there is an image recognition error when the first quantity or the second quantity is identified above, resulting in inconsistency between the two quantities. Therefore, the first quantity and the second quantity are compared with the number of goods sold respectively. If both are less than the number of goods sold, an alarm instruction is output, that is, the number of goods actually obtained by the customer is less than the number of goods sold, indicating that there is a staff member taking advantage of the store's discount rules to purchase goods with customers, thereby using the discount to obtain the interest rate difference, and an alarm message is output for this violation.

[0106] In some embodiments, Figure 7 As shown, the step of identifying the first quantity placed on the cashier counter based on the second detection image area includes:

[0107] Step S701, obtaining the identification frame of each commodity in the second detection image area;

[0108] Step S702, obtaining the identification frame of the cash register;

[0109] Step S703: If the identification box of the product intersects with the identification box of the checkout counter or with the identification boxes of other products, it is counted into the first quantity.

[0110] The second detection image area is identified to obtain identification frames of each commodity and the identification frame of the cash register. If the identification frame of the commodity intersects with the identification frame of the cash register or with the identification frame of other commodities, the identification frame is counted into the first quantity. For example, when there is only one commodity, the first quantity of commodities placed on the cash register can be determined based on the intersection of the identification frame of the commodity with the identification frame of the cash register. If there are multiple commodities, the first quantity of commodities on the cash register can be determined based on the intersection of the identification frame of the commodity with the identification frame of the cash register and the intersection of the identification frame of the commodity with the identification frames of other commodities.

[0111] Exemplarily, the method for recognizing the recognition frame of the commodity target and the recognition frame of the checkout counter target may adopt a convolutional neural network for target detection, such as a YOLO network.

[0112] In this embodiment, with regard to obtaining the first quantity placed on the cash register, any goods placed on the cash register, or multiple goods stacked on the cash register, are counted in the first quantity. Goods that are not placed on the cash register, such as goods held in the hands of customers, are not counted in the first quantity. Based on this implementation, the first quantity of goods placed on the cash register can be accurately determined, thereby providing a correct basis for judging between the subsequent actual quantity obtained by the customer target and the quantity of goods sold.

[0113] In some embodiments, Figure 8 As shown, the step of identifying the second quantity of commodities held by the customer target based on the third detection image area includes:

[0114] Step S801, identifying at least one outer packaging detection frame in the third detection image area, determining the size and / or pattern of the outer packaging detection frame, and determining the size type of the outer packaging based on the size and / or pattern of the outer packaging detection frame;

[0115] Step S802, based on a preset mapping relationship between the size type of the outer packaging and the quantity of the unit packaging, determine the quantity of the unit packaging put into the outer packaging based on the size type of the outer packaging;

[0116] Step S803: Based on a preset mapping relationship between unit packaging and commodity quantity, a second quantity of the commodity held by the target customer is determined according to the quantity of the unit packaging.

[0117] The packaging of the items in the third detection image area is detected, and at least one outer packaging detection frame is identified in the image area. The size type of the outer packaging can be determined based on the size of the identified outer packaging detection frame. In practical applications, the camera is generally fixedly installed, and the distance and angle between the camera and the cash register are also fixed. Therefore, the collected image can be used for image recognition to obtain the size of the outer packaging. Exemplarily, the recognition of the outer packaging can use a pre-trained packaging recognition model. The size type of the outer packaging can also be identified based on the pattern or QR code of the outer packaging.

[0118] It is understandable that the number of unit packages loaded into the outer packaging will be different depending on the size of the outer packaging. Taking a shoe box as an example, a small outer packaging can only hold one shoe box, while a large outer packaging can hold three shoe boxes. Therefore, based on the size type of the outer packaging and the size of the unit packaging, a mapping relationship between the size type of the outer packaging and the number of unit packages is pre-established, that is, a mapping relationship between how many unit packages can be loaded into outer packaging of different sizes is established. Therefore, based on the identified size type of the outer packaging, the number of unit packages loaded into the outer packaging is determined.

[0119] The unit package can be understood as the smallest packaging unit. The number of commodities that can be put into the unit package may be various, for example, 1, 2, 3 commodities, etc. Therefore, a mapping relationship between the unit package and the number of commodities is established, that is, the number of commodities put into the unit package. The number of unit packages put into the outer package is determined, and the number of commodities in the unit package can be determined based on the above mapping relationship, that is, the second number of commodities held by the target customer can be determined.

[0120] This embodiment can determine the quantity of goods purchased by the customer by identifying the size of the outer packaging, thereby providing a correct basis for the subsequent judgment between the actual quantity obtained by the customer target and the quantity of goods sold.

[0121] Please refer to Fig. 9 The embodiment of the present application provides a device for identifying abnormal order issuance, the device comprising:

[0122] A first acquisition module 901 is used to acquire a transaction event and determine a transaction time based on the transaction event, wherein the transaction event is generated by a cash register device according to a transaction instruction;

[0123] The second acquisition module 902 is used to acquire multiple frames of target images within a time period associated with the transaction time, wherein the associated time period is a preset time period before and / or a preset time period after the transaction time, and the target image is obtained by taking pictures when the image acquisition device monitors the target environment area;

[0124] The recognition module 903 is used to determine a first detection image area in the target image and identify whether there is a customer target in the first detection image area, wherein the first detection image area is an area within a preset range around the cash register device;

[0125] The alarm module 904 is used to determine the number of image frames in which the customer target does not exist, and output an alarm instruction if the image frame number is greater than a preset first frame number threshold; or to determine the number of image frames in which the customer target exists, and output an alarm instruction if the image frame number is less than a preset second frame number threshold.

[0126] This embodiment obtains the target image in the time period before / or after the transaction time, identifies the first detection image area in the target image, determines whether there are customers in the area, and if the number of image frames without customers is greater than the first frame number threshold, or if the number of image frames with customers is less than the second frame number threshold, determines that there is a transaction behavior with a transaction order but no customers, and determines that there is a violation of the staff's private purchase of goods in the store, for example, when the staff knows that the store has a discount, they take advantage of loopholes in the rules to buy goods themselves and take advantage of the discount to obtain the interest rate difference. It can accurately determine whether there are customers in the designated area (first image detection area) in the time period before / or after the transaction time, so as to accurately determine whether there is a violation of the staff, and can automatically and effectively identify the violation of the staff, thereby improving the efficiency of store management and reducing the difficulty and cost of managing the staff.

[0127] In one embodiment, the identification module 903 is specifically used for:

[0128] Determine the number of image frames in which the customer target does not exist, and if the number of image frames is less than a first frame number threshold, determine the number of goods sold based on the transaction event; or determine the number of image frames in which the customer target exists, and if the number of image frames is greater than a second frame number threshold, determine the number of goods sold based on the transaction event;

[0129] Determine the number of items actually acquired by the customer target;

[0130] If the quantity of goods sold is greater than the actual quantity of goods obtained by the customer target, an alarm instruction is output.

[0131] In one embodiment, the identification module 903 is specifically used for:

[0132] Determine a second detection image area in the target image, and identify a first quantity placed on the cash register based on the second detection image area, wherein the second detection image area includes the cash register;

[0133] Determine a third detection image area in the target image, and identify a second quantity of commodities held by the target customer based on the third detection image area; wherein the third image detection area is an area within a preset range surrounding the cash register device, and the third detection image area is the same as or different from the first detection image area;

[0134] If the first quantity is the same as the second quantity, it is determined that the first quantity or the second quantity is the quantity of the goods actually obtained by the target customer.

[0135] In one embodiment, the identification module 903 is specifically used for:

[0136] Determine the quantity of goods sold according to the transaction order in the transaction event;

[0137] If the first quantity is different from the second quantity, comparing the larger of the first quantity and the second quantity with the quantity of goods sold, and if the larger of the first quantity and the second quantity is less than the quantity of goods sold, outputting an alarm instruction;

[0138] or,

[0139] Determine the quantity of goods sold according to the transaction order in the transaction event;

[0140] If the first quantity is different from the second quantity, the first quantity and the second quantity are compared with the quantity of goods sold respectively, and if any one of them is less than the quantity of goods sold, an alarm instruction is output.

[0141] In one embodiment, the identification module 903 is specifically used for:

[0142] Obtaining an identification frame of each commodity in the second detection image area;

[0143] Get the identification frame of the cash register;

[0144] If the identification frame of the product intersects with the identification frame of the checkout counter, or intersects with the identification frames of other products, it is counted into the first quantity.

[0145] In one embodiment, the identification module 903 is specifically used for:

[0146] Identify at least one outer packaging detection frame in the third detection image area, determine the size and / or pattern of the outer packaging detection frame, and determine the size type of the outer packaging based on the size and / or pattern of the outer packaging detection frame;

[0147] Based on a preset mapping relationship between the size type of the outer packaging and the quantity of the unit packaging, determining the quantity of the unit packaging placed in the outer packaging based on the size type of the outer packaging;

[0148] Based on a preset mapping relationship between unit packages and commodity quantities, the second quantity is determined according to the quantity of unit packages placed in the outer package.

[0149] like Fig.10 As shown, an embodiment of the present application provides a system for identifying abnormal order issuance, the system comprising:

[0150] Image acquisition device 1001, used to acquire multiple frames of target images of a target environment area;

[0151] Server 1002 is used to obtain transaction events, determine transaction time based on transaction events, obtain multiple frames of target images within a time period associated with the transaction time, determine a first detection image area in the target image, identify whether there is a customer target in the first detection image area, determine the number of image frames where no customer target exists, and output an alarm instruction if the number of image frames is greater than a preset first frame number threshold; or determine the number of image frames where a customer target exists, and output an alarm instruction if the number of image frames is less than a preset second frame number threshold, wherein the transaction event is generated by the cash register device according to the transaction instruction, the associated time period is a preset time period before and / or a preset time period after the transaction time, and the first detection image area is an area within a preset range around the cash register device.

[0152] In one embodiment, the server 1002 is specifically used for:

[0153] Determine the number of image frames in which the customer target does not exist, and if the number of image frames is less than a first frame number threshold, determine the number of goods sold based on the transaction event; or determine the number of image frames in which the customer target exists, and if the number of image frames is greater than a second frame number threshold, determine the number of goods sold based on the transaction event;

[0154] Determine the number of items actually acquired by the customer target;

[0155] If the quantity of goods sold is greater than the actual quantity of goods obtained by the customer target, an alarm instruction is output.

[0156] In one embodiment, the server 1002 is specifically used for:

[0157] Determine a second detection image area in the target image, and identify a first quantity placed on the cash register based on the second detection image area, wherein the second detection image area includes the cash register;

[0158] Determine a third detection image area in the target image, and identify a second quantity of commodities held by the target customer based on the third detection image area; wherein the third image detection area is an area within a preset range surrounding the cash register device, and the third detection image area is the same as or different from the first detection image area;

[0159] If the first quantity is the same as the second quantity, it is determined that the first quantity or the second quantity is the quantity of the goods actually obtained by the target customer.

[0160] In one embodiment, the server 1002 is specifically used for:

[0161] Determine the quantity of goods sold according to the transaction order in the transaction event;

[0162] If the first quantity is different from the second quantity, comparing the larger of the first quantity and the second quantity with the quantity of goods sold, and if the larger of the first quantity and the second quantity is less than the quantity of goods sold, outputting an alarm instruction;

[0163] or,

[0164] Determine the quantity of goods sold according to the transaction order in the transaction event;

[0165] If the first quantity is different from the second quantity, the first quantity and the second quantity are compared with the quantity of goods sold respectively, and if any one of them is less than the quantity of goods sold, an alarm instruction is output.

[0166] In one embodiment, the server 1002 is specifically used for:

[0167] Obtaining an identification frame of each commodity in the second detection image area;

[0168] Get the identification frame of the cash register;

[0169] If the identification frame of the product intersects with the identification frame of the checkout counter, or intersects with the identification frames of other products, it is counted into the first quantity.

[0170] In one embodiment, the server 1002 is specifically used for:

[0171] Identify at least one outer packaging detection frame in the third detection image area, determine the size and / or pattern of the outer packaging detection frame, and determine the size type of the outer packaging based on the size and / or pattern of the outer packaging detection frame;

[0172] Based on a preset mapping relationship between the size type of the outer packaging and the quantity of the unit packaging, determining the quantity of the unit packaging placed in the outer packaging based on the size type of the outer packaging;

[0173] Based on a preset mapping relationship between unit packages and commodity quantities, the second quantity is determined according to the quantity of unit packages placed in the outer package.

[0174] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any one of the above-mentioned methods for identifying abnormal order issuance is implemented.

[0175] Fig.11 It is a schematic diagram of the hardware structure of a computer device provided in an embodiment of the present application. Fig.11 The computer device shown includes: a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104. The processor 1101, the communication interface 1102 and the memory 1103 communicate with each other via the communication bus 1104. Fig.11 The connection method between the processor 1101, the communication interface 1102, and the memory 1103 shown is merely exemplary. During implementation, the processor 1101, the communication interface 1102, and the memory 1103 may also be connected to each other in communication with each other using other connection methods besides the communication bus 1104.

[0176] The memory 1103 can be used to store a computer program, which may include instructions and data to implement the steps of any of the above methods for identifying abnormal billing. In an embodiment of the present application, the memory 1103 may be various types of storage media, such as random access memory (RAM), read only memory (ROM), non-volatile RAM (NVRAM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, optical storage, and registers. The memory 1103 may include a hard disk and / or a memory.

[0177] The processor 1101 may be a general-purpose processor, which may be a processor that performs specific steps and / or operations by reading and executing a computer program (e.g., a computer program) stored in a memory (e.g., memory 1103). The general-purpose processor may use data stored in the memory (e.g., memory 1103) during the execution of the steps and / or operations. The general-purpose processor may be, for example, but not limited to, a central processing unit (CPU). In addition, the processor 1101 may also be a special-purpose processor, which may be a processor specially designed to perform specific steps and / or operations. The special-purpose processor may be, for example, but not limited to, an ASIC and an FPGA. In addition, the processor 1101 may also be a combination of multiple processors, such as a multi-core processor.

[0178] The communication interface 1102 may include an input / output (I / O) interface, a physical interface, and a logical interface for interconnecting devices within the network device, as well as an interface for interconnecting the network device with other devices (e.g., network devices). The communication network may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 1102 may be a module, a circuit, a transceiver, or any device capable of implementing communication.

[0179] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 1101 or an instruction in the form of software. The method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in the processor for execution. The software module can be located in a mature storage medium in the art such as a random access memory flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1103, and the processor 1101 reads the information in the memory 1103 and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0180] Although the preferred embodiments of the present application have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the application as disclosed in the accompanying claims.

Claims

1. A method for identifying abnormal order issuance, characterized in that: The method comprises: Acquire a transaction event, and determine a transaction time based on the transaction event, wherein the transaction event is generated by a cash register device according to a transaction instruction; Acquire multiple frames of target images within a time period associated with the transaction time, wherein the associated time period is a preset time period before and a preset time period after the transaction time, and the target images are captured by an image acquisition device when monitoring a target environment area; Determine a first detection image area in the target image, and identify whether there is a customer target in the first detection image area, wherein the first detection image area is an area within a preset range surrounding the cash register device; Determine the number of image frames indicating that the customer target does not exist, and if the number of image frames is greater than a preset first frame number threshold, output an alarm instruction; or determine the number of image frames indicating that the customer target exists, and if the number of image frames is less than a preset second frame number threshold, output an alarm instruction, wherein the first frame number threshold is the number of image frames determined according to the interval time of image acquisition, the second frame number threshold is the number of image frames determined according to the interval time of image acquisition, and the second frame number threshold is less than the first frame number threshold; The method further comprises: Determine the number of image frames in which the customer target does not exist, and if the number of image frames is less than the first frame number threshold, determine the number of goods sold based on the transaction event; or determine the number of image frames in which the customer target exists, and if the number of image frames is greater than the second frame number threshold, determine the number of goods sold based on the transaction event; Determining the quantity of merchandise actually acquired by the target customer; If the quantity of goods sold is greater than the quantity of goods actually obtained by the target customer, output an alarm instruction; Determining the number of commodities actually acquired by the target customer includes: Determine a second detection image area in the target image, and recognize a first quantity placed on the cashier counter based on the second detection image area, wherein the second detection image area includes the cashier counter; Determine a third detection image area in the target image, and identify a second quantity of commodities held by the target customer based on the third detection image area; wherein the third detection image area is an area within a preset range surrounding the cash register device, and the third detection image area is the same as or different from the first detection image area; If the first quantity is the same as the second quantity, it is determined that the first quantity or the second quantity is the quantity of goods actually acquired by the target customer.

2. The method for identifying abnormal order issuance according to claim 1, characterized in that: The method further comprises: Determine the quantity of goods sold according to the transaction order in the transaction event; If the first quantity is different from the second quantity, comparing the larger of the first quantity and the second quantity with the quantity of goods sold, and if the larger of the first quantity and the second quantity is less than the quantity of goods sold, outputting an alarm instruction; or, Determine the quantity of goods sold according to the transaction order in the transaction event; If the first quantity is different from the second quantity, the first quantity and the second quantity are compared with the quantity of goods sold respectively, and if any one of them is less than the quantity of goods sold, an alarm instruction is output.

3. The method for identifying abnormal order issuance according to claim 1, characterized in that: The step of identifying the first quantity placed on the cashier counter based on the second detection image area includes: Obtaining an identification frame of each commodity in the second detection image area; Obtaining an identification frame of the cash register; If the identification frame of the product intersects with the identification frame of the checkout counter or with the identification frame of other products, it is counted into the first quantity.

4. The method for identifying abnormal order issuance according to claim 1, characterized in that: The step of identifying the second quantity of commodities held by the target customer based on the third detection image area includes: Identify at least one outer packaging detection frame in the third detection image area, determine the size and / or pattern of the outer packaging detection frame, and determine the size type of the outer packaging based on the size and / or pattern of the outer packaging detection frame; Based on a preset mapping relationship between the size type of the outer packaging and the quantity of the unit packaging, determining the quantity of the unit packaging placed in the outer packaging based on the size type of the outer packaging; Based on a preset mapping relationship between unit packaging and commodity quantity, the second quantity is determined according to the quantity of unit packaging placed in the outer packaging.

5. A device for identifying abnormal order issuance, characterized in that: The device: A first acquisition module, configured to acquire a transaction event and determine a transaction time based on the transaction event, wherein the transaction event is generated by a cash register device according to a transaction instruction; A second acquisition module is used to acquire multiple frames of target images within a time period associated with the transaction time, wherein the associated time period is a preset time period before and a preset time period after the transaction time, and the target image is obtained by taking pictures when the image acquisition device monitors the target environment area; A recognition module, used for determining a first detection image area in the target image, and identifying whether there is a customer target in the first detection image area, wherein the first detection image area is an area within a preset range surrounding the cash register device; an alarm module, for determining the number of image frames in which the customer target does not exist, and outputting an alarm instruction if the image frame number is greater than a preset first frame number threshold; or for determining the number of image frames in which the customer target exists, and outputting an alarm instruction if the image frame number is less than a preset second frame number threshold, wherein the first frame number threshold is the number of image frames determined according to the interval time of image acquisition, the second frame number threshold is the number of image frames determined according to the interval time of image acquisition, and the second frame number threshold is less than the first frame number threshold; The recognition module is further configured to determine the number of image frames in which the target customer does not exist, and if the number of image frames is less than the first frame number threshold, determine the number of goods sold based on the transaction event; or determine the number of image frames in which the target customer exists, and if the number of image frames is greater than the second frame number threshold, determine the number of goods sold based on the transaction event; determine the number of goods actually obtained by the target customer; and output an alarm instruction if the number of goods sold is greater than the number of goods actually obtained by the target customer; The recognition module is also used to determine a second detection image area in the target image, and identify a first quantity placed on the cash register based on the second detection image area, wherein the second detection image area includes the cash register; determine a third detection image area in the target image, and identify a second quantity of goods held by the customer target based on the third detection image area; wherein the third detection image area is an area within a preset range surrounding the cash register device, and the third detection image area is the same as or different from the first detection image area; if the first quantity is the same as the second quantity, determine that the first quantity or the second quantity is the quantity of goods actually obtained by the customer target.

6. A system for identifying abnormal orders, characterized in that: The system comprises, An image acquisition device, used for acquiring multiple frames of target images of a target environment area; A server, configured to obtain a transaction event, determine a transaction time based on the transaction event, obtain multiple frames of target images within a time period associated with the transaction time, determine a first detection image area in the target image, and identify whether a customer target exists in the first detection image area, wherein the first detection image area is an area within a preset range surrounding a cash register device, determine the number of image frames in which the customer target does not exist, and if the image frame number is greater than a preset first frame number threshold, output an alarm instruction; or determine the number of image frames in which the customer target exists, and if the image frame number is less than a preset second frame number threshold, output an alarm instruction, wherein the transaction event is generated by the cash register device according to a transaction instruction, the associated time period is a preset time period before and a preset time period after the transaction time, the first frame number threshold is the number of image frames determined according to an interval time of image acquisition, the second frame number threshold is the number of image frames determined according to an interval time of image acquisition, and the second frame number threshold is much smaller than the first frame number threshold; The server is further configured to determine the number of image frames in which the target customer does not exist, and if the number of image frames is less than the first frame number threshold, determine the number of goods sold based on the transaction event; or determine the number of image frames in which the target customer exists, and if the number of image frames is greater than the second frame number threshold, determine the number of goods sold based on the transaction event; determine the number of goods actually obtained by the target customer; and output an alarm instruction if the number of goods sold is greater than the number of goods actually obtained by the target customer; The server is further used to determine a second detection image area in the target image, and identify a first quantity placed on the cash register based on the second detection image area, wherein the second detection image area includes the cash register; determine a third detection image area in the target image, and identify a second quantity of goods held by the customer target based on the third detection image area; wherein the third detection image area is an area within a preset range surrounding the cash register device, and the third detection image area is the same as or different from the first detection image area; if the first quantity is the same as the second quantity, determine that the first quantity or the second quantity is the quantity of goods actually obtained by the customer target.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method for identifying abnormal order issuance as described in any one of claims 1 to 4 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for identifying abnormal order issuance as described in any one of claims 1 to 4 are implemented.

Images