Sales data processor and program

Abstract

To improve convenience in a sales data processor that can switch between a clerk mode and a self-mode.SOLUTION: A sales data processor according to an embodiment includes a mode setting unit, a threshold designation unit, and an article registration unit. The mode setting unit sets an operation mode of either a clerk mode in which a clerk registers an article or a self-mode in which a customer registers the article. The threshold designation unit changes a threshold value of a recognition rate in package recognition to identify the article included in an image obtained by taking the image, according to the set operation mode, so that the threshold value of the recognition rate in the self-mode is higher than the threshold value of the recognition rate in the clerk mode. The article registration unit registers the article identified by the package recognition.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] An embodiment of the present invention relates to a sales data processing device and a program. [Background technology]

[0002] Conventionally, in stores such as supermarkets, convenience stores, and mass retailers, there are known sales data processing devices such as POS (Point of Sales) terminals that are configured to be able to switch between a clerk mode in which product data processing and payment processing (hereinafter collectively referred to as "transaction processing") for products purchased at the store and a self-service mode in which the customer himself / herself performs the transaction processing. For example, this sales data processing device can be operated such that when the store is busy, the POS terminal is switched to the clerk mode and the clerk performs the transaction processing, and when the store is quiet, all or part of the POS terminals are switched to the self-service mode and the customer himself / herself performs the transaction operation.

[0003] In such a situation, when product data processing is performed in which a captured image of a product is subjected to package recognition to determine the product and the product information of the product is registered as a product related to the current transaction, erroneous registration may occur due to, for example, the accuracy of the image recognition. If an erroneous registration occurs during operation in self-mode, a process requiring the intervention of a store clerk, such as a process in which the erroneously registered product is erased, is required. If a process requiring the intervention of a store clerk occurs, the burden on the store clerk increases and it takes time to complete the payment, which may reduce convenience. Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to improve the convenience of a sales data processing device that is switchable between a store clerk mode and a self-service mode. [Means for solving the problem]

[0005] A sales data processing device according to an embodiment includes a mode setting unit, a threshold designation unit, and a product registration unit. The mode setting unit sets whether the device operates in a clerk mode in which a clerk registers a product or a self-service mode in which a customer registers a product. The threshold designation unit changes a recognition rate threshold for package recognition that identifies a product included in an image obtained by capturing a picture, according to the set operation mode, such that the recognition rate threshold for the self-service mode is higher than the recognition rate threshold for the clerk mode. The product registration unit registers the product identified by the package recognition. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing an example of a configuration of a POS (Point of Sales) system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the edge device of FIG. [Diagram 3] FIG. 3 is a block diagram showing an example of a hardware configuration of the POS terminal of FIG. [Figure 4] FIG. 4 is a block diagram illustrating an example of a functional configuration of the edge device of FIG. [Diagram 5] FIG. 5 is a block diagram showing an example of a functional configuration of the POS terminal of FIG. [Figure 6] FIG. 6 is a diagram illustrating a configuration of threshold information according to the embodiment. [Figure 7] FIG. 7 is a sequence diagram showing an example of a processing flow in the system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] A sales data processing device, a sales data processing system, a program, and a recording medium according to an embodiment will be described below with reference to the drawings.

[0008] The embodiment described below is an embodiment of the sales data processing device, sales data processing system, program, and recording medium according to the embodiment, and does not limit the configuration, specifications, etc. The embodiment described below is an example of application to a POS (Point Of Sales) system introduced in a store such as a supermarket.

[0009] 1 is a diagram showing an example of a configuration of a POS system 1 according to an embodiment. The POS system 1 includes a server 4, an edge device 6, and a POS terminal 8, as shown in FIG.

[0010] 1 illustrates a POS system 1 including two or more POS terminals 8, but is not limited to this. The number of POS terminals 8 in the POS system 1 may be one or two, or may be three or more. That is, the POS system 1 according to the embodiment includes at least one POS terminal 8.

[0011] Here, the POS system 1 according to the embodiment is an example of a sales data processing system, and the POS terminal 8 according to the embodiment is an example of a sales data processing device.

[0012] In the POS system 1, the server 4, the edge device 6, and at least one POS terminal 8 are communicatively connected via a communication line Na. The communication line Na may be the Internet, a virtual private network (VPN), a local area network (LAN), a public communication network, a mobile communication network, or the like, either alone or in appropriate combination.

[0013] The server 4 is at least one server device installed, for example, in the backyard of a store. The server 4 is not limited to being installed in a store, and may be configured as a cloud server installed on a cloud.

[0014] The server 4 has a processor such as a CPU (Central Processing Unit) and memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and is configured similarly to a general computer. For example, the server 4 is configured to load various programs stored in a memory such as a ROM into the RAM of the memory, and to realize each function of the server 4 by having the processor execute the various loaded programs. As an example, the server 4 manages the types and numbers of products sold in the store. As an example, the server 4 manages sales at the store based on sales information received from the POS terminal 8. Specifically, the server 4 stores the sales information received from the POS terminal 8, and manages sales at the store using the stored sales information.

[0015] The server 4 may be configured to perform machine learning to improve the accuracy of the matching dictionary 624 stored in the edge device 6, for example, by using an image captured by the imaging unit 87 (see FIG. 3) of the POS terminal 8 and decoded information indicating a result of symbol decoding. Alternatively, the machine learning may be performed by the edge device 6.

[0016] Note that, instead of or in addition to the CPU, another processor may be provided as the processor of the server 4. As the other processor, various processors such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor), or a dedicated arithmetic circuit realized by an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array) can be appropriately used.

[0017] As the memory of the server 4, in addition to ROM and RAM, non-volatile recording media such as a hard disk drive (HDD), a solid state drive (SSD), and a flash memory can be appropriately used.

[0018] The edge device 6 is, for example, a computer provided adjacent to the server 4 or the POS terminal 8, provided within the server 4 or the POS terminal 8, or configured on the cloud.

[0019] Fig. 2 is a block diagram showing an example of a hardware configuration of the edge device 6 in Fig. 1. As shown in Fig. 2, the edge device 6 has a control unit 61, a storage unit 62, a controller 63, and a communication unit 64. The control unit 61, the storage unit 62, the controller 63, and the communication unit 64 are connected to each other so as to be able to communicate with each other via a bus 69, for example.

[0020] The control unit 61 has a processor such as a CPU and memories such as a ROM and a RAM, and is configured similarly to a general computer. For example, the control unit 61 is configured to load various programs stored in a memory such as a ROM or the storage unit 62 into the RAM of the memory, and to realize each function of the edge device 6 described below by the processor executing the various loaded programs.

[0021] As the processor of the control unit 61 according to the embodiment, various processors such as a GPU or DSP, or a dedicated arithmetic circuit realized by an ASIC or FPGA can be appropriately used instead of or in addition to a CPU.

[0022] The storage unit 62 is a storage device equipped with a non-volatile recording medium such as an HDD, an SSD, a flash memory, etc. The storage unit 62 stores various programs and various data related to the operation of the edge device 6, device drivers of external devices connected to the edge device 6 via the controller 63, etc.

[0023] As an example, the storage unit 62 stores a control program 621, an object detection model 622, a feature extraction model 623, and a matching dictionary 624. For example, the control program 621 includes at least one program for implementing each functional unit of the edge device 6. The object detection model 622, the feature extraction model 623, and the matching dictionary 624 will be described later.

[0024] The controller 63 is an input / output interface that connects the edge device 6 to an external device.

[0025] The communication unit 64 is a communication interface for communicating with the outside of the edge device 6. The communication unit 64 includes either a communication circuit for wired communication or a communication circuit for wireless communication, and is connected to the communication line Na via the communication circuit.

[0026] The POS terminal 8 is installed in stores such as supermarkets, convenience stores, mass retailers, etc. The POS terminal 8 is configured to be able to execute product data processing and payment processing (collectively referred to as "transaction processing") for products purchased in the store.

[0027] Here, product data processing refers to product registration processing that reads out and registers product information (product name, product price, etc.) of products identified by package recognition processing etc. from the product master. Also, payment processing refers to processing that displays the total price and tax amount of products related to one transaction for which products have been registered, calculates and displays the change based on the deposit received from the customer, and issues a receipt on which product information and payment information (total price, deposit amount, change amount, etc.) are printed.

[0028] The POS terminal 8 is also configured to be switchable between a clerk mode in which a clerk processes a transaction and a self-mode in which a customer processes the transaction himself. This switching may be performed, for example, by operating a switch provided on the POS terminal 8 for switching, or by receiving an operation on software. The clerk mode may include a first clerk mode in which a clerk processes both product data to register products and payment, and a second clerk mode in which a clerk processes product data to register products and the customer processes payment.

[0029] Fig. 3 is a block diagram showing an example of a hardware configuration of the POS terminal 8 in Fig. 1. As shown in Fig. 3, the POS terminal 8 has a control unit 81, a storage unit 82, a controller 83, a communication unit 84, an operation unit 85, a display unit 86, and an imaging unit 87. The control unit 81, the storage unit 82, the controller 83, and the communication unit 84 are communicatively connected via, for example, a bus 89. The operation unit 85, the display unit 86, and the imaging unit 87 are each communicatively connected to the controller 83.

[0030] The control unit 81 has a processor such as a CPU and memories such as a ROM and a RAM, and is configured similarly to a general computer. For example, the control unit 81 is configured to load various programs stored in a memory such as a ROM or the storage unit 82 into the RAM of the memory, and to realize each function of the POS terminal 8 described below by the processor executing the various loaded programs.

[0031] As the processor of the control unit 81 according to the embodiment, various processors such as a GPU or DSP, or dedicated arithmetic circuits realized by an ASIC or FPGA can be appropriately used instead of or in addition to a CPU.

[0032] The storage unit 82 is a storage device equipped with a non-volatile recording medium such as an HDD, an SSD, a flash memory, etc. The storage unit 82 stores various programs and data related to the operation of the POS terminal 8, device drivers for external devices connected to the POS terminal 8 via the controller 83, etc.

[0033] As an example, the storage unit 82 stores a control program 821, a product master 822, and threshold information 823. For example, the control program 821 includes at least one program for implementing each functional unit of the POS terminal 8. The product master 822 stores product information of each product in association with product identification information that identifies the product. Here, the product identification information is, for example, a product code that indicates the product, but it does not have to be a product code as long as it is information that identifies the product. The threshold information 823 will be described later.

[0034] The controller 83 is an input / output interface that connects an external device to the POS terminal 8. In the example of Fig. 3, an operation unit 85, a display unit 86, and an imaging unit 87 are connected to the controller 83.

[0035] The communication unit 84 is a communication interface for communicating with the outside of the POS terminal 8. The communication unit 84 includes either a communication circuit for wired communication or a communication circuit for wireless communication, and is connected to the communication line Na via the communication circuit.

[0036] The operation unit 85 is an input interface that accepts operation input from the user. The operation unit 85 is, for example, a touch panel that is arranged on the display surface (screen) of the display unit 86, detects a touch operation by the operator, and outputs a signal to the control unit 81. As an example, the operation unit 85 is configured to be able to accept operation input by the operator to an operation screen displayed on the display unit 86. Note that the operation unit 85 is not limited to a touch panel, and various input devices such as a keyboard, a switch, and a button may be provided as appropriate.

[0037] The display unit 86 is configured to be capable of displaying various types of screens generated under the control of the control unit 81. As the display unit 86, various display devices such as a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) display can be appropriately used.

[0038] The imaging unit 87 is a camera having an imaging element such as a CCD (Charge Coupled Device), an optical system that forms an image of light from the subject on the imaging surface of the imaging element, and an image generating unit that generates an image or video by capturing an image based on the light from the subject based on a pixel signal from the imaging element. The imaging unit 87 captures an image of a product to be registered that is held by a customer or a store clerk. The image captured by the imaging unit 87 is, for example, a still image or a video in units of frames.

[0039] The POS terminal 8 may be realized by executing a POS application (program) installed on a terminal device carried by a customer or a store clerk, such as a tablet PC (Personal Computer) or a smartphone. The POS terminal 8 may also be realized as a cart POS mounted on a cart used for shopping by customers or picking by store clerks.

[0040] Here, a functional configuration of the edge device 6 will be described with reference to the drawings. Fig. 4 is a block diagram showing an example of the functional configuration of the edge device 6 of Fig. 3.

[0041] The control unit 61 realizes the functions of a threshold setting unit 611 and a package recognition unit 612 as shown in FIG. 4 by the processor executing a control program 621 loaded in the memory.

[0042] The threshold setting unit 611 changes the recognition rate threshold in package recognition according to the operation mode set in the POS terminal 8. Specifically, the threshold setting unit 611 sets the recognition rate threshold in package recognition processing, which will be described later, based on information on the recognition rate threshold from the POS terminal 8. Here, in this embodiment, the recognition rate is the recognition accuracy of package recognition, and is adjusted by the similarity threshold. The recognition rate of package recognition indicates, for example, the rate at which the similarity calculated in package recognition exceeds the similarity threshold, or the ease of exceeding it. In other words, the recognition rate threshold may be a threshold correlated with the similarity, such as the rate or the threshold of the ease of exceeding it, or may be the similarity threshold itself. In this embodiment, a case where a similarity threshold is used as the recognition rate threshold is illustrated. Specifically, in this embodiment, the recognition rate threshold in the package recognition processing is a similarity threshold calculated in the package recognition processing for identifying a product included in an image obtained by capturing an image. Note that, in this embodiment, as an example, the information on the recognition rate threshold is information indicating the value of the recognition rate threshold.

[0043] For example, in the package recognition process, if there is one product whose calculated similarity is equal to or greater than the recognition rate threshold, the recognized product is identified as the product imaged by the imaging unit 87. The higher the similarity of the product, the more similar it is to the imaged product. (That is, the higher the probability that it is the imaged product.) For this reason, the higher the recognition rate threshold set by the threshold setting unit 611, the higher the recognition accuracy of the package recognition process, but it often takes more time to identify the product. Similarly, the lower the recognition rate threshold set by the threshold setting unit 611, the lower the recognition accuracy of the package recognition process and the more likely it is that erroneous registration will occur, but it often takes less time to identify the product.

[0044] The package recognition unit 612 receives an image from the POS terminal 8 and executes package recognition processing based on the image. The package recognition unit 612 also identifies a product by package recognition using a recognition rate threshold value changed by the threshold setting unit 611 according to the operation mode of the POS terminal 8. Specifically, the package recognition unit 612 receives an image captured by the imaging unit 87 of the POS terminal 8, and executes package recognition processing using an object detection model 622, a feature extraction model 623, and a matching dictionary 624 to identify one product included in the image from the POS terminal 8. The package recognition unit 612 also transmits product identification information (e.g., a product code) indicating the identified product to the POS terminal 8.

[0045] Here, the package recognition process will be described. The package recognition process is a known technique that uses a general object recognition technique to recognize a product based on the shape, appearance, texture, etc. of an image of the product and identify one product. In the package recognition process, the package recognition unit 612 uses three types of dictionaries, an object detection model 622, a feature extraction model 623, and a matching dictionary 624, to identify a product included in an image captured by the POS terminal 8.

[0046] First, the package recognition unit 612 extracts captured images showing a commodity from at least one series of captured images captured by the imaging unit 87 (see FIG. 3) of the POS terminal 8. Methods for extracting captured images showing a commodity from a series of captured images include, for example, a method for extracting images showing a commodity by distinguishing between the commodity and a hand or background, but also includes a method for extracting images showing the commodity before and after an image in which a symbol attached to an object (commodity) is recognized, a method for extracting images showing the commodity by going back from the image in which a symbol is recognized, and a method for extracting images showing the commodity after the image in which a symbol is recognized.

[0047] Next, the package recognition unit 612 uses the object detection model 622 to identify an area in which the product is captured in the captured image in which the product is captured. Here, the object detection model 622 is a dictionary for identifying an area occupied by the product in the image. That is, the object detection model 622 is a dictionary for identifying an area in which the product is captured from the captured image based on the input captured image. In other words, the object detection model 622 is a dictionary for extracting an image of the product (hereinafter, referred to as a product image) surrounded by the identified area from the captured image. Note that the object detection model 622 may be a machine learning model in which parameters are determined (learned) to output a product image or information indicating a product image in response to input of a series of captured images or captured images in which a product is captured.

[0048] Next, the package recognition unit 612 uses the feature extraction model 623 to extract a feature amount (feature vector) of the product that indicates surface information such as the external shape, color, pattern, and unevenness of the product shown in the product image. Here, the feature extraction model 623 is a dictionary for calculating feature amounts that are effective for distinguishing the product shown in the product image. That is, the feature extraction model 623 outputs a feature vector indicating feature amounts that indicate the features of the product whose area is specified based on the input product image. Note that the feature extraction model 623 may be a machine learning model whose parameters are determined (trained) so as to output a feature vector in response to the input of the product image. Also, the object detection model 622 and the feature extraction model 623 may be configured as one model.

[0049] Next, the package recognition unit 612 uses the comparison dictionary 624 to compare the extracted features with feature data of many products registered in the dictionary, and calculates a similarity indicating how similar the product shown in the product image is to the registered product. Here, the comparison dictionary 624 is a dictionary for identifying a product based on a feature vector of the product shown in the product image. In other words, the comparison dictionary 624 is a dictionary in which, for each of a plurality of products included in the product master, features indicating the characteristics of the product are associated with product identification information that identifies the product and registered.

[0050] Then, the package recognition unit 612 compares the calculated similarity with the recognition rate threshold set by the threshold setting unit 611, and identifies a product whose similarity is equal to or greater than the recognition rate threshold as the captured product.

[0051] Here, a functional configuration of the POS terminal 8 will be described with reference to the drawings. Fig. 5 is a block diagram showing an example of the functional configuration of the POS terminal 8 of Fig. 3.

[0052] The control unit 81 realizes the functions of a mode setting unit 811, a threshold designation unit 812, a reading control unit 813, and a product registration unit 814, as shown in FIG. 4, by the processor executing a control program 821 loaded into the memory.

[0053] The mode setting unit 811 sets the operation mode of the POS terminal 8. As an example, the mode setting unit 811 sets whether the operation mode is a clerk mode in which a clerk registers an item or a self-service mode in which a customer registers an item. Specifically, the mode setting unit 811 accepts an operation input from a clerk who sets the operation mode via the operation unit 85, and switches the operation mode of the POS terminal 8 between the clerk mode and the self-service mode.

[0054] The threshold designation unit 812 designates a recognition rate threshold according to the set operation mode of the POS terminal 8. As an example, the threshold designation unit 812 changes the recognition rate threshold in package recognition for identifying a commodity included in an image obtained by capturing, according to the set operation mode, by designating the recognition rate threshold according to the operation mode. For example, the threshold designation unit 812 changes the recognition rate threshold in package recognition so that the recognition rate threshold in the self mode is higher than the recognition rate threshold in the store clerk mode. Specifically, the threshold designation unit 812 refers to threshold information 823 stored in the storage unit 82, and reads out the recognition rate threshold according to the set operation mode of the POS terminal 8.

[0055] Fig. 6 is a diagram showing a schematic configuration of threshold information 823 according to an embodiment. The threshold information 823 is information showing the correspondence between an operation mode 8231 of the POS terminal and a recognition rate threshold 8232, as shown in Fig. 6, for example. In the example of Fig. 6, when the operation mode 8231 of the POS terminal is a "clerk mode", the recognition rate threshold 8232 is "TA", and when the operation mode 8231 of the POS terminal is a "self-service mode", the recognition rate threshold 8232 is "TB" which is higher than "TA". As an example, "TB", which is the recognition rate threshold 8232 for the "self-service mode", is a threshold for identifying a product as a commodity when the similarity calculated in the package recognition process is 90% or more.

[0056] The operation mode of the POS terminal 8 may be set to a semi-self-service mode (second clerk mode) in which the clerk processes the product data of the transaction and the customer processes the payment himself / herself. In this case, the second clerk mode may be set to the same recognition rate threshold as that of the clerk mode.

[0057] In the POS system 1 according to the embodiment, the threshold specification unit 812 of the POS terminal 8 may be referred to as a threshold setting unit. Alternatively, the threshold specification unit 611 of the edge device 6 and the threshold specification unit 812 of the POS terminal 8 may be collectively referred to as a threshold setting unit.

[0058] The reading control unit 813 controls the imaging unit 87 to obtain an image of the area (imaging range) over which the product to be read is held. The reading control unit 813 also transmits data of the obtained image to the edge device 6. The reading control unit 813 also controls the imaging unit 87 to decode a symbol such as a barcode or two-dimensional code attached to the product when the symbol is imaged, and generate decoded information. The decoded information includes product identification information that identifies the product to which the symbol is affixed.

[0059] In addition, a scanner for reading symbols via the controller 83 may be separately connected to the POS terminal 8, and the reading control unit 813 may control the scanner to read the symbols and decode the symbols to generate decoded information.

[0060] The product registration unit 814 registers a product identified by package recognition. Specifically, the product registration unit 814 registers a product based on either product identification information based on package recognition or product identification information based on symbol decoding, whichever is acquired first.

[0061] In addition, in the POS terminal 8, the control unit 81 performs payment processing for the products registered in the product data processing (registration processing) by the product registration unit 814, and transmits product information and payment information (i.e., sales information) of the products for which payment has been processed to the server 4.

[0062] Here, an example of the operation of the POS system 1 having the above configuration will be described with reference to the drawings. Fig. 7 is a sequence diagram showing an example of the process flow in the POS system 1 according to the embodiment. Note that the process flow in Fig. 7 is merely an example, and steps can be added, deleted, and the order can be changed as desired.

[0063] Based on the operation input by the store clerk, the POS terminal 8 sets the operation mode of the POS terminal 8 to either a store clerk mode in which the store clerk processes the transaction, or a self-service mode in which the customer processes the transaction himself (S1). The POS terminal 8 also reads out a recognition rate threshold corresponding to the set operation mode from the threshold information 823, and outputs it to the edge device 6 (S2). The edge device 6 then sets the recognition rate threshold from the POS terminal 8 as the similarity threshold calculated in the package recognition process (S3).

[0064] The threshold information 823 may be registered on the edge device 6 side. In this case, in the process of S2, mode information indicating in which mode the device operates may be provided from the POS terminal 8 to the edge device 6 as information related to the threshold of the recognition rate. Alternatively, after the threshold information 823 and the mode information are provided from the POS terminal 8 to the edge device 6 in the process of S2, the edge device 6 may select the threshold of the recognition rate.

[0065] The processes of S1 to S3 may be appropriately executed, for example, at the start or end of business hours, or when the congestion status of the store has changed or is expected to change. For example, when the store is busy, the operation mode of the POS terminal 8 is set to the clerk mode so that the clerk can process the transaction. For example, when the store is quiet, the operation mode of all or part of the POS terminal 8 is set to the self-service mode so that the customer can perform the transaction operation by himself.

[0066] After a recognition rate threshold is set in accordance with the selection of one of the operation modes, the POS terminal 8 captures an image using the imaging unit 87 (S4) and outputs the captured image to the edge device 6 (S5).

[0067] The edge device 6 that has received the image performs package recognition using the image (S6). At this time, the edge device 6 identifies the product included in the image based on a comparison between the similarity calculated based on the image and the similarity threshold set in the process of S3. Then, the edge device 6 outputs information on the product identified by package recognition, i.e., product identification information, to the POS terminal 8 (S7).

[0068] On the other hand, in the process of S5, the POS terminal 8 outputs the image to the edge device 6 and waits for the image of the symbol to be captured by the imaging unit 87. Then, when the image of the symbol is captured, the POS terminal 8 decodes the symbol and generates decoded information including the product identification information (S8).

[0069] Depending on the timing at which the symbol is imaged, the order of input of the product identification information by the processes of S5 to S7 and generation of the decoded information by S8 may change. If the store clerk or customer does not correctly face the symbol to the imaging unit 87, the image of the symbol may be captured late, and the generation of the decoded information may be delayed, which may result in the input of the product identification information from the edge device 6 being input first. Conversely, if the image of the symbol is captured early, the generation of the decoded information may be generated earlier than the input of the product identification information from the edge device 6.

[0070] Thereafter, the POS terminal 8 registers the product (S9) based on either the product identification information based on the package recognition (S5-S7) or the product identification information based on the symbol decode (S8), whichever is acquired first. Then, the flow of FIG. 7 ends, and the payment process and the like for the registered product are carried out.

[0071] In the POS system 1 according to the embodiment, the imaging unit 87 and part of the function related to package recognition of the POS terminal 8 may be provided as a device independent of the POS terminal 8. Alternatively, the imaging unit 87 may be attached to another device of the POS terminal 8.

[0072] In the POS system 1 according to the embodiment, the object detection model 622, the feature extraction model 623, and the matching dictionary 624 may be provided outside the edge device 6, such as in the server 4, or in a headquarters server installed in a headquarters for managing and controlling a plurality of servers 4.

[0073] In the POS system 1 according to the embodiment, the edge device 6 may be integrated into the server 4 or the POS terminal 8 and realized as a single device.

[0074] In this way, the POS terminal 8 according to the embodiment is configured to be switchable between a clerk mode in which a clerk registers a product and a self-service mode in which a customer registers a product. The POS terminal 8 according to the embodiment is also configured to change a similarity threshold for determining a product in package recognition, i.e., a recognition rate threshold, depending on the operation mode. Here, in the POS system 1 according to the embodiment, the recognition rate threshold for the self-service mode is higher than the recognition rate threshold for the clerk mode.

[0075] In general, an image used for package recognition can be acquired before or during the operation of the operator pointing the symbol toward the imaging unit 87. Therefore, by speeding up the identification of a product by package recognition, the throughput of transaction processing can be improved, and the convenience of both customers and store clerks can be improved. For example, even if an erroneous registration occurs when a store clerk operates the system, the store clerk can perform a deletion process to delete the erroneously registered product. Therefore, the speed of package recognition can be improved by setting the threshold value of the recognition rate of package recognition low, that is, the accuracy of image recognition low. However, if the accuracy of package recognition is reduced, the number of erroneous registrations may also increase when a customer operates the system, and therefore, for example, a store clerk needs to perform a deletion process from the viewpoint of preventing fraud, which causes a problem of an increase in processes involving store clerks. If the number of processes involving store clerks increases, the burden on store clerks increases, and the throughput of transaction processing decreases, which causes a problem of a decrease in the convenience of both customers and store clerks.

[0076] In this situation, the POS terminal 8 according to the embodiment can improve the recognition accuracy of package recognition (image recognition) in the POS terminal 8 operating in the self-service mode compared to when operating in the store clerk mode. Therefore, in the self-service mode, erroneous registration due to erroneous recognition in image recognition is suppressed, and processing involving store clerk intervention can be suppressed. Therefore, the POS terminal 8 according to the embodiment can improve the convenience of the POS terminal 8 that can be switched between the store clerk mode and the self-service mode.

[0077] (First Modification) In the above embodiment, the POS system 1 is exemplified in which the recognition rate threshold for package recognition is changed depending on the operation mode. In this case, the recognition rate threshold for package recognition may be further changed depending on the type of store clerk.

[0078] For example, the recognition rate threshold for package recognition may be different depending on whether or not the clerk who processes sales data using the POS terminal 8 has the cancellation authority to erase an erroneous registration. In this case, the recognition rate threshold for the clerk mode in which a clerk who has the cancellation authority processes sales data using the POS terminal 8 is smaller than the recognition rate threshold for the clerk mode in which a clerk who does not have the cancellation authority processes sales data using the POS terminal 8. However, the recognition rate threshold for the clerk mode in which a clerk who does not have the cancellation authority processes sales data using the POS terminal 8 is smaller than, for example, the recognition rate threshold for the self mode, but may be the same.

[0079] For example, threshold information 823 according to this modification is information indicating the correspondence between an operation mode 8231 of a POS terminal, clerk information indicating a clerk or a type of clerk, and a recognition rate threshold 8232. In other words, in threshold information 823 according to this modification, the recognition rate threshold for the clerk mode differs for each clerk or clerk information indicating a type of clerk. The clerk information indicating a clerk is, for example, information such as a clerk ID for uniquely identifying a clerk. The clerk information indicating a type of clerk is, for example, information indicating whether or not the clerk has cancellation authority.

[0080] For example, in the POS system 1 according to this modification, the POS terminal 8 reads out the recognition rate threshold value corresponding to the set operation mode and store clerk information from the threshold value information 823, and outputs it to the edge device 6 (S2).

[0081] The threshold information 823 according to this modification may be registered on the edge device 6 side. In this case, in the process of S2, as information related to the threshold of the recognition rate, in addition to mode information indicating in which mode the device will operate, store clerk information may be provided from the POS terminal 8 to the edge device 6. Alternatively, after the threshold information 823, mode information, and store clerk information are provided from the POS terminal 8 to the edge device 6 in the process of S2, the edge device 6 may select the threshold of the recognition rate.

[0082] According to this configuration, the recognition rate threshold can be set differently depending on whether the clerk has the cancellation authority. For example, if the clerk has the cancellation authority, the clerk can erase the erroneous registration by himself, so by setting the recognition rate threshold lower, the time required for package recognition can be shortened. For example, even if the clerk does not have the cancellation authority, if the clerk requests erasure between other clerks, the confirmation can be simplified compared to the erasure request from the customer in the self-mode, for example, from the viewpoint of preventing fraud, so by setting the recognition rate threshold lower than in the self-mode, the time required for package recognition can be shortened.

[0083] (Second Modification) Further, as the type of store clerk, the store clerk's skill level may be used instead of or in addition to the presence or absence of cancellation authority.

[0084] For example, in this modified example, the clerk information indicating the type of clerk is at least one of information indicating whether the clerk has cancellation authority and information indicating the proficiency of the clerk.

[0085] For example, the recognition rate threshold value for the clerk mode when a highly skilled, ie, experienced, clerk processes sales data using the POS terminal 8 is smaller than that for a less skilled clerk.

[0086] The proficiency of a store clerk may be information registered based on, for example, input by a store clerk in a store or input by, for example, a manager in a headquarters that manages and supervises multiple stores, or may be calculated by the POS system 1 based on history information of past package recognition, such as the similarity in past package recognition and the presence or absence or frequency of cancellation processing. For example, the higher the similarity or a statistical value such as the average or median of the similarity in past package recognition, or the smaller the number or frequency of cancellation processing, the higher the proficiency may be determined to be.

[0087] According to this configuration, the recognition rate threshold can be varied depending on the proficiency of the store clerk. For example, a store clerk with high proficiency can hold the product over the reading area so that an image suitable for package recognition is acquired, for example by adjusting the way the product is held or the position of the hand. Alternatively, for example, a store clerk with high proficiency is less likely to overlook erroneous registration. For these reasons, the more skilled the store clerk is, the less likely erroneous recognition will occur, and by setting the recognition rate threshold lower, the time required for package recognition can be shortened.

[0088] (Third Modification) The recognition rate threshold for package recognition may be changed according to the time of day instead of or in addition to the type of store clerk.

[0089] For example, the recognition rate threshold for package recognition may be different depending on a predetermined time period, in which case the recognition rate threshold for busy times, such as when the store is crowded or when there are few store staff, is higher than the recognition rate threshold for quiet times.

[0090] For example, threshold information 823 according to this modification is information indicating the correspondence between an operation mode 8231 of a POS terminal, store clerk information and / or time period information indicating a time period, and a recognition rate threshold 8232. In other words, in threshold information 823 according to this modification, the recognition rate threshold for the store clerk mode differs for each store clerk information and / or time period information.

[0091] For example, in the POS system 1 according to this modified example, the POS terminal 8 reads out a recognition rate threshold value corresponding to the set operation mode, store clerk information and / or time zone information from the threshold information 823, and outputs it to the edge device 6 (S2).

[0092] The threshold information 823 according to this modification may be registered on the edge device 6 side. In this case, in the process of S2, in addition to mode information indicating in which mode the device will operate, store clerk information and / or time zone information may be provided from the POS terminal 8 to the edge device 6 as information related to the recognition rate threshold. Alternatively, after the POS terminal 8 provides the edge device 6 with store clerk information and / or time zone information in addition to the threshold information 823 and mode information in the process of S2, the edge device 6 may select the recognition rate threshold.

[0093] Note that changing the recognition rate threshold for package recognition based on time zone information may be applied not only to the store clerk mode but also to the self-service mode.

[0094] The time zone information may be information registered based on input from, for example, a store clerk at the store or input from, for example, a manager at a headquarters that manages and supervises multiple stores, or may be calculated by the POS system 1 based on various information such as the history of past transaction processing and store clerk shift information.

[0095] According to this configuration, the recognition rate threshold can be changed depending on the congestion state of the store. For example, by setting the recognition rate threshold higher during busy hours or at night when there are fewer store staff, it is possible to reduce erroneous registrations caused by erroneous recognition in package recognition and reduce the occurrence of processing that requires the intervention of store staff.

[0096] According to at least one of the embodiments described above, it is possible to improve the convenience of a sales data processing device that is switchable between a store clerk mode and a self-service mode.

[0097] In the above-mentioned embodiment, "determining whether it is A" may mean "determining that it is A", or "determining that it is not A", or "determining whether it is A or not".

[0098] Each control program executed by each device in the POS system 1 of this embodiment is provided in a state that it is pre-installed in a recording medium such as a ROM.

[0099] Each control program executed by each device of the POS system 1 of this embodiment may be configured to be provided by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

[0100] Furthermore, each control program executed by each device of the POS system 1 of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, each control program executed by each device of the POS system 1 of the present embodiment may be provided or distributed via a network such as the Internet.

[0101] The control program 621 executed by the edge device 6 of this embodiment has a modular configuration including the above-mentioned units (threshold setting unit 611 and package recognition unit 612). The processor of the control unit 61 reads out the control program 621 from the recording medium and loads the above-mentioned units onto a main storage device such as a RAM of the control unit 61. As a result, the above-mentioned units are generated on the main storage device.

[0102] The control program 821 executed by the POS terminal 8 of this embodiment has a modular configuration including the above-mentioned units (mode setting unit 811, threshold designation unit 812, reading control unit 813, and product registration unit 814). The processor of the control unit 81 reads out the control program 821 from the recording medium and loads the above-mentioned units onto a main storage device such as a RAM of the control unit 81. As a result, the above-mentioned units are generated on the main storage device.

[0103] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims.

[0104] (Additional Note) (1) a mode setting unit that sets whether the device operates in a clerk mode in which a clerk registers products or a self-service mode in which a customer registers products; a threshold specifying unit that changes a recognition rate threshold for package recognition that identifies a product included in an image obtained by capturing an image in accordance with a set operation mode such that the recognition rate threshold in the self mode is higher than the recognition rate threshold in the store clerk mode; and a product registration unit that registers a product identified by the package recognition. Sales data processing device. (2) a storage unit that stores threshold information indicating a correspondence between the operation mode and a threshold value of the recognition rate, A sales data processing device as described in (1) above. (3) The recognition rate threshold value in the store clerk mode differs for each store clerk or store clerk information indicating a type of the store clerk. A sales data processing device according to (1) or (2) above. (4) The recognition rate threshold value differs depending on time period information indicating a time period. A sales data processing device according to any one of (1) to (3) above. (5) a mode setting unit for setting whether the sales data processing device is to operate in a clerk mode in which a clerk registers products or a self-service mode in which a customer registers products; a threshold setting unit that changes a threshold for a recognition rate in package recognition for identifying a product included in an image obtained by capturing the package according to the set operation mode; a package recognition unit that identifies a product by package recognition using the changed recognition rate threshold; A product registration unit that registers a product identified by the package recognition. Sales data processing system. (6) Set whether the device will operate in a clerk mode in which a clerk registers products or a self-service mode in which a customer registers products; changing a recognition rate threshold for package recognition for identifying a product included in an image obtained by capturing the image in accordance with the set operation mode such that the recognition rate threshold in the self mode is higher than the recognition rate threshold in the store clerk mode; registering the product identified by the package recognition; A program that makes a computer do something. (7) A recording medium (Computer Program Product) on which the program described in (6) above is recorded, the program being executed by a computer. [Explanation of symbols]

[0105] 1. POS System 4 Server 6 Edge Devices 61 Control section 611 Threshold setting unit 612 Package Recognition Unit 62 Storage section 621 Control Program 622 Object Detection Model 623 Feature Extraction Model 624 Matching Dictionary 63 Controller 64 Communications Department 69 Bus 8. POS terminals 81 Control Unit 811 Mode setting section 812 Threshold specification part 813 Reading control unit 814 Product Registration Department 82 Memory section 821 Control Program 822 Product Master 823 Threshold Information 8231 POS terminal operating mode 8232 Recognition rate threshold 83 Controller 84 Communications Department 85 Control section 86 Display section 87 Imaging unit 89 Bus [Prior art documents] [Patent documents]

[0106] [Patent Document 1] JP 2019-139778 A

Claims

1. a mode setting unit that sets whether the device operates in a clerk mode in which a clerk registers products or a self-service mode in which a customer registers products; a threshold specifying unit that changes a recognition rate threshold for package recognition that identifies a product included in an image obtained by capturing an image in accordance with a set operation mode such that the recognition rate threshold in the self mode is higher than the recognition rate threshold in the store clerk mode; and a product registration unit that registers a product identified by the package recognition. Sales data processing device.

2. a storage unit that stores threshold information indicating a correspondence between the operation mode and a threshold value of the recognition rate, The sales data processing device according to claim 1 .

3. The recognition rate threshold value in the store clerk mode differs for each store clerk or store clerk information indicating a type of the store clerk.

3. A sales data processing device according to claim 1 or 2.

4. The recognition rate threshold value differs depending on time period information indicating a time period.

3. A sales data processing device according to claim 1 or 2.

5. a mode setting unit for setting whether the sales data processing device is to operate in a clerk mode in which a clerk registers products or a self-service mode in which a customer registers products; a threshold setting unit that changes a threshold for a recognition rate in package recognition for identifying a product included in an image obtained by capturing the package according to the set operation mode; a package recognition unit that identifies a product by package recognition using the changed recognition rate threshold; A product registration unit that registers a product identified by the package recognition. Sales data processing system.

6. Set whether the device will operate in a clerk mode in which a clerk registers products or a self-service mode in which a customer registers products; changing a recognition rate threshold for package recognition for identifying a product included in an image obtained by capturing the image in accordance with the set operation mode such that the recognition rate threshold in the self mode is higher than the recognition rate threshold in the store clerk mode; registering the product identified by the package recognition; A program that makes a computer do something.

Images