Sales data processing device and program

Abstract

To improve convenience in a sales data processor that can switch between a clerk mode and a self-mode.SOLUTION: A sales data processor according to an embodiment includes a mode setting unit, a threshold designation unit, and an article registration unit. The mode setting unit sets an operation mode of either a clerk mode in which a clerk registers an article or a self-mode in which a customer registers the article. The threshold designation unit changes a threshold value of a recognition rate in package recognition to identify the article included in an image obtained by taking the image, according to the set operation mode, so that the threshold value of the recognition rate in the self-mode is higher than the threshold value of the recognition rate in the clerk mode. The article registration unit registers the article identified by the package recognition.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] Embodiments of the present invention relate to a sales data processing device and a program.

Background Art

[0002] Conventionally, in stores such as supermarkets, convenience stores, and discount stores, for the products purchased at the store, sales data processing devices such as POS (Point of Sales) terminals configured to be switchable between a clerk mode in which a clerk performs product data processing and settlement processing (hereinafter collectively referred to as "transaction processing") and a self mode in which the customer himself / herself performs the processing are known. For example, during peak hours at the store, this sales data processing device can be operated by switching the POS terminal to the clerk mode so that the clerk performs the transaction processing, and during slack hours, all or part of the POS terminals can be switched to the self mode so that the customer himself / herself performs the transaction operation.

[0003] In such a situation, when product data processing is performed by recognizing the package of the captured product image to determine the product and registering the product information of the product as the product related to the current transaction, for example, due to the accuracy of the image recognition, misregistration may occur. When misregistration occurs during operation in the self mode, clerk intervention processes such as the clerk erasing the misregistered product may occur. When clerk intervention processes occur, there is a risk that the burden on the clerk increases and the time required to complete the settlement lengthens, resulting in a decrease in convenience.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to improve the convenience of a sales data processing device that can switch between a clerk mode and a self mode.

Means for Solving the Problems

[0005] The sales data processing device according to the embodiment comprises a mode setting unit, a threshold specification unit, and a product registration unit. The mode setting unit sets whether to operate in an employee mode in which an employee registers products, or a self-service mode in which a customer registers products. The threshold specification unit changes the recognition rate threshold in package recognition, which identifies products included in the image obtained by capturing, according to the set operating mode, such that the recognition rate threshold in the self-service mode is higher than the recognition rate threshold in the employee mode. The product registration unit registers the products identified by package recognition. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 is a block diagram showing an example of the configuration of a POS (Point of Sales) system according to an embodiment. [Figure 2] Figure 2 is a block diagram showing an example of the hardware configuration of the edge device shown in Figure 1. [Figure 3] Figure 3 is a block diagram showing an example of the hardware configuration of the POS terminal shown in Figure 1. [Figure 4] Figure 4 is a block diagram showing an example of the functional configuration of the edge device shown in Figure 3. [Figure 5] Figure 5 is a block diagram showing an example of the functional configuration of the POS terminal shown in Figure 3. [Figure 6] Figure 6 is a schematic diagram showing the configuration of threshold information according to the embodiment. [Figure 7] Figure 7 is a sequence diagram showing an example of the processing flow in the system according to the embodiment. [Modes for carrying out the invention]

[0007] The following description of the sales data processing device, sales data processing system, program, and recording medium according to the embodiment will be given with reference to the drawings.

[0008] The embodiments described below are one embodiment of the sales data processing device, sales data processing system, program, and recording medium according to the embodiment, and do not limit their configuration, specifications, etc. The embodiments described below are examples of applications to a POS (Point of Sales) system introduced in stores such as supermarkets.

[0009] Figure 1 shows an example of the configuration of a POS system 1 according to an embodiment. As shown in Figure 1, the POS system 1 includes a server 4, an edge device 6, and a POS terminal 8.

[0010] Note that Figure 1 illustrates a POS system 1 that includes two or more POS terminals 8, but is not limited to this. In POS system 1, there may be one or two POS terminals 8, or three or more. That is, the POS system 1 according to this embodiment includes at least one POS terminal 8.

[0011] Here, the POS system 1 according to this embodiment is an example of a sales data processing system. Furthermore, the POS terminal 8 according to this embodiment is an example of a sales data processing device.

[0012] In POS system 1, the server 4, the edge device 6, and at least one POS terminal 8 are each connected to communicate via a communication line Na. The communication line Na can use the internet, VPN (virtual private network), LAN (local area network), public communication network, mobile communication network, etc., either alone or in appropriate combinations.

[0013] Server 4 is, for example, at least one server device installed in the back room of a store. Note that Server 4 is not limited to being installed in a store; it may also be configured as a cloud server installed in the cloud.

[0014] Server 4 is configured similarly to a typical computer, having a processor such as a CPU (Central Processing Unit) and memory such as ROM (Read Only Memory) and RAM (Random Access Memory). For example, Server 4 is configured to implement its various functions by loading various programs stored in memory such as ROM into the RAM, and having the processor execute the loaded programs. As an example, Server 4 manages the types and quantities of products sold in the store. As an example, Server 4 manages sales in the store based on sales information received from POS terminal 8. Specifically, Server 4 stores the sales information received from POS terminal 8 and manages the store's sales using the stored sales information.

[0015] The server 4 may be configured to use the image captured by the imaging unit 87 (see Figure 3) of the POS terminal 8 and the decoded information indicating the result of symbol decoding to perform machine learning to improve the accuracy of the matching dictionary 624 stored in the edge device 6, for example. Alternatively, the machine learning may be performed by the edge device 6.

[0016] Furthermore, the processor of Server 4 may be other than the CPU, or in addition to the CPU. Other processors that can be used include various types of processors such as GPUs (Graphics Processing Units) and DSPs (Digital Signal Processors), as well as dedicated arithmetic circuits implemented using ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays), as appropriate.

[0017] In addition to ROM and RAM, non-volatile recording media such as HDDs (Hard Disk Drives), SSDs (Solid State Drives), and Flash memory can be used as appropriate for the memory of Server 4.

[0018] The edge device 6 is a computer provided adjacent to, for example, the server 4 or the POS terminal 8, or provided within the server 4 or the POS terminal 8, or configured on the cloud.

[0019] FIG. 2 is a block diagram showing an example of the hardware configuration of the edge device 6 in FIG. 1. As shown in FIG. 2, the edge device 6 includes a control unit 61, a storage unit 62, a controller 63, and a communication unit 64. The control unit 61, the storage unit 62, the controller 63, and the communication unit 64 are communicably connected via, for example, a bus 69.

[0020] The control unit 61 includes, for example, a processor such as a CPU and a memory such as a ROM and a RAM, and is configured in the same manner as a general computer. For example, the control unit 61 loads various programs stored in a memory such as a ROM or the storage unit 62 into the RAM of the memory, and the processor executes the loaded various programs to realize each function of the edge device 6 described later.

[0021] In addition, as the processor of the control unit 61 according to the embodiment, various processors such as a GPU and a DSP, or a dedicated arithmetic circuit realized by an ASIC or an FPGA can be appropriately used instead of or in addition to the CPU.

[0022] The storage unit 62 is a storage device including a non-volatile recording medium such as an HDD, an SSD, or a Flash memory. The storage unit 62 stores various programs related to the operation of the edge device 6, various data, and a device driver of an external device connected to the edge device 6 via the controller 63.

[0023] As an example, the storage unit 62 stores a control program 621, an object detection model 622, a feature extraction model 623, and a verification dictionary 624. For example, the control program 621 includes at least one program for realizing each functional unit of the edge device 6. The object detection model 622, the feature extraction model 623, and the verification dictionary 624 will be described later.

[0024] The controller 63 is an input / output interface for connecting an external device to the edge device 6.

[0025] The communication unit 64 is a communication interface for communicating with the outside of the edge device 6. The communication unit 64 includes either a communication circuit for wired communication or a communication circuit for wireless communication, and is connected to the communication line Na by the communication circuit.

[0026] The POS terminal 8 is installed in stores such as supermarkets, convenience stores, and mass retailers. The POS terminal 8 is configured to be able to execute product data processing and payment processing (collectively referred to as "transaction processing") for products purchased at the store.

[0027] Here, the product data processing is a registration process of a product that reads and registers product information (product name, product price, etc.) of a product specified by package recognition processing or the like from a product master. Also, the payment processing is a process of displaying the total amount and tax amount of products related to a single transaction in which products are registered, a process of calculating and displaying change based on the deposit received from a customer, a process of issuing a receipt on which product information and payment information (total amount, deposit amount, change amount, etc.) are printed, and the like.

[0028] Also, the POS terminal 8 is configured to be switchable between a clerk mode in which a clerk performs transaction processing and a self mode in which a customer himself / herself performs transaction processing. This switching may be performed, for example, by operating a switch provided on the POS terminal 8 for switching, or may be performed by accepting an operation on software. Note that in the clerk mode, in addition to the first clerk mode in which a clerk performs both product data processing and payment processing for registering products, a second clerk mode in which a clerk performs product data processing for registering products and a customer himself / herself performs payment processing may be provided.

[0029] Figure 3 is a block diagram showing an example of the hardware configuration of the POS terminal 8 shown in Figure 1. As shown in Figure 3, the POS terminal 8 includes a control unit 81, a storage unit 82, a controller 83, a communication unit 84, an operation unit 85, a display unit 86, and an imaging unit 87. The control unit 81, storage unit 82, controller 83, and communication unit 84 are communicated together, for example, via a bus 89. The operation unit 85, display unit 86, and imaging unit 87 are each communicated together with the controller 83.

[0030] The control unit 81 has a processor such as a CPU and memory such as ROM and RAM, and is configured in the same way as a general computer. For example, the control unit 81 is configured to load various programs stored in memory such as ROM or in the storage unit 82 into the RAM of the memory, and the processor executes the loaded programs to realize the various functions of the POS terminal 8 described later.

[0031] In addition, as the processor of the control unit 81 according to this embodiment, various processors such as GPUs and DSPs, or dedicated arithmetic circuits implemented with ASICs and FPGAs, can be used as appropriate, instead of or in addition to a CPU.

[0032] The storage unit 82 is a storage device equipped with a non-volatile recording medium such as an HDD, SSD, or Flash memory. The storage unit 82 stores various programs and data related to the operation of the POS terminal 8, as well as device drivers for external devices connected to the POS terminal 8 via the controller 83.

[0033] As an example, the memory unit 82 stores a control program 821, a product master 822, and threshold information 823. For example, the control program 821 includes at least one program for implementing each function of the POS terminal 8. The product master 822 stores product information for each product, associating it with product identification information that identifies the product. Here, the product identification information is, for example, a product code that indicates a product, but it does not have to be a product code; any information that identifies a product is acceptable. The threshold information 823 will be described later.

[0034] The controller 83 is an input / output interface for connecting external devices to the POS terminal 8. In the example shown in Figure 3, the controller 83 is connected to the operation unit 85, the display unit 86, and the imaging unit 87.

[0035] The communication unit 84 is a communication interface for communicating with the outside world of the POS terminal 8. The communication unit 84 includes either a communication circuit for wired communication or a communication circuit for wireless communication, and connects to the communication line Na via the communication circuit.

[0036] The operation unit 85 is an input interface that receives user input. The operation unit 85 is, for example, a touch panel located on the display surface (screen) of the display unit 86, which detects the operator's touch operation and outputs a signal to the control unit 81. As an example, the operation unit 85 is configured to receive operator input for the operation screen displayed on the display unit 86. Note that the operation unit 85 is not limited to a touch panel; various input devices such as keyboards, switches, and buttons can be provided as appropriate.

[0037] The display unit 86 is configured to display various screens generated in accordance with the control of the control unit 81. Various display devices such as liquid crystal displays (LCDs) and organic electro-luminescence (EL) displays can be used as the display unit 86 as appropriate.

[0038] The imaging unit 87 is a camera comprising an image sensor, such as a CCD (Charge Coupled Device), an optical system that forms an image of light from the field of view onto the imaging surface of the image sensor, and an image generation unit that generates images or videos by capturing images based on the light from the field of view based on the pixel signals from the image sensor. The imaging unit 87 captures images of registered products held by a customer or store clerk. The images captured by the imaging unit 87 are, for example, still images or videos in frame units.

[0039] Furthermore, the POS terminal 8 may be implemented by running an installed POS application (program) on a terminal device carried by the customer or store clerk, such as a tablet PC (Personal Computer) or smartphone. Alternatively, the POS terminal 8 may be implemented as a cart POS mounted on a cart used for customer shopping or store clerk picking.

[0040] Here, the functional configuration of the edge device 6 will be explained with reference to the drawings. Figure 4 is a block diagram showing an example of the functional configuration of the edge device 6 in Figure 3.

[0041] The control unit 61 functions as a threshold setting unit 611 and a package recognition unit 612, as shown in Figure 4, by having the processor execute the control program 621 loaded into memory.

[0042] The threshold setting unit 611 changes the recognition rate threshold in package recognition according to the operating mode set in the POS terminal 8. Specifically, the threshold setting unit 611 sets the recognition rate threshold in the package recognition process described later based on the information regarding the recognition rate threshold from the POS terminal 8. Here, in this embodiment, the recognition rate is the recognition accuracy of package recognition and is adjusted by the similarity threshold. The recognition rate of package recognition indicates, for example, the percentage of the similarity calculated in package recognition that exceeds the similarity threshold, or how easily it exceeds the similarity threshold. In other words, the recognition rate threshold may be a threshold correlated with similarity, such as the percentage or the threshold of how easily it exceeds the similarity threshold, or it may be the similarity threshold itself. In this embodiment, an example is given of a case where the similarity threshold is used as the recognition rate threshold. Specifically, in this embodiment, the recognition rate threshold in the package recognition process is the similarity threshold calculated in the package recognition process that identifies the products contained in the image obtained by capturing. In this embodiment, as an example, the information regarding the recognition rate threshold is information indicating the value of the recognition rate threshold.

[0043] For example, in package recognition processing, if there is one product whose calculated similarity is equal to or greater than the recognition rate threshold, the recognized product is identified as the product captured by the imaging unit 87. Products with higher similarity are more similar to the captured product (i.e., have a higher probability of being the captured product). Therefore, the higher the recognition rate threshold set by the threshold setting unit 611, the higher the recognition accuracy of the package recognition processing, but it often takes longer to identify the product. Similarly, the lower the recognition rate threshold set by the threshold setting unit 611, the lower the recognition accuracy of the package recognition processing and the more likely incorrect registrations are to occur, but it often shortens the time required to identify the product.

[0044] The package recognition unit 612 receives an image from the POS terminal 8 and performs package recognition processing based on the image. The package recognition unit 612 also identifies products by package recognition using a recognition rate threshold that has been changed according to the operating mode of the POS terminal 8 by the threshold setting unit 611. Specifically, the package recognition unit 612 receives an image captured by the imaging unit 87 of the POS terminal 8, performs package recognition processing using the object detection model 622, the feature extraction model 623, and the matching dictionary 624, and identifies one product included in the image from the POS terminal 8. The package recognition unit 612 also transmits product identification information (e.g., product code) indicating the identified product to the POS terminal 8.

[0045] Here, we will explain the package recognition process. Package recognition is a known technique that uses general object recognition technology to recognize products based on their shape, appearance, texture, etc., and identify a single product. In the package recognition process, the package recognition unit 612 uses three types of dictionaries—an object detection model 622, a feature extraction model 623, and a matching dictionary 624—to identify products included in the image captured by the POS terminal 8.

[0046] First, the package recognition unit 612 extracts an image containing the product from at least one series of images captured by the imaging unit 87 (see Figure 3) of the POS terminal 8. Methods for extracting an image containing the product from a series of images include, for example, distinguishing the product from hands or the background and extracting an image containing the product. Other methods include extracting images of the product before and after an image in which a symbol attached to an object (product) has been recognized, extracting images of the product by working backward from an image in which a symbol has been recognized, and extracting images of the product after an image in which a symbol has been recognized.

[0047] Next, the package recognition unit 612 uses the object detection model 622 to identify the region in the captured image in which the product is located. Here, the object detection model 622 is a dictionary for identifying the region occupied by the product in an image. In other words, the object detection model 622 is a dictionary for identifying the region in the captured image in which the product is located, based on the input captured image. To put it another way, the object detection model 622 is a dictionary for extracting an image of the product (hereinafter referred to as the product image) enclosed by the identified region from the captured image. Note that the object detection model 622 may also be a machine learning model whose parameters have been determined (learned) to output a product image or information indicating a product image in response to a series of captured images or an image in which the product is located.

[0048] Next, the package recognition unit 612 uses the feature extraction model 623 to extract feature quantities (feature vectors) of the product that represent surface information such as the external shape, color, pattern, and unevenness of the product shown in the product image. Here, the feature extraction model 623 is a dictionary for calculating feature quantities that are effective in distinguishing products shown in the product image. That is, the feature extraction model 623 outputs a feature vector that represents feature quantities that represent the characteristics of the product whose region has been identified, based on the input product image. Note that the feature extraction model 623 may be a machine learning model whose parameters have been determined (learned) to output a feature vector in response to the input product image. Also, the object detection model 622 and the feature extraction model 623 may be configured as a single model.

[0049] Next, the package recognition unit 612 uses the matching dictionary 624 to compare the extracted features with the feature data of numerous products registered in the dictionary and calculates a similarity score indicating how similar the product shown in the product image is to the registered product. Here, the matching dictionary 624 is a dictionary for identifying a product based on the feature vector of the product shown in the product image. In other words, for example, the matching dictionary 624 is a dictionary in which feature quantities indicating the characteristics of a product are associated with each of the multiple products included in the product master, according to the product identification information that identifies the product.

[0050] The package recognition unit 612 then compares the calculated similarity with the recognition rate threshold set by the threshold setting unit 611, and identifies products whose similarity is equal to or greater than the recognition rate threshold as the products that have been imaged.

[0051] Here, the functional configuration of the POS terminal 8 will be explained with reference to the drawings. Figure 5 is a block diagram showing an example of the functional configuration of the POS terminal 8 in Figure 3.

[0052] The control unit 81 functions as a mode setting unit 811, a threshold specification unit 812, a read control unit 813, and a product registration unit 814, as shown in Figure 4, by having the processor execute the control program 821 loaded into memory.

[0053] The mode setting unit 811 sets the operating mode of the POS terminal 8. For example, the mode setting unit 811 sets whether the terminal operates in employee mode, where an employee registers products, or in self-service mode, where a customer registers products. Specifically, the mode setting unit 811 receives input from an employee setting the operating mode via the operation unit 85, and switches the operating mode of the POS terminal 8 between employee mode and self-service mode.

[0054] The threshold specification unit 812 specifies a recognition rate threshold according to the configured operating mode of the POS terminal 8. For example, by specifying a recognition rate threshold according to the operating mode, the threshold specification unit 812 changes the recognition rate threshold in package recognition, which identifies products included in the image obtained by capturing, according to the configured operating mode. For example, the threshold specification unit 812 changes the recognition rate threshold in package recognition so that the recognition rate threshold in self-service mode is higher than the recognition rate threshold in employee mode. Specifically, the threshold specification unit 812 refers to the threshold information 823 stored in the storage unit 82 and reads the recognition rate threshold according to the configured operating mode of the POS terminal 8.

[0055] Figure 6 is a schematic diagram showing the configuration of threshold information 823 according to the embodiment. Threshold information 823 is information that indicates the correspondence between the operating mode 8231 of the POS terminal and the recognition rate threshold 8232, as shown in Figure 6, for example. In the example in Figure 6, when the operating mode 8231 of the POS terminal is "clerk mode", the recognition rate threshold 8232 is "TA", and when the operating mode 8231 of the POS terminal is "self mode", the recognition rate threshold 8232 is "TB", which is higher than "TA". As an example, "TB", which is the recognition rate threshold 8232 for "self mode", is the threshold for identifying a product when the similarity calculated in the package recognition process is 90% or higher.

[0056] Furthermore, the POS terminal 8 may be configured in a semi-self-service mode (second staff mode) in which the staff member processes the product data during the transaction, while the customer handles the payment process. In this case, the second staff mode may be configured with the same recognition rate threshold as the staff mode.

[0057] In relation to the POS system 1 according to this embodiment, the threshold specification unit 812 of the POS terminal 8 may be referred to as the threshold setting unit. Alternatively, the threshold setting unit 611 of the edge device 6 and the threshold specification unit 812 of the POS terminal 8 may be collectively referred to as the threshold setting unit.

[0058] The reading control unit 813 controls the imaging unit 87 to acquire an image of the area (imaging range) over which the product to be read is held. The reading control unit 813 also transmits the acquired image data to the edge device 6. Furthermore, if a symbol such as a barcode or two-dimensional code attached to the product is captured, the reading control unit 813 decodes the symbol and generates decoded information. The decoded information includes product identification information that identifies the product to which the symbol is attached.

[0059] Furthermore, the POS terminal 8 may also have a separate scanner connected to it for reading symbols, for example via a controller 83, and the reading control unit 813 may read the symbols by controlling the scanner, decode the symbols, and generate decoded information.

[0060] The product registration unit 814 registers the product identified by package recognition. Specifically, the product registration unit 814 registers the product based on the product identification information obtained earlier, whichever is obtained first: product identification information based on package recognition or product identification information based on symbol decoding.

[0061] Furthermore, in the POS terminal 8, the control unit 81 performs payment processing for products registered by the product registration unit 814 (product data processing / registration processing), and transmits the product information and payment information (i.e., sales information) of the processed products to the server 4.

[0062] Here, with reference to the drawings, an example of the operation of the POS system 1 with the above configuration will be described. Figure 7 is a sequence diagram showing an example of the processing flow in the POS system 1 according to the embodiment. Note that the processing flow in Figure 7 is merely an example, and steps can be added, deleted, and their order changed as desired.

[0063] Based on the store clerk's input, the POS terminal 8 sets its operating mode to either a store clerk mode where the store clerk processes the transaction, or a self-service mode where the customer processes the transaction themselves (S1). The POS terminal 8 also reads the recognition rate threshold corresponding to the set operating mode from the threshold information 823 and outputs it to the edge device 6 (S2). The edge device 6 then sets the recognition rate threshold from the POS terminal 8 as the similarity threshold calculated in the package recognition process (S3).

[0064] The threshold information 823 may be registered on the edge device 6. In this case, during processing S2, mode information indicating which mode to operate in should be provided from the POS terminal 8 to the edge device 6 as information regarding the recognition rate threshold. Alternatively, after the threshold information 823 and mode information are provided from the POS terminal 8 to the edge device 6 during processing S2, the recognition rate threshold may be selected at the edge device 6.

[0065] The processes S1 to S3 may be executed as needed, for example, at the start or end of business hours, or when the store's congestion changes or is expected to change. For example, during busy periods, the operating mode of the POS terminal 8 is set to employee mode so that store employees can process transactions. For example, during quiet periods, the operating mode of all or some of the POS terminals 8 is set to self-service mode so that customers can perform transactions themselves.

[0066] After a recognition rate threshold is set in accordance with the selection of one of the operating modes, the POS terminal 8 takes an image using the imaging unit 87 (S4), and outputs the image obtained from the imaging to the edge device 6 (S5).

[0067] The edge device 6, upon receiving the image, performs package recognition using the image (S6). At this time, the edge device 6 identifies the products included in the image based on a comparison between the similarity calculated from the image and the similarity threshold set in the processing in S3. The edge device 6 then outputs the product information identified by package recognition, i.e., product identification information, to the POS terminal 8 (S7).

[0068] Meanwhile, in processing S5, the POS terminal 8 outputs an image to the edge device 6 and waits for the imaging unit 87 to capture a symbol. When a symbol is captured, the POS terminal 8 decodes the symbol and generates decoded information including product identification information (S8).

[0069] Note that the order of inputting product identification information via processing S5-S7 and generating decoded information via S8 may vary depending on when the symbol is captured. If the store clerk or customer does not orient the symbol correctly towards the imaging unit 87, the image capture of the symbol may be delayed, and if the generation of decoded information is delayed, the input of product identification information from the edge device 6 may occur first. Conversely, if the image capture of the symbol is performed early, the generation of decoded information may occur earlier than the input of product identification information from the edge device 6.

[0070] Subsequently, the POS terminal 8 registers the product (S9) based on the product identification information obtained earlier, whichever is first: the product identification information based on package recognition (S5-S7) or the product identification information based on symbol decoding (S8). After that, the process shown in Figure 7 is completed, and payment processing and other operations related to the registered product are carried out.

[0071] In the POS system 1 according to this embodiment, some of the functions related to package recognition of the imaging unit 87 and the POS terminal 8 may be provided as devices independent of the POS terminal 8. Alternatively, the imaging unit 87 may be attached to other devices of the POS terminal 8.

[0072] In the POS system 1 according to this embodiment, the object detection model 622, the feature extraction model 623, and the matching dictionary 624 may be located outside the edge device 6, for example, on server 4, or on a headquarters server located at headquarters that manages and coordinates multiple servers 4.

[0073] In the POS system 1 according to this embodiment, the edge device 6 may be integrated with the server 4 or the POS terminal 8 and implemented as a single device.

[0074] Thus, the POS terminal 8 according to this embodiment is configured to switch between a staff mode in which a staff member registers items and a self-service mode in which a customer registers items. Furthermore, the POS terminal 8 according to this embodiment is configured to change the similarity threshold for product identification in package recognition, i.e., the recognition rate threshold, depending on the operating mode. Here, in the POS system 1 according to this embodiment, the recognition rate threshold in self-service mode is higher than the recognition rate threshold in staff mode.

[0075] Generally, images used for package recognition can be acquired before or during the operator's action of pointing the symbol towards the imaging unit 87. Therefore, by accelerating the identification of products through package recognition, the throughput of transaction processing can be improved, and convenience for both customers and store staff can be enhanced. For example, even if an incorrect registration occurs when a store staff member is operating the system, the staff member can perform a clearing process to delete the incorrectly registered product. Therefore, by setting a low threshold for the recognition rate of package recognition, i.e., a low accuracy for image recognition, the speed of package recognition can be improved. However, if the accuracy of package recognition is reduced, incorrect registrations may also increase when customers are operating the system. For example, from the perspective of preventing fraud, store staff members may need to perform clearing processes, which increases the amount of processing that requires staff intervention. An increase in processing that requires staff intervention increases the burden on staff and also reduces the throughput of transaction processing, thus reducing convenience for both customers and store staff.

[0076] In this context, the POS terminal 8 according to the embodiment can improve the recognition accuracy of package recognition (image recognition) in the POS terminal 8 operating in self-service mode compared to when it operates in employee mode. Therefore, in self-service mode, erroneous registrations due to misrecognition in image recognition are suppressed, and processing involving an employee can be reduced. Accordingly, the POS terminal 8 according to the embodiment can improve the convenience of a POS terminal 8 that can switch between employee mode and self-service mode.

[0077] (First variation) In the above-described embodiment, a POS system 1 was illustrated that changes the recognition rate threshold for package recognition according to the operating mode. In this case, the recognition rate threshold for package recognition may be further changed according to the type of store employee.

[0078] For example, the recognition rate threshold for package recognition can be set differently depending on whether the employee processing sales data using the POS terminal 8 is an employee with the authority to cancel incorrect entries. In this case, the recognition rate threshold for employee mode, where an employee with cancellation authority processes sales data using the POS terminal 8, is smaller than the recognition rate threshold for employee mode, where an employee without cancellation authority processes sales data using the POS terminal 8. However, the recognition rate threshold for employee mode, where an employee without cancellation authority processes sales data using the POS terminal 8, is smaller than, for example, the recognition rate threshold for self-mode, but can also be the same.

[0079] For example, the threshold information 823 in this modified example is information that shows the correspondence between the operating mode 8231 of the POS terminal, employee information indicating the type of employee, and the recognition rate threshold 8232. In other words, in the threshold information 823 in this modified example, the recognition rate threshold for the employee mode differs for each employee or employee information indicating the type of employee. Employee information indicating an employee is, for example, information such as an employee ID to uniquely identify an employee. Employee information indicating the type of employee is, for example, information indicating whether or not the employee has the authority to cancel.

[0080] For example, in the POS system 1 according to this modified example, the POS terminal 8 reads the recognition rate threshold corresponding to the set operating mode and store clerk information from the threshold information 823 and outputs it to the edge device 6 (S2).

[0081] The threshold information 823 related to this modified example may be registered on the edge device 6. In this case, during processing S2, the POS terminal 8 provides the edge device 6 with information regarding the recognition rate threshold, including mode information indicating which mode to operate in, as well as store clerk information. Alternatively, during processing S2, the threshold information 823, mode information, and store clerk information may be provided from the POS terminal 8 to the edge device 6, after which the recognition rate threshold may be selected by the edge device 6.

[0082] This configuration allows for different recognition thresholds depending on whether the employee has the authority to cancel entries. For example, if an employee has the authority to cancel entries, they can perform the process of correcting incorrect entries themselves, so setting a lower recognition threshold can shorten the time required for package recognition. For example, even if an employee does not have the authority to cancel entries, if the correction request is made between employees, the verification process can be simpler compared to a customer's correction request in self-service mode, for example, from the perspective of fraud prevention. Therefore, setting a lower recognition threshold than in self-service mode can shorten the time required for package recognition.

[0083] (Second variation) Furthermore, instead of, or in addition to, whether or not the employee has the authority to cancel, the employee's level of proficiency may be used as a classification of the employee.

[0084] For example, in this modified example, the employee information indicating the type of employee is at least one of the following: information indicating whether or not the employee has the authority to cancel an order, and information indicating the employee's level of proficiency.

[0085] For example, the recognition rate threshold for the employee mode is lower when a highly skilled employee processes sales data using the POS terminal 8 than when a less skilled employee does.

[0086] The proficiency level of store staff may be registered based on information such as input by store staff at the store or input by managers at headquarters that manage and oversee multiple stores, or it may be calculated by the POS system 1 based on historical information of past package recognition, such as the similarity in past package recognition or the presence or frequency of reconciliation processing. For example, the higher the statistical value such as the similarity, mean, or median of similarity in past package recognition, or the fewer the number or frequency of reconciliation processing, the higher the proficiency level may be judged to be.

[0087] This configuration allows for different recognition thresholds depending on the clerk's skill level. For example, a highly skilled clerk can adjust the way they hold the product and the position of their hands to ensure that an image suitable for package recognition is obtained when scanning the product in the reading area. Alternatively, a highly skilled clerk is less likely to miss a misrecognition. Therefore, the higher the clerk's skill level, the less likely misrecognitions are to occur, and by setting a lower recognition threshold, the time required for package recognition can be reduced.

[0088] (Third variation) Furthermore, the recognition rate threshold for package recognition may be changed depending on the time of day, either in addition to or instead of the type of store employee.

[0089] For example, the recognition rate threshold for package recognition can be set differently depending on a predetermined time period. In this case, the recognition rate threshold during busy periods, such as when a store is crowded or when there are fewer staff members, will be higher than the recognition rate threshold during off-peak periods.

[0090] For example, the threshold information 823 in this modified example is information that shows the correspondence between the operating mode 8231 of the POS terminal, time information indicating store clerk information and / or time period, and the recognition rate threshold 8232. In other words, in the threshold information 823 in this modified example, the recognition rate threshold for the store clerk mode differs for each piece of store clerk information and / or time period information.

[0091] For example, in the POS system 1 according to this modified example, the POS terminal 8 reads a recognition rate threshold corresponding to the set operating mode and store clerk information and / or time information from threshold information 823 and outputs it to the edge device 6 (S2).

[0092] The threshold information 823 related to this modified example may be registered on the edge device 6. In this case, during processing S2, in addition to mode information indicating which mode to operate in, store clerk information and / or time zone information should be provided from the POS terminal 8 to the edge device 6 as information regarding the recognition rate threshold. Alternatively, during processing S2, in addition to the threshold information 823 and mode information, store clerk information and / or time zone information may be provided from the POS terminal 8 to the edge device 6, after which the recognition rate threshold may be selected at the edge device 6.

[0093] Furthermore, the change in the recognition rate threshold for package recognition based on time-of-day information may be applied not only to the staff mode but also to the self-service mode.

[0094] The time-of-day information may be registered based on input from, for example, store employees at a store or from, for example, a manager at headquarters that manages and oversees multiple stores, or it may be calculated by the POS system 1 based on various information such as past transaction history and employee shift information.

[0095] This configuration allows for different recognition thresholds depending on the store's congestion level. For example, by setting a higher recognition threshold during busy hours or at night when there are fewer staff members, it is possible to suppress misregistrations caused by misrecognition in package recognition and reduce the need for staff intervention.

[0096] According to at least one embodiment described above, the convenience of a sales data processing device that can switch between a store clerk mode and a self-service mode can be improved.

[0097] In the embodiments described above, "determining whether it is A" may mean "determining that it is A," "determining that it is not A," or "determining whether or not it is A."

[0098] Furthermore, each control program executed by each device of the POS system 1 in this embodiment is provided pre-loaded onto a recording medium such as ROM.

[0099] Each control program executed by each device of the POS system 1 in this embodiment may be configured to be provided as an installable or executable file recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0100] Furthermore, the control programs executed by each device of the POS system 1 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the control programs executed by each device of the POS system 1 of this embodiment may be provided or distributed via a network such as the Internet.

[0101] The control program 621 executed by the edge device 6 in this embodiment has a modular configuration including the above-described parts (threshold setting unit 611 and package recognition unit 612). The processor of the control unit 61 reads the control program 621 from the recording medium and loads the above-described parts onto the main memory of the control unit 61, such as RAM. As a result, the above-described parts are generated on the main memory.

[0102] The control program 821 executed in the POS terminal 8 of this embodiment has a modular configuration that includes the above-mentioned parts (mode setting unit 811, threshold specification unit 812, read control unit 813, and product registration unit 814). The processor of the control unit 81 reads the control program 821 from the recording medium and loads the above-mentioned parts onto the main memory of the control unit 81, such as RAM. As a result, the above-mentioned parts are generated on the main memory.

[0103] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0104] (Note) (1) A mode setting unit that sets whether the device operates in a staff mode where staff register products or a self-service mode where customers register products, A threshold specification unit that changes the recognition rate threshold for package recognition, which identifies products included in the image obtained by capturing, according to the set operating mode, such that the recognition rate threshold for the self-mode is higher than the recognition rate threshold for the store clerk mode, The system includes a product registration unit that registers the product identified by the package recognition. Sales data processing device. (2) The system further includes a storage unit that stores threshold information indicating the correspondence between the aforementioned operating mode and the aforementioned recognition rate threshold. The sales data processing device described in (1) above. (3) The threshold for the recognition rate in the aforementioned employee mode differs for each employee or employee information indicating the type of employee. The sales data processing device described in (1) or (2) above. (4) The aforementioned recognition rate threshold differs for each time period information indicating a time period. A sales data processing device as described in any one of the above items (1) to (3). (5) A mode setting unit that sets whether the sales data processing device operates in a store clerk mode where a store clerk registers the products, or a self-service mode where a customer registers the products, A threshold setting unit that changes the recognition rate threshold in package recognition for identifying products included in the image obtained by capturing, according to the set operating mode, A package recognition unit identifies a product by package recognition using the modified recognition rate threshold, A product registration unit that registers the product identified by the package recognition, Sales data processing system. (6) This setting determines whether the system operates in "staff mode," where staff register products, or "self-service mode," where customers register products. The recognition rate threshold for package recognition, which identifies products included in the image obtained by capturing, is changed according to the set operating mode such that the recognition rate threshold in the self-mode is higher than the recognition rate threshold in the store clerk mode. Register the product identified by the aforementioned package recognition. A program that causes a computer to perform a task. (7) A computer program that is executed by a computer, and on which the program described in (6) above is recorded (Computer Program Product). [Explanation of symbols]

[0105] 1 POS system 4 servers 6 Edge devices 61 Control Unit 611 Threshold setting section 612 Package Recognition Unit 62 Storage section 621 Control Program 622 Object Detection Models 623 Feature Extraction Models 624 Dictionary for comparison 63 Controllers 64 Communications Department 69 bus 8 POS terminals 81 Control Unit 811 Mode setting section 812 Threshold specification section 813 Reading Control Unit 814 Product Registration Department 82 Memory section 821 Control Program 822 Product Master 823 Threshold Information 8231 POS terminal operating modes 8232 Recognition rate threshold 83 Controllers 84 Communications Department 85 Control section 86 Display section 87 Imaging Unit 89 Bus [Prior art documents] [Patent Documents]

[0106] [Patent Document 1] Japanese Patent Publication No. 2019-139778

Claims

1. A mode setting unit that sets whether the device operates in a staff mode where staff register products or a self-service mode where customers register products, A threshold specification unit that changes the recognition rate threshold for package recognition, which identifies products included in the image obtained by capturing, according to the set operating mode, such that the recognition rate threshold for the self-mode is higher than the recognition rate threshold for the store clerk mode, The system includes a product registration unit that registers the product identified by the package recognition. Sales data processing device.

2. The system further includes a storage unit that stores threshold information indicating the correspondence between the aforementioned operating mode and the aforementioned recognition rate threshold. The sales data processing device according to claim 1.

3. The threshold for the recognition rate in the aforementioned employee mode differs for each employee or employee information indicating the type of employee. The sales data processing device according to claim 1 or claim 2.

4. The aforementioned recognition rate threshold differs for each time period information indicating a time period. The sales data processing device according to claim 1 or claim 2.

5. A mode setting unit that sets whether the sales data processing device operates in a store clerk mode where a store clerk registers the products, or a self-service mode where a customer registers the products, A threshold setting unit that changes the recognition rate threshold in package recognition for identifying products included in the image obtained by capturing, according to the set operating mode, A package recognition unit identifies a product by package recognition using the modified recognition rate threshold, A product registration unit that registers the product identified by the package recognition, Sales data processing system.

6. This setting determines whether the system operates in "staff mode," where staff register products, or "self-service mode," where customers register products. The recognition rate threshold for package recognition, which identifies products included in the image obtained by capturing, is changed according to the set operating mode such that the recognition rate threshold in the self-mode is higher than the recognition rate threshold in the store clerk mode. Register the product identified by the aforementioned package recognition. A program that causes a computer to perform a task.

Images