Wearable devices and programs

Abstract

We provide a wearable device and program that enable efficient processing of sales data. [Solution] The wearable device of the embodiment is a wearable device for use in a store that can be worn in front of the operator's eyes, and comprises: acquisition means for acquiring an image captured by an imaging unit that images the operator's field of view; first detection means for detecting the operator's movements; second detection means for detecting code symbols included in the image; reading means for reading information from the code symbols detected by the second detection means in accordance with the operator's movements detected by the first detection means; and processing means for performing sales data processing related to commercial transactions in the store based on the information read by the reading means.

Description

[Technical Field]

[0001] Embodiments of the present invention relate to a wearable device and a program. [Background Art]

[0002] In stores such as mass retailers, sales data processing such as product registration processing and checkout processing is performed using a sales data processing device such as a POS (Point Of Sales) terminal. For example, product registration processing is also performed using terminal devices such as smartphones and tablet terminals.

[0003] Recently, glasses-type and goggle-type wearable devices using AR (Augmented Reality) technology and MR (Mixed Reality) technology have appeared. It is expected that sales data processing will be performed using such wearable devices in the near future.

[0004] However, no practical consideration has been given to performing sales data processing using wearable devices. [Summary of Invention] [Problem to be Solved by the Invention]

[0005] The problem to be solved by the present invention is to provide a wearable device and a program capable of efficiently performing sales data processing. [Means for Solving the Problem]

[0006] The wearable device of the embodiment is a wearable device for use in a store that can be worn in front of the operator's eyes, and comprises: an acquisition means for acquiring an image captured by an imaging unit that images the operator's field of view; a first detection means for detecting the operator's movements; a second detection means for detecting code symbols included in the image; a reading means for reading information from the code symbols detected by the second detection means in response to the operator's movements detected by the first detection means; and a processing means for performing sales data processing related to commercial transactions in the store based on the information read by the reading means. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows an example of a sales data processing system according to an embodiment. [Figure 2] Figure 2 shows an example of the hardware configuration of a wearable device according to the embodiment. [Figure 3] Figure 3 is a schematic diagram showing an example of the data structure of a product master according to the embodiment. [Figure 4] Figure 4 shows an example of the hardware configuration of a server device according to this embodiment. [Figure 5] Figure 5 shows an example of the data structure of a customer management table according to the embodiment. [Figure 6] Figure 6 shows an example of the functional configuration of a wearable device and server device according to an embodiment. [Figure 7] Figure 7 shows an example of an image displayed on the display unit of a wearable device according to the embodiment. [Figure 8] Figure 8 is a flowchart showing an example of the processing performed by the wearable device of the embodiment. [Modes for carrying out the invention]

[0008] The wearable device and program according to the embodiment will be described below with reference to the drawings. In the following embodiment, a form used in stores such as mass retailers will be described, but the invention is not limited to this form.

[0009] Figure 1 shows an example of a sales data processing system according to an embodiment. As shown in Figure 1, the sales data processing system 1 includes a wearable device 10 and a server device 20. The wearable device 10 and the server device 20 are connected to each other via a network N.

[0010] Network N can utilize the internet, VPN (virtual private network), LAN (local area network), public communication network, mobile communication network, etc., individually or in appropriate combinations. In this embodiment, a LAN installed within a facility is used as an example of network N. The number of wearable devices 10 in the sales data processing system 1 is not limited to the illustrated example.

[0011] Wearable device 10 is an example of a wearable device that can be worn in front of the operator's eyes. Wearable device 10 is a glasses-type or goggle-type device that uses AR technology, MR technology, etc., and is used while being worn in front of the operator's eyes, as shown in Figure 1.

[0012] The wearable device 10 displays various information and images in front of the operator's eyes. Furthermore, the wearable device 10 can detect the operator's movements via the imaging unit 18 and sensor unit 19 (described later), and accept the detected movements as commands to the device.

[0013] Furthermore, the wearable device 10 is connected by wire or wirelessly to peripheral devices such as a printer 31 and a credit terminal 32 for credit card payments. The peripheral devices to be connected are not limited to those shown in the illustration. For example, an automated payment machine for cash payments may be connected.

[0014] In addition, the area in front of the operator's eyes in the wearable device 10 may be covered by the housing of the wearable device 10, or it may be formed of a light-transmitting resin or glass that allows the outside to be seen. In this embodiment, a configuration in which the area in front of the operator's eyes is covered by the housing of the wearable device 10 will be described.

[0015] The wearable device 10 performs sales data processing related to commercial transactions in response to operations from the operator. Sales data processing refers to both or either the registration process, which registers the products that customers purchase from among the products sold in the store (also called product registration), and the settlement process, which settles the payment for the registered products (also called settlement). Therefore, the wearable device 10 can also be referred to as a sales data processing device.

[0016] In this embodiment, the operator of the wearable device 10 is assumed to be a store employee, but it is not limited to this and may be a customer. When a customer is the operator, the wearable device 10 can also be described as a self-service type sales data processing device.

[0017] Server device 20 is a server device such as a store server. Server device 20 exchanges various data with wearable device 10 via network N.

[0018] Although Figure 1 shows the server device 20 as a single device, in practice, the server device 20 may be realized through the cooperation of multiple devices. Furthermore, the server device 20 may be a cloud server (or cloud system) realized by multiple network-connected devices.

[0019] The following describes an example configuration of the wearable device 10 and server device 20 mentioned above.

[0020] FIG. 2 is a diagram illustrating an example of the hardware configuration of the wearable device 10 according to the embodiment. As shown in FIG. 2, the wearable device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a communication unit 14, and a storage unit 16.

[0021] The CPU 11 is an example of a processor, and comprehensively controls each part of the wearable device 10. The ROM 12 stores various programs. The RAM 13 is a work space for expanding programs and various data. Further, the RAM 13 functions as a storage area for holding products registered in a registration process described later.

[0022] The CPU 11, the ROM 12, and the RAM 13 are connected via a bus or the like, and constitute a control unit 100 having a computer configuration. In the control unit 100, the CPU 11 operates according to a program stored in the ROM 12 or the storage unit 16 and expanded in the RAM 13, thereby executing various processes.

[0023] The communication unit 14 is a wired or wireless communication interface connectable to a network N. The communication unit 14 communicates with an external device such as a server device 20 via the network N.

[0024] The short-range communication unit 15 is a communication interface capable of performing short-range wireless communication such as Bluetooth (registered trademark). The short-range communication unit 15 is communicatively connected to peripheral devices such as a printer 31 and a credit terminal 32 (see FIG. 1), for example.

[0025] The storage unit 16 includes a storage medium such as an SSD (Solid State Drive) or a flash memory, and retains stored content even when power is cut off. The storage unit 16 stores a program 161 executable by the CPU 11 and various types of setting information. The program 161 includes a program (for example, an application) for realizing a functional configuration described later.

[0026] Furthermore, the storage unit 16 stores the product master 162. The product master 162 is a table data or database that stores information about the products handled by the store.

[0027] Figure 3 is a schematic diagram showing an example of the data structure of the product master 162 according to the embodiment. As shown in Figure 3, the product master 162 associates the type of product with an identifiable product code and stores product information for the product corresponding to that product code. Product information includes, for example, the product name and price (unit price).

[0028] In this embodiment, the wearable device 10 is configured to refer to the product master 162 stored in the memory unit 16, but it is not limited to this configuration, and it may also be configured to refer to the product master 162 stored in an external device such as a server device 20.

[0029] The wearable device 10 also includes a display unit 17, an imaging unit 18, a sensor unit 19, and the like.

[0030] The display unit 17, under the control of the control unit 100, displays various information and images in front of the eyes of the operator wearing the wearable device 10. The display method of the display unit 17 is not particularly limited. For example, the display unit 17 may be an LCD (Liquid Crystal Display) or other type of display. For example, the display unit 17 may be a transmissive display. For example, the display unit 17 may be a retinal projection display.

[0031] The imaging unit 18 is an imaging device having an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). The imaging unit 18 captures images in the field of view of the operator wearing the wearable device 10. In other words, the imaging direction of the imaging unit 18 is directed outwards from the wearable device 10. Note that there is not limited to one imaging unit 18, and there may be multiple units. Also, the imaging unit 18 may be a separate imaging device from the wearable device 10.

[0032] The sensor unit 19 is a variety of sensor devices. The sensor unit 19 includes a gaze detection sensor that detects eye movements, such as the direction and position of the gaze of an operator wearing the wearable device 10. Such gaze detection sensors can utilize technologies such as near-infrared LEDs or eye-tracking cameras.

[0033] Furthermore, the sensor unit 19 may include motion sensors, such as acceleration sensors and gyroscopes, that can detect the tilt and movement of the operator (wearable device 10). Preferably, the motion sensors can detect at least the operator's elevation and depression angles. The sensor unit 19 may also include a position detection sensor that detects the position of the wearable device 10 within the store. The sensor unit 19 may also include a depth sensor that can capture the distance to an object in the operator's field of view and the shape of the object as a three-dimensional object.

[0034] Figure 4 shows an example of the hardware configuration of a server device 20 according to the embodiment. As shown in Figure 4, the server device 20 includes a CPU 21, a ROM 22, a RAM 23, a communication unit 24, and a storage unit 25.

[0035] The CPU 21 is an example of a processor that comprehensively controls all parts of the server device 20. The ROM 22 stores various programs. The RAM 23 is a workspace for displaying programs and various data.

[0036] The CPU 21, ROM 22, and RAM 23 are connected via a bus or the like to form a control unit 200 of the computer configuration. The control unit 200 performs various processes by having the CPU 21 operate according to a program stored in the ROM 22 or memory unit 25 and loaded into the RAM 23.

[0037] The communication unit 24 is a wired or wireless communication interface that can be connected to the network N. The communication unit 24 communicates with external devices such as the wearable device 10 via the network N.

[0038] The storage unit 25 has a storage medium such as an HDD (Hard Disk Drive), SSD, or flash memory, and retains its contents even when the power is cut off. The storage unit 25 stores programs 251 that the CPU 21 can execute, as well as various setting information. Program 251 includes programs for realizing the functional configuration described later.

[0039] Furthermore, the storage unit 16 stores the customer management table 252. The customer management table 252 is a data table or database for storing and managing information about customers who use the store.

[0040] Figure 5 shows an example of the data structure of the customer management table 252 according to the embodiment. As shown in Figure 5, the customer management table 252 stores facial images and customer information associated with the customer ID.

[0041] Here, the customer ID is an example of identifying information that can identify a customer. For example, the customer ID can be a membership number assigned to a customer who has registered as a member. The facial image is image data of the customer's face corresponding to the customer ID, or feature information indicating the facial features. The facial image can be used, for example, for customer facial recognition.

[0042] Customer information may include, for example, the customer's name corresponding to the customer ID, and information regarding their membership grade. It may also include cautionary information, such as records of past shoplifting incidents.

[0043] In this embodiment, the customer management table 252 is stored in the server device 20, but this is not the only configuration. For example, each wearable device 10 may store the customer management table 252.

[0044] Next, the functional configuration of the wearable device 10 and the server device 20 will be described with reference to Figure 6. Figure 6 is a diagram showing an example of the functional configuration of the wearable device 10 and the server device 20 according to this embodiment.

[0045] The wearable device 10 includes, as its functional configuration, an image acquisition unit 101, a customer recognition unit 102, a display control unit 103, an operation detection unit 104, a code reading unit 105, a sales data processing unit 106, and the like.

[0046] The functional configuration of the wearable device 10 is realized, for example, by the CPU 11 of the wearable device 10 shown in Figure 2 executing the program 161 stored in the memory unit 16. Note that some or all of the functional configuration of the wearable device 10 may be realized by hardware configurations such as dedicated circuits provided by the wearable device 10.

[0047] The image acquisition unit 101 is an example of an acquisition means. The image acquisition unit 101 acquires images (hereinafter also referred to as captured images) captured by the imaging unit 18. In this embodiment, the captured image is assumed to be real-time video captured by the imaging unit 18.

[0048] The customer recognition unit 102 performs an authentication process to identify the customer corresponding to the face image, based on the face image contained in the captured image acquired by the image acquisition unit 101. Specifically, the customer recognition unit 102 detects the region in the captured image where a person's face is represented (face image) and identifies the customer captured by the imaging unit 18 by comparing it with the face image stored in the customer management table 252. The customer recognition unit 102 then obtains the customer information of the identified customer from the customer management table 252. The customer recognition unit 102 may also obtain the customer ID and face image of the identified customer from the customer management table 252 along with the customer information.

[0049] Here, the authentication process based on the facial image can utilize known techniques. For example, the customer recognition unit 102 can use a technique that derives the feature quantities of the detected facial image and identifies the customer based on the similarity with the feature quantities of the facial image stored in the customer management table 252. In this case, the customer whose facial image has the highest similarity, and whose similarity is above a threshold, can be identified as the customer captured by the imaging unit 18.

[0050] Furthermore, the authentication process based on the facial image may be performed by the customer recognition unit 102 or by the server device 20. In the latter case, the customer recognition unit 102 sends a facial authentication request including the detected facial image to the server device 20 and obtains the result of the authentication process performed by the server device 20 from the server device 20.

[0051] Furthermore, it is preferable that the customer recognition unit 102 performs authentication processing using face images from the captured image whose size is greater than or equal to a threshold. Alternatively, the customer recognition unit 102 may perform authentication processing using face images specified by gaze control, as described later. This allows the customer recognition unit 102 to efficiently detect the face image of a customer facing the operator (store clerk) of the wearable device 10.

[0052] The display control unit 103 is an example of a display control means. The display control unit 103 controls the display of various information and images on the display unit 17. For example, the display control unit 103 displays the captured image acquired by the image acquisition unit 101 on the display unit 17, thereby presenting the scenery in the operator's field of view to the operator.

[0053] Furthermore, the display control unit 103 displays a screen for processing sales data on the display unit 17. For example, the display control unit 103 displays a registration screen for processing registration on the display unit 17. For example, the display control unit 103 displays a settlement screen for processing settlement on the display unit 17. In addition, the display control unit 103 displays customer information acquired by the customer recognition unit 102 on the display unit 17.

[0054] Here, Figure 7 shows an example of an image displayed on the display unit 17 of the wearable device 10 according to the embodiment. As shown in Figure 7, the display unit 17 of the wearable device 10 displays a landscape image Ga representing the external scenery captured by the imaging unit 18. The landscape image Ga includes not only the scenery inside the store, but also, for example, the figures of customers conducting transactions.

[0055] Furthermore, the display control unit 103 overlays an operation screen Gb for processing sales data onto the landscape image Ga. Here, the operation screen Gb is an example of a registration screen for registering products. The operation screen Gb includes, for example, a product display field Gba that lists information such as the product name and price of registered products in the order they were registered, and a total field Gbb that displays the total number of registered products and the total amount. The operation screen Gb also includes an operator field Gbc that is equipped with various operators such as value keys and accounting keys.

[0056] Furthermore, it is preferable for the display control unit 103 to superimpose and display the operation screen Gb within the landscape image Ga at a predetermined position and size. For example, as shown in Figure 7, by displaying the operation screen Gb below the center of the landscape image Ga, the face of the customer being faced can be displayed above the operation screen Gb. The display control unit 103 may also change the position and size of the operation screen Gb in response to the operator's actions.

[0057] Furthermore, when the customer recognition unit 102 acquires customer information, the display control unit 103 overlays and displays the customer information on the landscape image Ga. For example, as shown in Figure 7, the display control unit 103 displays customer information Gc in the upper right portion of the landscape image Ga. Here, an example is shown where the customer's membership grade is displayed as customer information Gc. An example is also shown where a face image is displayed together with the customer information Gc.

[0058] Furthermore, when the customer recognition unit 102 acquires customer information, the display control unit 103 may cooperate with the customer recognition unit 102 to display the area of ​​the face image detected by the customer recognition unit 102 in an identifiable manner. Figure 7 shows an example in which the area of ​​the face image detected by the customer recognition unit 102 is displayed identifiable with a circular mark Gd. This allows the operator of the wearable device 10 to easily confirm which customer among the customers in the field of view the authentication process was performed on, and thus easily verify the correspondence with customer information.

[0059] Returning to Figure 6, the operation detection unit 104 is an example of the first detection means. The operation detection unit 104 detects the operator's movements. Specifically, the operation detection unit 104 detects an area representing the operator's hand (hereinafter also referred to as the hand image) included in the captured image acquired by the image acquisition unit 101 or the image displayed by the display control unit 103, and detects the action performed by the operator as an operation based on the position and movement of the hand image.

[0060] For example, the operation detection unit 104 detects touch operations on the controls of the operation screen Gb shown in Figure 7. The operation detection unit 104 also detects operations such as picking up (grabbing) an item such as a product or the operation screen Gb, or moving it. Here, the operation of grasping an item such as a product is an example of a first operation.

[0061] Furthermore, the operation detection unit 104 works in cooperation with the sensor unit 19 to detect the operator's gaze movements as operations. For example, the operation detection unit 104 detects that an area has been selected when the operator's gaze is focused on a specific area on the image displayed on the display unit 17 for a predetermined time (e.g., 2 seconds). Here, the operator's gaze movements are an example of a second operation.

[0062] The display control unit 103 controls the image to be displayed in accordance with the operation detected by the operation detection unit 104. For example, if the operation detection unit 104 detects an operation to grab or move the operation screen Gb, the display control unit 103 deforms or moves the operation screen Gb in accordance with that operation.

[0063] The code reading unit 105 is an example of a second detection means and reading means. The code reading unit 105 detects areas representing code symbols such as barcodes and two-dimensional codes contained in the captured image acquired by the image acquisition unit 101 or the image displayed by the display control unit 103. The code reading unit 105 also reads the information held in the detected code symbol by decoding it.

[0064] For example, the code reading unit 105 reads the product code of a product from the code symbol attached to the product. Also, for example, the code reading unit 105 reads information related to code payment from the code symbol displayed on the customer's mobile terminal.

[0065] Furthermore, it is preferable that the code reading unit 105 works in cooperation with the operation detection unit 104 to read the code symbol in accordance with the operator's movements.

[0066] For example, when the operation detection unit 104 detects that an operator is picking up an item such as a product, the code reading unit 105 will read the code symbol attached to the item that was the target of the operation, and exclude the code symbols attached to other products from being read.

[0067] Furthermore, for example, the code reading unit 105 targets the code symbol selected by the operator's gaze and excludes other code symbols from being read. Alternatively, the code reading unit 105 may target the code symbol attached to the item for reading only if both the operation of picking up the item and the operation of selecting a code symbol by gaze are detected.

[0068] This reduces the likelihood of the code reading unit 105 reading code symbols unintended by the operator of the wearable device 10, thereby improving the convenience of reading code symbols.

[0069] Furthermore, the code reading unit 105 may switch its code symbol reading operation according to the sensing results of the motion sensor. For example, the code reading unit 105 may switch the code symbol reading operation based on the elevation angle of the operator (head) sensed by the motion sensor.

[0070] For example, when the operator is facing approximately straight ahead, the code reading unit 105 reads information from a specific code symbol according to the detection result of the operation detection unit 104 described above. Also, when the operator is facing downwards, the code reading unit 105 reads information from each of the code symbols contained in the image.

[0071] By performing the above control, the code reading unit 105 can switch the operation related to reading the code symbol depending on whether the operator is holding the product in their hand to read the code symbol or whether multiple products are placed on the top surface of a checker table or the like to read the code symbol. This allows for efficient code symbol reading, as information can be read from each of the products when multiple products are placed on the checker table to read the code symbol. The elevation angle setting related to switching the reading operation can be adjusted arbitrarily.

[0072] Furthermore, by using AR technology or the like, a reading area is set in real space where a reading operation is specified, and the code reading unit 105 may switch its reading operation depending on whether or not an item is included in the reading area. For example, by setting a reading area on the top surface of a checker table or the like where information is read from each of the code symbols included in the image, the same effect as the control described above can be achieved. Alternatively, a reading area where code symbols are read and a non-reading area where the reading of code symbols is prohibited may be set in real space. In this case, the code reading unit 105 will detect or read code symbols from items present in the reading area.

[0073] Furthermore, it is preferable that the display control unit 103 cooperates with the code reading unit 105 to display the location of the code symbol read by the code reading unit 105 in an identifiable manner. For example, as shown in Figure 7, the display control unit 103 displays the code symbol read by the code reading unit 105 in an identifiable manner by surrounding it with a rectangular mark Ge. This allows the operator of the wearable device 10 to easily confirm which code symbol has been read among the code symbols present in their field of view, thereby improving the convenience of the code symbol reading operation.

[0074] Returning to Figure 6, the sales data processing unit 106 is an example of a processing means. Based on the information read by the code reading unit 105, the sales data processing unit 106 performs sales data processing related to commercial transactions at the store. In this embodiment, the sales data processing unit 106 performs at least the registration process, which is one of the registration process and settlement process included in the sales data processing.

[0075] Specifically, during the registration process, when the code reading unit 105 reads a product code, the sales data processing unit 106 reads the product information corresponding to that product code from the product master 162 and stores it in RAM 13 in association with the product code. Furthermore, the sales data processing unit 106 works in cooperation with the display control unit 103 to display the product name and price included in the product information read from the product master 162 in the product display field Gba of the operation screen Gb shown in Figure 7, and also displays the total number of registered products and the total amount in the total field Gbb.

[0076] Furthermore, the sales data processing unit 106 works in cooperation with the operation detection unit 104 to execute processing in response to the operator's actions. For example, when the sales data processing unit 106 receives an operation on the accounting key displayed in the operation field Gbc on the operation screen Gb, it terminates the registration process.

[0077] After completing the registration process, the sales data processing unit 106 may proceed to the settlement process, or it may perform a process to transfer the data related to the registered products to a settlement machine (not shown) installed in the store, since the settlement process is performed using the settlement machine.

[0078] When processing a payment using the wearable device 10, the sales data processing unit 106 works in cooperation with the display control unit 103 to display a payment screen (not shown) on the display unit 17 for processing the payment. The payment screen includes, for example, the total price of the goods, as well as controls for selecting a payment method. Payment methods include cash, credit card payment, and code payment.

[0079] The operator of the wearable device 10 confirms the payment method with the customer and selects the payment method operator desired by the customer using their hand, gaze, etc. The sales data processing unit 106, in cooperation with the operation detection unit 104, receives the payment method selection operation and settles the price of the registered goods based on the payment amount paid by that payment method.

[0080] For example, if cash is selected, the sales data processing unit 106 performs the settlement process using an automatic change dispenser connected to the short-range communication unit 15. Also, for example, if credit card payment is selected, the sales data processing unit 106 performs the settlement process using a credit terminal 32 connected to the short-range communication unit 15. Furthermore, for example, if code payment is selected, the sales data processing unit 106 works in cooperation with the code reading unit 105 to read the code payment information from the code symbol for code payment displayed on the customer's mobile terminal, and performs the settlement process based on that information.

[0081] Then, once the settlement process is complete, the sales data processing unit 106 uses the printer 31 connected to the short-range communication unit 15 to print a receipt detailing the transaction. The receipt will include, for example, the registered items, the total number of items, the total amount, the payment amount, and the amount of change.

[0082] Furthermore, when transferring data to a payment machine, the sales data processing unit 106 executes a process to transfer the product code and product information (hereinafter also referred to as transaction data) of the registered product held in RAM 13 to the payment machine at the destination. For example, the sales data processing unit 106 may send the transaction data to the payment machine at the destination via the network N, or it may send the transaction data to the payment machine at the destination via the server device 20. This allows the payment machine to perform the payment processing for the product registered on the wearable device 10.

[0083] On the other hand, the server device 20 includes a data management unit 201 as part of its functional configuration, as shown in Figure 6. The functional configuration of the server device 20 is realized, for example, by the CPU 21 of the server device 20 shown in Figure 4 executing a program 251 stored in the storage unit 25. The functional configuration of the server device 20 may also be realized by hardware configurations such as dedicated circuits provided in the server device 20.

[0084] The data management unit 201 manages various types of data related to the sales data processing system 1. For example, the data management unit 201 performs processing to register the facial images and customer information of registered members in the customer management table 252. The data management unit 201 also performs processing to provide the product master 162 to each of the wearable devices 10.

[0085] Furthermore, when the data management unit 201 receives a facial recognition request from the wearable device 10, it performs authentication processing by comparing the facial image included in the facial recognition request with the facial images of each customer stored in the customer management table 252. The data management unit 201 then provides the result of the authentication processing to the wearable device 10 that sent the facial recognition request.

[0086] For example, if the data management unit 201 can identify a customer through the authentication process, it reads the customer's information from the customer management table 252 and provides it to the wearable device 10 as part of the authentication process. Alternatively, if the data management unit 201 cannot identify a customer through the authentication process, it provides the wearable device 10 with error information indicating that the customer does not exist as part of the authentication process.

[0087] The operation of the wearable device 10 will be described below with reference to Figure 8. Figure 8 is a flowchart showing an example of the processing performed by the wearable device 10 of this embodiment. It is assumed that the captured image acquired by the image acquisition unit 101 is displayed on the display unit 17 as a prerequisite for this processing.

[0088] First, the sales data processing unit 106 waits until it is instructed to start a transaction (step S11; No). Here, there is no particular method for instructing the start of a transaction, and various methods can be used. For example, an operator for instructing the start of a transaction may be displayed on the display unit 17, and when this operator is operated, it may be determined that the start of a transaction has been instructed. Alternatively, the customer recognition unit 102 detecting a customer's face may be used as the instruction to start a transaction.

[0089] When the sales data processing unit 106 determines that a transaction initiation has been instructed (step S11; Yes), it works in cooperation with the display control unit 103 to display the registration screen on the display unit 17 (step S12). Next, the customer recognition unit 102 performs facial recognition based on the customer's facial image and determines whether or not it was able to recognize (identify) the customer (step S13). If the customer recognition unit 102 determines that it cannot identify the customer (step S13; No), it proceeds to step S16.

[0090] If the customer recognition unit 102 determines that it has identified a customer (step S13; Yes), it acquires the customer information of the identified customer (step S14). Next, the display control unit 103 displays the customer information acquired in step S14 on the display unit 17 (step S15) and proceeds to step S16.

[0091] Next, the code reading unit 105 determines whether or not it has detected a code symbol (step S16). If the code reading unit 105 determines that it cannot detect a code symbol (step S16; No), the process proceeds to step S22.

[0092] If the code reading unit 105 determines that it has detected a code symbol (step S16; Yes), it determines whether the detected code symbol is the target of an operation by the operator's manual operation and / or gaze operation (step S17). If the code reading unit 105 determines that it is not the target of an operation (step S17; No), it proceeds to step S22.

[0093] If the code reading unit 105 determines that the detected code symbol is the target of the operation (step S17; Yes), it reads the information held in the code symbol (step S18). Next, the sales data processing unit 106 determines whether the information read in step S18 is a product code or not (step S19). If the sales data processing unit 106 determines that it is not a product code (step S19; No), it proceeds to step S22.

[0094] When the sales data processing unit 106 determines that a product code has been read (step S19; Yes), it registers the product based on that product code (step S20). Next, the sales data processing unit 106 updates the registration screen by displaying information such as the product name and price of the registered product on the registration screen (step S21).

[0095] Next, the sales data processing unit 106 determines whether or not the registration process has been instructed to end (step S22). If the sales data processing unit 106 determines that the registration process has not been instructed to end (step S22; No), it proceeds to step S16. If it determines that the registration process has been instructed to end (step S22; Yes), the sales data processing unit 106 terminates this process.

[0096] As described above, the wearable device 10 of this embodiment acquires an image captured by the imaging unit 18 that captures the field of view of the operator, and detects the code symbols contained in the image. The wearable device 10 also detects the operator's movements and reads information from the code symbols according to the operator's movements. Then, the wearable device 10 performs sales data processing based on the information read from the code symbols.

[0097] This allows the wearable device 10 to read information from the code symbol intended by the operator, thus enabling efficient code symbol reading. Therefore, sales data processing can be performed efficiently using the wearable device 10.

[0098] Furthermore, the wearable device 10 reads information from the code symbol of the product held by the operator among the detected code symbols. This allows the wearable device 10 to focus on reading information from the code symbol held by the operator, thus enabling efficient code symbol reading. Therefore, using the wearable device 10 allows for efficient sales data processing.

[0099] Furthermore, the wearable device 10 reads information from the code symbols that are in the operator's line of sight among the detected code symbols. This allows the wearable device 10 to focus on reading information from the code symbols that the operator is paying attention to, thus enabling efficient code symbol reading. Therefore, using the wearable device 10 enables efficient sales data processing.

[0100] Furthermore, the wearable device 10 displays the information read from the code symbol and also displays the location of the read code symbol in an identifiable manner. As a result, the wearable device 10 can present the operator with both the information read from the code symbol and the location of the code symbol, thereby improving the convenience of reading the code symbol.

[0101] The above-described embodiments can also be modified and implemented as appropriate by changing some of the configurations or functions of each of the above-described devices. Therefore, several modifications of the above-described embodiments will be described below as other embodiments. In the following, we will mainly describe the differences from the above-described embodiments, and will omit detailed explanations of points that are common with what has already been described. Furthermore, the modifications described below may be implemented individually or in combination as appropriate.

[0102] (Variation 1) In the above-described embodiment, a store employee uses the wearable device 10, but a customer may also use the wearable device 10 to process sales data. However, if a customer uses the wearable device 10, it is preferable to disable the facial recognition and customer information display functions described above.

[0103] For example, the wearable device 10 may be configured to enable facial recognition and customer information display functions when an employee code indicating that the user is an employee is entered in the form of a code symbol or the like, and to disable facial recognition and customer information display functions when an employee code is not entered.

[0104] Alternatively, for example, an application capable of realizing the functions (processing) of the wearable device 10 of the above-described embodiment may be installed on a wearable device owned by the customer, thereby enabling sales data processing using the wearable device owned by the customer. In this case, the receipt issued at the end of settlement may not be printed, but rather displayed or stored on the wearable device in the form of an electronic receipt.

[0105] (Modification 2) In the embodiments described above, examples of hand movements (operations) detected by the operation detection unit 104 included grasping and movement, but the detectable movements are not limited to these. For example, the operation detection unit 104 may work in cooperation with the code reading unit 105, etc., to detect complex movements performed by the operator.

[0106] For example, the operation detection unit 104 may detect an operation in which an operator places an item (i.e., a product) on which a code symbol has been detected into a storage container such as a basket or bag without reading the code symbol, as a potentially fraudulent operation. Such detection of fraudulent operations is particularly useful when customers are using a wearable device 10, and by providing notification, it is possible to reduce theft such as so-called basket theft (also known as checkout theft).

[0107] Furthermore, if fraudulent activity is detected, it is preferable for the operation detection unit 104 to provide notification. The method of notification is not particularly limited; for example, the operation detection unit 104 may cooperate with the display control unit 103 to display information on the display unit 17 indicating that fraudulent activity is suspected. Alternatively, the operation detection unit 104 may transmit information indicating that fraudulent activity is suspected to the server device 20 or the terminal device operated by the store clerk via the network N.

[0108] While embodiments (and modifications) of the present invention have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, combinations, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0109] Furthermore, the programs executed by each device in the above embodiments are provided pre-installed in the storage medium (ROM or storage unit) provided by the device, but are not limited to this. For example, the programs may be configured to be provided as installable or executable files recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk). Moreover, the storage medium is not limited to a medium independent of the computer or embedded system, but also includes storage media that have downloaded and stored or temporarily stored programs transmitted via a LAN, the Internet, etc.

[0110] Furthermore, the programs executed by each device of the above embodiment may be stored on a computer connected to a network such as the Internet and provided by allowing users to download them via the network, or they may be configured to be provided or distributed via a network such as the Internet. [Explanation of Symbols]

[0111] 1. Sales data processing system 10 Wearable Devices 20 Server Devices 101 Image acquisition unit 102 Customer Recognition Department 103 Display Control Unit 104 Operation detection unit 105 Code reading unit 106 Sales Data Processing Unit 201 Data Management Department [Prior art documents] [Patent Documents]

[0112] [Patent Document 1] Japanese Patent Publication No. 2022-91841

Claims

1. A wearable device for use in stores, which can be worn in front of the operator's eyes. An acquisition means for acquiring an image captured by an imaging unit that captures the field of view of the operator, A first detection means for detecting the movements of the operator, A second detection means for detecting code symbols contained in the aforementioned image, A reading means reads information from a code symbol detected by the second detection means in response to the operator's movement detected by the first detection means, A processing means that performs sales data processing related to commercial transactions at the store based on the information read by the reading means, A wearable device equipped with [features / equipment].

2. The first detection means detects the first action of the operator picking up the item, The reading means reads information from the code symbols that the second detection means has detected, specifically those attached to the article that was the target of the first operation. The wearable device according to claim 1.

3. The first detection means detects the operator's eye movement as a second action, The reading means reads information from the code symbols that are located in the line of sight detected in the second operation, among the code symbols detected by the second detection means. A wearable device according to claim 1 or 2.

4. The wearable device according to claim 1, further comprising a display control means for causing the display unit that displays an image in front of the operator's eyes to display the information read by the reading means.

5. The wearable device according to claim 4, wherein the display control means causes the position of the code symbol read by the reading means to be displayed in an identifiable manner.

6. A computer for use as a wearable device in a store, which can be worn in front of the operator's eyes. An acquisition means for acquiring an image captured by an imaging unit that captures the field of view of the operator, A first detection means for detecting the movements of the operator, A second detection means for detecting code symbols contained in the aforementioned image, A reading means reads information from a code symbol detected by the second detection means in response to the operator's movement detected by the first detection means, A processing means that performs sales data processing related to commercial transactions at the store based on the information read by the reading means, A program to make it work.

Images