Electronic device system

Abstract

To provide an electronic device system that is easy to use.SOLUTION: An electronic device system comprises: a detection unit for detecting the entry of a user into a store based on information from a device owned by the user; an image recognition unit for recognizing an item in the store by inputting information captured within the store; an acquisition unit for acquiring information related to the item recognized by the image recognition unit; and a transmission unit for transmitting information to be displayed on the device owned by the user, the information related to the item acquired by the acquisition unit.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to an electronic device system. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there have been proposed sales promotion systems that acquire behavioral information regarding product purchases by customers in a store and link the information to sales promotion (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2010-277256 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional sales promotion systems have not been able to acquire sufficient necessary information, and are not user-friendly.

[0005] The present invention has been made in view of the above-mentioned problems, and has an object to provide an electronic device system that is easy to use. [Means for solving the problem]

[0006] The electronic device of the present invention includes an input unit (46, 42) that inputs information about an item in response to at least one of a user picking up an item and an item entering a predetermined range within the user's field of vision, and an alarm unit (46) that issues an alarm regarding the information about the item input by the input unit.

[0007] In this case, the device may include a first control unit (46) that limits the information notified by the notification unit when the user picks up the item multiple times or when the item enters a predetermined range in the user's field of vision multiple times. In this case, the first control unit may vary the information notified by the notification unit depending on the number of times the item is picked up or the number of times the item enters a predetermined range in the user's field of vision. The device may also include a second control unit (46) that limits the information notified by the notification unit when the user picks up a similar item that is similar to the item or when the similar item enters a predetermined range in the user's field of vision.

[0008] In the electronic device of the present invention, the input unit may input information about a product that the user plans to purchase as the item information, and the notification unit may notify information corresponding to the product that the user plans to purchase. In this case, the input unit may input the information about the product that the user plans to purchase from an external device (20'). Also, the input unit may input the information about the product that the user plans to purchase from the external device via a storage device (120) that stores the product that the user plans to purchase.

[0009] In the electronic device of the present invention, the notification unit may include at least one of a display unit (12) that displays and an audio output unit (18) that outputs audio. In this case, the audio output unit may be a directional audio output unit. In addition, the electronic device of the present invention may include a mounting unit (110) that can be mounted near the user's face.

[0010] In order to easily explain the present invention, the above description is made with reference to the reference numerals in the drawings that represent one embodiment, but the present invention is not limited to this, and the configuration of the embodiment described below may be appropriately improved, and at least a part of the configuration may be replaced with another configuration. Furthermore, components that are not particularly limited in terms of their location may be located in a position that can achieve their function, without being limited to the location disclosed in the embodiment. Effect of the Invention

[0011] The present invention has an effect of providing an electronic device system that is easy to use. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a configuration of an electronic device system according to a first embodiment. [Diagram 2] FIG. 13 is a diagram showing the usage status of the mounting device, the basket device, and the article device. [Diagram 3] FIG. [Figure 4] FIG. 4(a) is a flowchart showing the processing of the mounting device according to the first embodiment, and FIG. 4(b) is a flowchart showing the processing of the article device according to the first embodiment. [Diagram 5] 4 is a flowchart showing a process of the cage apparatus according to the first embodiment. [Figure 6] 5 is a flowchart showing a process of a server according to the first embodiment. [Figure 7] FIG. 7(a) and FIG. 7(b) are diagrams (part 1) showing examples of gaze images. [Figure 8] FIG. 2 is a diagram showing an example of a gaze image (part 2). [Figure 9] FIG. 13 is a diagram showing the order of items a user picks up from the time he or she enters a store until the time he or she goes through a cash register. [Figure 10] FIG. 11 is a block diagram showing a configuration of an electronic device system according to a second embodiment. [Figure 11] FIG. 11(a) is a flowchart showing the processing of the mounting device according to the second embodiment, and FIG. 11(b) is a flowchart showing the processing of the article device according to the second embodiment. [Figure 12] 10 is a flowchart showing a process of a cage apparatus according to a second embodiment. [Figure 13] 10 is a flowchart showing a process of a server according to the second embodiment. [Figure 14]14(a) and 14(b) are diagrams showing an example of the shopping DB. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] First Embodiment Hereinafter, the electronic device system according to the first embodiment will be described in detail with reference to Fig. 1 to Fig. 9. Fig. 1 shows a block diagram of the configuration of an electronic device system 100 according to the first embodiment.

[0014] As shown in FIG. 1, the electronic device system 100 includes a wearing device 10, a basket device 20, an article device 30, and a server 40. As shown in FIG. 2, the wearing device 10 is a glasses-type terminal worn on the face of a user (shopper) who visits a store. The basket device 20 is a device provided in a basket 120 installed in a store. The article device 30 is a device provided in an article (goods) 130 displayed in the store. The server 40 is an information processing device provided in a store, a data center, or the like. Note that communication via the human body (human body communication) is possible between the wearing device 10 and the basket device 20 and between the article device 30 and the basket device 20, and wireless communication is possible between the basket device 20 and the server 40.

[0015] (Mounting device 10) As shown in Fig. 1, the wearing device 10 includes an imaging unit 11, a display unit 12, an operation unit 13, a microphone 14, a storage unit 15, a communication unit 16, and a control unit 17. Fig. 3 shows a perspective view of the wearing device 10. As shown in Fig. 3, the wearing device 10 includes a glasses-type frame 110. The components of the wearing device 10 that are shown in Fig. 1 but not shown in Fig. 3 are assumed to be provided inside the frame 110 or in a part of the frame 110.

[0016] The imaging unit 11 includes a lens, an imaging element, an image processor, and the like, and captures still images and videos. The imaging unit 11 is provided near an end of the frame 110 (near the right eye of the user) as shown in Fig. 3. Therefore, when the user wears the wearing device 10 (as shown in Fig. 2), an image can be captured in the direction in which the user is facing (looking). In addition, a predetermined portion of the captured image (for example, a portion slightly shifted to the left from the center) can be regarded as the portion in which the user is directing his or her gaze (the portion being gazed at).

[0017] The display unit 12 has a projector provided inside the frame 110 or near the frame 110, and a prism for directing an image projected from the projector to the user's eyes. The display unit 12 displays various information under the control of the control unit 17.

[0018] The operation unit 13 is a touch pad provided on the frame 110 , which detects the movement of the user's finger, accepts operations from the user, and transmits the accepted operation information to the control unit 17 .

[0019] The imaging unit 11, the display unit 12, and the operation unit 13 are disclosed in detail in, for example, US Patent Publication No. 2013 / 0044042.

[0020] The microphone 14 is provided in the frame 110 and collects voice uttered by the user and voices around the user. The voice collected by the microphone 14 is recognized by a voice recognition unit (not shown), and the voice recognition result is transmitted to the control unit 17. The control unit 17 executes processing based on the voice recognition result (e.g., command execution processing, etc.). Note that the control unit 17 may execute the voice recognition. The control unit 17 also stores the collected voice data in the memory unit 15.

[0021] The storage unit 15 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores image data captured by the imaging unit 11, audio data collected by the microphone 14, display data to be displayed on the display unit 12, and the like.

[0022] The communication unit 16 performs human body communication with other devices. The communication unit 16 has an electrode unit 16a provided on the frame 110 and capable of contacting the user, and a human body communication unit 16b that performs human body communication using the electrode unit 16a. Note that human body communication includes a current method in which a weak current is passed through the human body and the current is modulated to transmit information, and an electric field method in which an electric field induced on the surface of the human body is modulated to transmit information. In this embodiment, either the current method or the electric field method can be used. Note that human body communication is possible not only when the user directly contacts the electrode unit 16a of the wearing device 10, but also when the electrode unit 16a and the user are not directly in contact with each other. In this embodiment, the communication unit 16 transmits image data captured by the imaging unit 11 and voice data collected by the microphone 14 to the cage device 20. Note that the communication unit 16 may perform close-proximity communication with other devices, such as Bluetooth (registered trademark), RFID (Radio Frequency Identification), TransferJet (registered trademark), etc.

[0023] Note that the mounting device 10 may have a wireless communication unit instead of or in combination with the communication unit 16. In this case, image data and audio data may be directly transmitted to the server 40 via wireless communication, or image data and audio data may be transmitted to the basket device 20 via wireless communication.

[0024] The control unit 17 performs overall control of the entire wearing device 10. The control unit 17 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc. In this embodiment, the control unit 17 performs control related to imaging by the imaging unit 11, control related to display on the display unit 12, etc.

[0025] The wearing device 10 does not have to be a glasses-type terminal. For example, it may be a wearable terminal that can be worn anywhere on the body, and some of its functions may be separated. For example, it may be a contact lens-type terminal equipped with a display unit, with other functions being provided in a wristwatch-type terminal.

[0026] (Cage device 20) As shown in FIG. 1, the basket device 20 includes a biosensor 21, a calendar unit 22, a storage unit 23, a first communication unit 24, a second communication unit 25, a control unit 26, and the like.

[0027] The biosensor 21 is provided on a portion of the basket 120 that the user touches when holding the basket 120 (handle portion). The biosensor 21 detects changes in biometric information of the body, such as heart rate, blood pressure, and sweat rate, and includes, for example, an LED (light emitting diode), a photosensor, and a sweat sensor. For example, by using an LED and a photosensor, the heart rate can be detected by receiving light reflected by a finger when light emitted by the LED is applied to the finger with the photosensor. In addition, the sweat sensor can detect the amount of sweat by measuring the impedance of the hand with a plurality of electrodes. In this embodiment, the output from the biosensor 21 is sent to the control unit 26. The control unit 26 estimates changes in the user's emotions (joy, anger, sadness, and happiness) based on the output from the biosensor 21.

[0028] The calendar unit 22 acquires time information such as the year, month, date, and time, and outputs it to the control unit 26. The calendar unit 22 also measures time.

[0029] The storage unit 23 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores information about an item picked up by a user, as described below. The storage unit 23 also stores an ID number (basket ID) for uniquely identifying the basket device 20.

[0030] The first communication unit 24 performs human body communication with other devices. The first communication unit 24 may perform short-range communication with other devices. In the present embodiment, the first communication unit 24 communicates with the communication unit 16 of the wearing device 10 and the communication unit 32 of the article device 30 using human body communication.

[0031] First communication unit 24 has an electrode unit 24a that is provided on the handle portion of basket 120 and can come into contact with the user's hand or arm, and a human body communication unit 24b that performs human body communication via electrode unit 24a.

[0032] The second communication unit 25 performs wireless communication with other devices. In this embodiment, wireless communication is performed with the server 40, and the second communication unit 25 transmits to the server 40 the article information of the article touched by the user, the image data captured by the imaging unit 11, the voice data collected by the microphone 14, and the detection result of the biosensor 21. The basket device 20 of this embodiment has a function as a communication relay device that relays the communication of the article information between the article device 30 and the server 40, and the communication of the image data and the voice data between the wearing device 10 and the server 40.

[0033] The control unit 26 controls the entire basket device 20, and includes a CPU, RAM, ROM, etc. In this embodiment, the control unit 26 controls information communication between the article device 30, the mounting device 10, and the server 40.

[0034] The basket device 20 may be provided in a cart used in a store. In this case, the biosensor 21 and the electrode unit 24a of the first communication unit 24 are preferably provided in a grip portion of the cart.

[0035] (Article device 30) The article device 30 is provided on an article (goods) sold in a store or the like, such as food, clothes, shoes, watches, accessories, golf clubs, bags, etc., that comes into contact with the body of a customer (user). For convenience of explanation, in this embodiment, the article device 30 is described as being provided on a good (food, etc.) displayed in a supermarket.

[0036] As shown in FIG. 1, the article device 30 includes a storage unit 31, a communication unit 32, and a control unit 33.

[0037] The storage unit 31 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores information about the item (product) to which the product device 30 is attached. Specifically, the storage unit 31 stores product information (product information) of the item to which the product device 30 is attached. For example, when the product device 30 is attached to a processed food (e.g., canned food), the storage unit 31 stores the manufacturer (seller), product name, ingredient names, expiration date, content amount, country of origin, etc. as product information.

[0038] The control unit 33 controls the entire article device 30 and includes a CPU, RAM, ROM, etc. In this embodiment, the control unit 33 controls the transmission of article information stored in the memory unit 31 to the basket device 20. The control unit 33 and the memory unit 31 may be provided at any position on the article device.

[0039] The communication unit 32 performs human body communication with other devices. The communication unit 32 may perform short-range communication with other devices. In this embodiment, the communication unit 32 includes an electrode unit 32a provided at a position that contacts a user (e.g., a hand), and a human body communication unit 32b that performs human body communication using the electrode unit 32a. In this embodiment, as shown in FIG. 2, when a user holding a basket 120 holds an article 130 in his / her hand, human body communication is established between the first communication unit 24 of the basket device 20 and the communication unit 32 of the article device 30. As a result, the article information stored in the storage unit 31 is transmitted to the basket device 20. Note that human body communication is possible not only when the user directly contacts the electrode unit 32a of the article device 30, but also when the user does not directly contact the electrode unit 32a (e.g., when wearing gloves).

[0040] (Server 40) As shown in FIG. 1, the server 40 includes a storage unit 41, a communication unit 42, an OCR (Optical Character Reader) unit 43, an image recognition unit 44, and a control unit 46.

[0041] The storage unit 41 is, for example, a non-volatile semiconductor memory such as a flash memory or a hard disk drive, and stores information acquired from the basket device 20, i.e., image data captured by the imaging unit 11 of the wearing device 10, voice data collected using the microphone 14, and item information transmitted from the item device 30. The storage unit 41 also stores face image data and item image data of store employees (clerks), as well as various data used when analyzing user purchasing behavior. In this embodiment, the control unit 46 detects whether the employee's face or an item is included in the image data captured by the imaging unit 11 of the wearing device 10 in cooperation with the image recognition unit 44, as described below.

[0042] The communication unit 42 performs wireless communication with other devices. In the present embodiment, the communication unit 42 performs wireless communication with the second communication unit 25 of the car device 20 via wireless communication.

[0043] The OCR unit 43 recognizes characters included in the image data captured by the imaging unit 11 of the wearing device 10 and converts the characters into text data. For example, when the imaging unit 11 captures the image data of Fig. 7(b), the OCR unit 43 performs character recognition and obtains the text data "Sorry, we are currently waiting for the item to arrive. The next arrival date has not been decided."

[0044] The image recognition unit 44 performs image recognition by extracting feature points of image data captured by the imaging unit 11 of the wearing device 10 using a feature detection algorithm such as SURF (Speed ​​Up Robust Features) or SHIFT (Scale Invariant Feature Transform), and comparing the extracted feature points with multiple standard patterns of recognition targets stored in the storage unit 41. In this embodiment, the image recognition unit 44 performs comparison with employee face image data and product image data stored in the storage unit 41 to detect whether the image captured by the imaging unit 11 includes an employee or a product.

[0045] The control unit 46 controls the entire server 40, and includes a CPU, RAM, ROM, and the like.

[0046] Next, the processing of the electronic device system 100 will be described in detail with reference to Fig. 4 to Fig. 9. In this embodiment, it is assumed that a user (shopper) wearing the wearing device 10 carries a basket 120 and goes shopping when visiting a store (e.g., a supermarket). The user decides on an item to be purchased by picking up an item (goods) 130 provided with an item device 30 in the store or by placing the item in the basket 120, and performs a purchase process at a cash register.

[0047] (Processing of mounting device 10) FIG. 4(a) shows a flow chart of the processing of the mounting device 10.

[0048] In the process of FIG. 4(a), the control unit 17 of the wearing device 10 waits until the human body communication with the basket device 20 is established in step S10. In this case, when a user wearing the wearing device 10 enters a store and touches the basket device 20 with his / her hand by holding the basket 120, the communication between the communication unit 16 of the wearing device 10 and the first communication unit 24 of the basket device 20 is established. When the human body communication with the basket device 20 is established, the control unit 17 proceeds to step S12. Note that instead of step S10, the process of step S12 and subsequent steps may be executed when the user turns on the power supply of the wearing device 10 after entering the store. In this case, the wearing device 10 may be provided with a position detection device capable of detecting the position according to the communication status with a GPS (Global Positioning System) module or a wireless LAN (Local Area Network) base station installed in the store, and the control unit 17 may determine whether the user has entered the store based on the detection result of the position detection device. Conversely, the wearing device 10 may be configured to constantly supply power to the communication unit 16 and start up when human body communication with the car device 20 is established.

[0049] When the process proceeds to step S12, the control unit 17 issues an instruction to the imaging unit 11 to capture an image from the user side. In this case, the imaging unit 11 may capture either a still image or a video. In the present embodiment, the imaging unit 11 is described as capturing a still image. The control unit 17 stores the image captured by the imaging unit 11 (hereinafter, referred to as a gaze image) in the storage unit 15. In step S12, in addition to capturing the gaze image by the imaging unit 11, the control unit 17 may collect audio including the user's voice and surrounding sounds using the microphone 14, and store the collected audio in the storage unit 15 as audio data.

[0050] Next, in step S14, the control unit 17 transmits the gaze image to the basket device 20 via the communication unit 16. The control unit 17 may measure the user's moving speed by providing an accelerometer or an gyrometer in the wearing device 10, for example, and transmit the gaze image to the basket device 20 when the user's moving speed reaches a predetermined moving speed (for example, 1 km / h or less). The user may take his / her hands off the basket device 20 during shopping. Therefore, the control unit 17 may transmit the gaze image to the basket device 20 by human body communication when human body communication is possible, and transmit the gaze image to the basket device 20 by short-range communication or wireless communication when human body communication is not possible. When wireless communication is performed, the wearing device 10 needs to be equipped with a wireless communication unit.

[0051] Next, in step S16, the control unit 17 judges whether or not the human body communication with the cart device 20, which has received the input from the cash register terminal, has been established. Here, the input from the cash register terminal means the input of shopping completion information transmitted from the cart device 20 to the cart device 20 by communication between the cart device 20 and the cash register terminal when the user takes the cart 120 to the cash register. In other words, when the human body communication between the cart device 20 and the wearing device 10, which has received the input from the cash register terminal, has been established, it means that the user's shopping has been completed. Therefore, if the shopping has not been completed and the judgment in step S16 is negative, the process returns to step S12, and the processing and judgment in steps S12, S14, and S16 are repeated. In other words, the imaging unit 11 captures the gaze image and transmits it to the cart device 20 repeatedly. On the other hand, if the user's shopping has been completed and the judgment in step S16 is positive, the entire process in FIG. 4(a) is completed.

[0052] Instead of step S16, the control unit 17 may determine whether the user (wearing device 10) has left the store. In this case, as described above, the wearing device 10 needs to be provided with a position detection device such as a GPS.

[0053] (Processing of goods device 30) FIG. 4(b) shows a flow chart of the processing of the article device 30.

[0054] 4(b), in step S20, the control unit 33 of the article device 30 waits until new human body communication is established with the basket device 20. In this case, as shown in FIG. 2, when a user holding the basket 120 picks up the article 130 from a product shelf or the like, new human body communication is established with the basket device 20, and the process proceeds to step S22.

[0055] When the process proceeds to step S22, the control unit 33 issues an instruction to the communication unit 32 to transmit the article information stored in the storage unit 31 to the basket device 20.

[0056] Thereafter, the process returns to step S20, and the control unit 33 repeats the processes and determinations of steps S20 and S22.

[0057] 4B, a user carrying a basket 120 can transmit information about an item 130 picked up during shopping to the basket device 20 one by one.

[0058] (Processing of the cage device 20) FIG. 5 shows a flow chart of the processing of the car apparatus 20. In FIG.

[0059] 5, in step S30, the control unit 26 of the car device 20 waits until human body communication is established with the wearing device 10. When human body communication with the wearing device 10 is established, the control unit 26 proceeds to step S32. The timing of the transition to step S32 is the same as the timing of the transition to step S12 in FIG. 4(a) described above.

[0060] When the process proceeds to step S32, the control unit 26 acquires the gaze image transmitted from the wearing device 10 and stores it in the storage unit 23. Note that, for example, an accelerometer or an gyrometer may be provided in the basket device 20 to measure the user's moving speed, and when the user's moving speed reaches a predetermined moving speed (for example, 1 km / h or less), the gaze image transmitted from the wearing device 10 may be acquired and stored in the storage unit 23.

[0061] Next, in step S34, the control unit 26 determines whether or not new product information has been received. If the determination in step S34 is negative, i.e., if new product information has not been received, the process proceeds to step S38. On the other hand, if the determination in step S34 is positive, i.e., if new product information has been received, the process proceeds to step S36.

[0062] In step S36, the control unit 26 acquires the product information and stores it in the storage unit 23. After that, the process proceeds to step S38.

[0063] When the process proceeds to step S38, the control unit 26 executes detection using the biosensor 21. The order of the processes of steps S32, S34, S36, and S38 may be changed as appropriate, and the processes of each step may be executed simultaneously in parallel.

[0064] Next, in step S40, the control unit 26 judges whether or not the user has experienced a change in emotion based on the detection result of the biosensor 21. In this case, for example, when the detection result of the biosensor 21 shows a sudden increase in heart rate, a sudden increase in blood pressure, or a sudden increase in sweating, it is judged that the user has experienced a change in emotion. Examples of when the user's emotion changes include when a time sale starts and the user encounters a chance to purchase a product at a low price, when a product that the user had planned to purchase is sold out, or when the cash register is crowded. If the judgment in step S40 is negative, the process proceeds to step S44, but if the judgment in step S40 is positive, the process proceeds to step S42.

[0065] When the process proceeds to step S42, the control unit 26 issues an instruction to the second communication unit 25 to transmit the gaze image and the article information (hereinafter referred to as first information) that are stored in the memory unit 23 and that are acquired immediately before the emotion change to the server 40. Note that immediately before the emotion change means, for example, several seconds before the emotion change occurs. Note that there are cases where the article information is not acquired immediately before the emotion change. This is the case when the emotion change is not caused by holding the article 130. In this case, the control unit 26 only needs to transmit the gaze image to the server 40 as the first information.

[0066] Here, gaze images as shown in Fig. 7(a), Fig. 7(b), and Fig. 8 are assumed as gaze images acquired immediately before an emotion change. The gaze image in Fig. 7(a) is an image indicating that the user has encountered a limited-time special offer, specifically, an image indicating that the user has seen a limited-time special offer sticker attached to a product. The gaze image in Fig. 7(b) is an image indicating that the product the user was trying to purchase has sold out, specifically, an image indicating that the user has seen a notice indicating that the product is on backorder. The gaze image in Fig. 8 is an image indicating that the user has seen a crowded cash register.

[0067] The first information is transmitted to the server 40 with the basket ID added thereto. That is, the server 40 manages the first information by using the basket ID.

[0068] After step S40 is negative or after the process of step S42 is executed, the control unit 26 judges in step S44 whether or not there has been an input from the cash register terminal. In this case, the input from the cash register terminal means, as described above, the input of shopping completion information that is transmitted from the cash register terminal to the cart device 20 through communication between the cart device 20 and the cash register terminal when the user finishes shopping and takes the cart 120 to the cash register. If the judgment in step S44 is negative, that is, if the user has not finished shopping, the process returns to step S32 and the processes and judgments of steps S32 to S44 are repeated. On the other hand, if the judgment in step S44 is positive, the process proceeds to step S46.

[0069] When the process proceeds to step S46, the control unit 26 issues an instruction to the second communication unit 25 to transmit all of the gaze images, item information, and biological information (hereinafter referred to as second information) acquired after step S30 to the server 40. That is, the control unit 26 transmits the gaze images acquired while the user is shopping, the item information of the item held by the user, and the emotion change information acquired while the user is shopping to the server 40 via the second communication unit 25. The control unit 26 may transmit to the server 40 the voice data acquired using the microphone 14 of the wearing device 10 while the user is shopping. It is assumed that the acquisition time of the information (obtained from the calendar unit 22) is added as a time stamp to each piece of information transmitted from the control unit 26. The second information is transmitted to the server 40 with the basket ID added thereto. That is, the server 40 manages the second information using the basket ID. After that, the control unit 26 ends all the processing of FIG. 5.

[0070] As described above, by the control unit 26 executing the processing of Figure 5, the first information is transmitted from the basket device 20 to the server 40 when there is a change in the user's emotions while shopping, and the second information is transmitted from the basket device 20 to the server 40 when the user finishes shopping.

[0071] (Server 40 processing) FIG. 6 is a flowchart showing the processing of the server 40.

[0072] 6, in step S50, the control unit 46 of the server 40 waits for an input from the car device 20. That is, the control unit 46 proceeds to step S52 at the stage where the first information or the second information is transmitted as a result of the car device 20 executing step S42 or S46 in FIG.

[0073] When the process proceeds to step S52, the control unit 46 judges whether the input information is the first information. That is, the control unit 46 judges whether the line of sight information or the article information acquired by the basket device 20 immediately before the user's emotion changed is input. If the judgment here is positive, the process proceeds to step S54.

[0074] When the process proceeds to step S54, the control unit 46 executes emotion analysis based on the first information. Specifically, the control unit 46 performs emotion analysis of the user (analysis of whether the user is satisfied or dissatisfied) based on the result of analyzing the gaze image acquired immediately before the emotion changed using the OCR unit 43 and the image recognition unit 44, and the result of analyzing the product information acquired immediately before the emotion changed.

[0075] For example, when the gaze image is the image of FIG. 7(a), the control unit 46 can detect the characters "Morning market Limited time offer 20% off the displayed price" by using the OCR unit 43. In this case, since positive words for shoppers such as limited time offer and 20% off are included, the control unit 46 analyzes that the user is happy and satisfied (positive state). Note that the words that are positive for shoppers are stored in the storage unit 41.

[0076] Furthermore, when the gaze image is the image of FIG. 7(b), the control unit 46 can use the OCR unit 43 to detect the text "Sorry, we are currently on backorder. The next arrival date is undecided." In this case, since the text contains negative words for shoppers, such as "sorry," "on backorder," and "undecided," the control unit 46 analyzes that the user is in a sad and dissatisfied state (negative state). Note that it is assumed that negative words for shoppers are stored in the storage unit 41.

[0077] Furthermore, when the gaze image is the image of Fig. 8, the control unit 46 can detect that the register is crowded or that the person waiting at the register has a dissatisfied face (is not smiling) by using the image recognition unit 44. In this case, the control unit 46 determines that the user is sad and dissatisfied (in a negative state).

[0078] Furthermore, when the gaze image includes an image of the employee's face, the control unit 46 determines that the employee has caused a change in the user's emotion. In this case, the control unit 46 uses the image recognition unit 44 to recognize whether the employee's face is smiling or not. The image recognition unit 44 determines whether the face image included in the gaze image is an image of the employee's face and whether the employee is smiling or not by using the facial image data of the employee stored in the storage unit 41. When the employee is not smiling, the control unit 46 determines that the user is in a dissatisfied state (negative state) where the employee is sad or angry, and when the employee is smiling, the control unit 46 determines that the user is in a satisfied state (positive state).

[0079] Furthermore, when the gaze image shows the user holding an item in his / her hand, the control unit 46 determines that the user's emotions have changed due to seeing or touching the item. The control unit 46 may also be able to determine whether the user is happy or sad by using, for example, the item information acquired by the basket device 20 in combination. For example, when the item information includes information such as limited quantity, it can be determined that the user is happy. Also, when it is found from the item information that the expiration date has passed, it can be determined that the user is sad.

[0080] Returning to FIG. 6, in the next step S56, the control unit 46 judges whether or not a response is necessary based on the emotion analysis result. Specifically, the control unit 46 judges that a response is necessary when the user is in a dissatisfied state (negative state), such as sad or angry. If the judgment in step S56 is negative, the process returns to step S50. On the other hand, if the judgment in step S56 is positive, the process proceeds to step S58.

[0081] When the process proceeds to step S58, the control unit 46 executes a response process based on the emotion analysis result. For example, when the gaze image is that of FIG. 4(b), the control unit 46 transmits various messages, such as an apology message or a next arrival schedule, stored in the storage unit 41 to the basket device 20 via the communication unit 42. Then, the control unit 26 of the basket device 20 transmits the various messages to the wearing device 10 via the first communication unit 24. In this case, in the wearing device 10, the control unit 17 displays (projects) the various messages using the display unit 12.

[0082] Also, for example, when the line-of-sight image is that of FIG. 8, the control unit 46 transmits a message of apology for the busy register to the control unit 17 of the mounting device 10 via the basket device 20. In this case, the control unit 17 displays (projects) the message of apology on the display unit 12. The control unit 46 may output the message of apology for the busy register by using an in-store announcement. The control unit 46 may also output a message (message of cashier support) to the effect that the cashier should be replenished by using an in-store announcement. Alternatively, the control unit 46 may output the message of cashier support to a terminal held by an employee. The control unit 46 may also notify a user who sees a situation where the cashier is busy of the progress of the cashier (processing time per customer) and the skill level of the clerk in charge of the cashier (apprentice or veteran).

[0083] Furthermore, for example, if it is determined from the gaze image that there is a problem with the employee's response, the control unit 46 outputs a message to a terminal held by another employee, the store manager, etc., to the effect that a problem has occurred between a customer and an employee. If the cart device 20 is provided with a position detection device capable of detecting a position according to the communication sensitivity with a wireless LAN base station, the control unit 46 can notify other employees, the store manager, etc. of the location of the problem based on the detection result of the position detection device.

[0084] Furthermore, for example, when it is determined from the gaze image that the customer is sad or angry about an item, the control unit 46 may output a message to the store via an announcement or a terminal held by the employee that the employee should directly handle the situation. In this case, the control unit 46 may also notify the employee of the customer's location.

[0085] The first information may include voice data collected by the microphone 14 immediately before the emotion change. When the voice data includes an angry voice or a sigh from the user, or an apology from an employee, the control unit 46 can determine by analyzing the voice data that the user is angry or in a dissatisfied state (in a negative state).

[0086] It is assumed that the above-mentioned response process patterns are determined in advance and stored in the storage unit 41.

[0087] After the process of step S58 is completed as described above, the process returns to step S50.

[0088] On the other hand, if the determination in step S52 is negative, that is, if the information input from the car device 20 is the second information, the process proceeds to step S60.

[0089] When the process proceeds to step S60, the control unit 46 executes a purchase behavior analysis based on the second information. In this case, the control unit 46 analyzes what the user looked at, picked up, and purchased while shopping. In other words, it can be said that the control unit 46 analyzes which products the user did not look at or pick up.

[0090] Fig. 9 shows the order of items picked up by a user from the time the user enters the store until the time the user goes through the cash register. In addition, in Fig. 9, the items purchased by the user are underlined. Note that the control unit 46 can use purchase information acquired from the cash register terminal as information on the items purchased by the user.

[0091] In FIG. 9, the user picks up curry roux A and curry roux B, but ultimately purchases curry roux A. In this case, the control unit 46 refers to the gaze image when the user picks up curry roux A and the gaze image when the user picks up curry roux B, and analyzes what criteria was used to determine whether curry roux A was purchased. For example, if price information is included for a long period of time in the area the user is gazing at (a predetermined position (almost the center) of the gaze image), it is determined that curry roux A was purchased based on the price. Also, if the area the user is gazing at includes a spiciness indication, a photo of the package, an ingredient indication, or an expiration date for a long period of time, it is determined that curry roux A was purchased based on that information. Note that the control unit 46 uses the OCR unit 43 to extract whether the area the user is gazing at includes a price, a spiciness indication, an ingredient, an expiration date, or the like.

[0092] Similarly, the control unit 46 may determine the purchase criteria for beef A and beef B. For example, when the area the user is gazing at includes the content amount, the part of the product, the price per 100 g, or the like for a long period of time, the control unit 46 determines that beef B was purchased based on that information.

[0093] Furthermore, by referring to data acquired from multiple users, the control unit 46 can compile information on what other products a user who purchased a certain product compared and considered, and what was the deciding factor for the product purchased as a result of comparison and consideration. The analysis results of the decision criteria at the time of purchase may be provided to the manufacturers or producers of each product. The manufacturers or producers can use the analysis results for future product development and product improvement.

[0094] Furthermore, the control unit 46 can tally up the order in which customers picked up items by referring to the data in Fig. 9. In this case, the control unit 46 can use the tallying results to analyze the tendency of customers' movements within the store. This allows the control unit 46 to determine an appropriate arrangement of items and shelves based on the movements of customers.

[0095] Furthermore, the control unit 46 can analyze the customer's satisfaction or dissatisfaction by location (sales area) by tallying up the number of times and locations where the user was in a positive state and the number of times and locations where the user was in a negative state between entering and leaving the store.

[0096] The control unit 46 may also analyze the gaze image when the customer changes from a negative state to a positive state. If the gaze image includes the face of an employee during the change from a negative state to a positive state as a result of analyzing the gaze image, the control unit 46 can analyze that the trouble was resolved by the response of the employee. Also, if a response process was performed using the wearing device 10 or an in-store announcement during the change from a negative state to a positive state, it can be analyzed that the response process was appropriate.

[0097] As described above in detail, according to the first embodiment, the control unit 46 and the communication unit 42 of the server 40 input the gaze image and the item information as the behavior information related to the user's purchase, and input the detection result of the biosensor 21 as the emotion information related to the user's purchase. This allows the control unit 46 to perform an analysis of the purchase based on the gaze image, the item information, and the detection result of the biosensor 21, and to acquire and provide useful information related to the purchase. Therefore, usability for the user can be improved. In this case, by using the gaze image captured from the user's side, it is possible to perform an analysis that takes into account the situation that the user is actually looking at, that is, the item, display, situation, etc. in front of the gaze, compared to the case where an image captured from the shelf side of the store is used.

[0098] Conventionally, when a user wearing a device equipped with an imaging unit 11 such as the wearable device 10 enters a store, the state of the store and the products are captured without the user's permission, which is not advantageous for the store. In contrast, in the first embodiment, instead of allowing the user to enter the store wearing the wearable device 10, the store receives image data captured by the imaging unit 11 via the basket device 20. This allows the store to analyze the user's actions and emotions leading up to the purchase of a product by analyzing the image data.

[0099] In addition, in the past, shopping satisfaction and the like were collected by questionnaires of users, but this was troublesome for the user as it required accessing the store's website to answer questions and writing down the answers on paper. In contrast, in the first embodiment, image data acquired by the wearing device 10 is linked to the user's biometric information and provided to the store, so that the user can unconsciously convey the user's satisfaction or dissatisfaction to the store.

[0100] In addition, in the first embodiment, messages to users and employees are output based on the gaze image, the product information, and the detection results of the biosensor 21. This makes it possible to immediately respond to users who feel dissatisfied.

[0101] In the first embodiment, the control unit 46 determines that the user has held an item from the gaze image and the item information. This allows the control unit 46 to acquire and provide useful information regarding purchases by analyzing information that the user has held an item and the order in which the user held the item.

[0102] In the first embodiment, the control unit 46 uses the detection result of the biosensor 21, and therefore can easily determine whether the user has experienced a change in emotion. Moreover, by analyzing the user's gaze image together with the detection result of the biosensor 21, it can accurately determine whether the user's emotion has become positive or negative.

[0103] In the first embodiment, the wearing device 10 captures a gaze image and transmits the captured gaze image to the basket device 20 in contact with the user via human body communication. This allows the basket device 20 to acquire only the gaze image of the user holding the basket 120. That is, it is possible to reliably acquire a gaze image from the wearing device 10 of a user who has entered the store, and reliably prevent acquisition of a gaze image from the wearing device 10 of a person who has not entered the store (for example, a person who is passing by the store).

[0104] In the first embodiment, when there is a change in the detection result of the biosensor 21, that is, when there is a change in the user's emotion, the first information is transmitted to the server 40 (S42). This allows information requiring analysis to be transmitted to the server 40 at an appropriate timing.

[0105] In the above first embodiment, the case where the basket device 20 is provided with the biosensor 21 has been described, but the present invention is not limited to this, and the biosensor may be provided in the wearing device 10. In this case, a biosensor that comes into contact with a part of the face may be provided on a part of the frame of the wearing device 10 or in the vicinity of the frame. Also, a biosensor that can be attached to an earlobe may be connected to the wearing device 10.

[0106] In the above first embodiment, the article device 30 is provided on the article 130. However, this is not limiting, and if article information is not required for the emotion analysis in step S54 or the purchase behavior analysis in step S60, the article device 30 does not necessarily need to be provided on the article 130. For example, if an article is captured in a gaze image in which the article is held in the hand, the article being held in the hand can be identified by performing image recognition.

[0107] In the above first embodiment, a case has been described in which the article device 30 performs human body communication with the car device 20, but this is not limiting, and the article device 30 may perform human body communication with the wearing device 10. In this case, when human body communication is established between the wearing device 10 and the car device 20, the article information acquired by the wearing device 10 may be transmitted from the wearing device 10 to the car device 20.

[0108] In the first embodiment, the detection result of the biosensor 21 is used, but the present invention is not limited to this, and the detection result of the biosensor 21 may not be used. In this case, the control unit 46 may estimate that the user is in a positive state when it is determined that the user is staring at the mark in FIG. 7(a) for a predetermined time based on the gaze image. Also, the control unit 46 may estimate that the user is in a negative state when it is determined that the user is staring at the notice in FIG. 7(b) or the waiting state in FIG. 7(c) for a predetermined time based on the gaze image. In this case, the control unit 46 may store in advance image patterns of marks, notices, and situations that are predicted to put the user in a positive or negative state when viewed by the user, and compare the image patterns with the gaze image.

[0109] In the above-mentioned first embodiment, a case where a predetermined position of the gaze image is regarded as the center position of the gaze has been described, but the present invention is not limited to this. For example, the wearing device 10 may be provided with a gaze detection unit that has an infrared ray irradiating unit and an infrared ray receiving unit, irradiates infrared rays from the infrared ray irradiating unit to the eyeball, and detects the gaze direction by receiving the infrared rays reflected by the eyeball at the infrared ray receiving unit.

[0110] The server 40 may also aggregate the satisfaction and dissatisfaction levels of customers at multiple stores and publish them as user ratings of each store on one website. In this case, for example, the ratings of each store can be published as a number of stars, a satisfaction score, or the like.

[0111] Second Embodiment Next, an electronic device system 100' according to a second embodiment will be described in detail with reference to FIGS.

[0112] A configuration of an electronic device system 100' is shown in a block diagram in Fig. 10. In Fig. 10, the configuration of the electronic device system 100' that is different from the electronic device system 100 of the first embodiment is indicated by a thick frame.

[0113] 10, in the electronic device system 100′, the mounting device 10′ includes a speaker 18, the basket device 20′ includes a weight sensor 27, and the article device 30′ includes a light-emitting unit 34. The configuration of the server 40′ is similar to that of the server 40 of the first embodiment.

[0114] The speaker 18 is used to provide audio information to a user wearing the wearing device 10'. As the speaker 18, in addition to earphones or headphones, a directional speaker capable of providing audio information only to a user wearing the wearing device 10' can be used. Note that the memory unit 15 of the wearing device 10' stores article information corresponding to the light emission pattern of a light emitting unit 34 of an article device 30' described later.

[0115] The weight sensor 27 is provided in a part of the basket device 20 ′, and is a sensor for detecting a change in the content amount of the basket 120 .

[0116] The light emitting unit 34 includes a light emitting element such as an LED (Light Emitting Diode) and emits light in a light emitting pattern that is predetermined for each item. The light emission of the light emitting unit 34 is controlled by the control unit 33. When an illumination pattern is captured within a predetermined range of the angle of view during video capture by the imaging unit 11, the control unit 17 of the wearing device 10' presumes that the user has seen an item corresponding to the illumination pattern. The timing of switching on and off the illumination of the light emitting unit 34 (the blinking interval of the illumination pattern) is set according to the frame rate of the imaging unit 11. Either a visible light LED or an infrared LED may be used as the light emitting unit 34. When an infrared LED is used, an infrared imaging element may be used for the imaging unit 11.

[0117] Next, the processing of the electronic device system 100' in the second embodiment will be described in detail with reference to the flowcharts of Figures 11 to 13 and other drawings as appropriate. Note that in the second embodiment as well, a case will be described in which the electronic device system 100' is used in a supermarket.

[0118] (Processing of mounting device 10') Fig. 11(a) shows a flowchart of the processing of the mounting device 10'. In Fig. 11(a), steps different from the processing of the mounting device 10 of the first embodiment (Fig. 4(a)) are indicated by bold frames.

[0119] 11A, the control unit 17 of the wearing device 10' waits until the human body communication with the car device 20' is established in step S10. When the human body communication with the car device 20' is established, the control unit 17 proceeds to step S110.

[0120] In step S110, the control unit 17 issues an instruction to the imaging unit 11 to capture an image (moving image) from the user side.

[0121] Next, in step S112, the control unit 17 judges whether or not the light emitted by the light emitting unit 34 has been captured within a predetermined range of the angle of view. In other words, it judges whether or not the user has seen the article on which the article device 30' is provided. If the judgment in step S112 is negative, the process proceeds to step S116, but if the judgment in step S112 is positive, the process proceeds to step S114.

[0122] When the process proceeds to step S114, the control unit 17 transmits the article information seen by the user to the basket device 20'. Specifically, the control unit 17 acquires, from the storage unit 15, article information corresponding to the light emission pattern detected within a predetermined range of the angle of view of the imaging unit 11, and transmits the article information to the first communication unit 24 of the basket device 20' via the communication unit 16.

[0123] After the determination in step S112 is negative or after the processing in step S114 is performed, the process proceeds to step S116, where the control unit 17 determines whether or not information has been received from the cart device 20'. The information received from the cart device 20' is information about goods and store services transmitted from the server 40'. If the determination here is negative, the process proceeds to step S16, but if the determination here is positive, the process proceeds to step S118.

[0124] When the process proceeds to step S118, the control unit 17 outputs the information received from the car device 20'. Details of the process of step S118 will be described later.

[0125] After the determination in step S116 is negative or after the process in step S118 is performed, the control unit 17 moves to step S16 and determines whether or not the human body communication with the basket device 20' to which the input from the cash register terminal has been established. Here, the input from the cash register terminal means the input of shopping completion information transmitted from the basket device 20' to the basket device 20' by communication between the basket device 20' and the cash register terminal when the user finishes shopping and takes the basket 120 to the cash register. In other words, when the human body communication between the basket device 20' to which the input from the cash register terminal has been established and the mounting device 10', it means that the user's shopping has been completed. Therefore, if the shopping has not been completed and the determination in step S16 is negative, the process returns to step S112 and the processing and determination in steps S112 to S16 are repeated. On the other hand, if the user finishes shopping and the determination in step S16 is positive, the entire process in FIG. 11(a) is completed.

[0126] (Processing of goods device 30') Fig. 11(b) is a flowchart showing the processing of the article processing device 30'. In Fig. 11(b), steps different from the processing of the article processing device 30 of the first embodiment (Fig. 4(b)) are indicated by bold frames.

[0127] As shown in Fig. 11(b), in step S120, in the article device 30', the control unit 33 causes the light emitting unit 34 to emit light. In this case, the control unit 33 causes the light emitting unit 34 to emit light in a light emitting pattern corresponding to the article on which the article device 30' is installed. The other processing is the same as that in Fig. 4(b).

[0128] (Processing of the cage device 20') Fig. 12 shows a flowchart of the processing of the basket device 20'. In the processing of Fig. 12, in step S130, the control unit 26 waits until intrabody communication with the wearing device 10' is established. When intrabody communication with the wearing device 10' is established and the process proceeds to step S132, the control unit 26 determines whether or not article information has been received from the wearing device 10'.

[0129] If the determination in step S132 is positive, the process proceeds to step S134, where the control unit 26 transmits the article information received from the mounting device 10' to the server 40' as article information of the viewed article, and then proceeds to step S136. On the other hand, if the determination in step S132 is negative, the process proceeds to step S136 without passing through step S134.

[0130] In step S136, the control unit 26 judges whether or not article information has been received from the article device 30'. If the judgment in step S136 is positive, the control unit 26 proceeds to step S138, where the control unit 26 transmits the article information received from the article device 30' to the server 40' as article information of the touched article, and proceeds to step S140. On the other hand, if the judgment in step S136 is negative, the control unit 26 proceeds to step S140 without passing through step S138.

[0131] When the process proceeds to step S140, the control unit 26 judges whether or not the detection value of the weight sensor 27 has changed. Here, when the detection value of the weight sensor 27 has changed, it means that an article has been newly put into the basket 120. Note that the detection value of the weight sensor 27 also changes when an article is removed from the basket 120, but for the sake of convenience, it is assumed here that an article is not removed from the basket 120. When the determination in step S140 is positive, the process proceeds to step S142, and the control unit 26 transmits the article information of the article that was touched immediately before to the server 40' as the article information of the article put into the basket, i.e., the information of the product that the user plans to purchase. On the other hand, when the determination in step S140 is negative, the process proceeds to step S144 without passing through step S142.

[0132] In step S144, the control unit 26 judges whether or not information has been received from the server 40'. The information received from the server 40' is information related to products and store services. If the judgment in step S144 is positive, the control unit 26 proceeds to step S146, where the control unit 26 transmits the information received from the server 40' to the wearing device 10', and proceeds to step S148. On the other hand, if the judgment in step S144 is negative, the control unit 26 proceeds to step S148 without going through step S146.

[0133] In step S148, control unit 26 determines whether or not there has been an input from the cash register terminal. Here, input from the cash register terminal means input of shopping completion information, as described above.

[0134] If the determination in step S148 is negative, the process proceeds to step S132, and the processes and determinations in steps S132 to S148 are repeated, but if the determination in step S148 is positive, the control unit 26 ends all the processes in FIG.

[0135] (Server 40' processing) FIG. 13 is a flow chart showing the processing of the server 40'.

[0136] 13, in step S150, the control unit 46 waits until it receives the article information from the basket device 20'. That is, when any one of the article information of the article seen by the user, the article information of the article touched by the user, and the article information of the article placed in the basket 120 is input, the control unit 46 proceeds to step S152.

[0137] When the process proceeds to step S152, the control unit 46 updates the shopping DB stored in the memory unit 41. Here, the shopping DB has a data structure as shown in Fig. 14(a). As shown in Fig. 14(a), it has columns for item name, seen, touched, and added to basket. When the control unit 46 receives item information for an item seen by the user, it stores a circle in the seen column for that item, when it receives item information for an item touched by the user, it stores a circle in the touched column for that item, and when it receives item information for an item added to the basket by the user, it stores a circle in the added to basket column for that item.

[0138] Next, in step S154, the control unit 46 analyzes the shopping DB. For example, the control unit 46 can refer to the shopping DB in FIG. 14(a) and analyze that the user may need information on an item (e.g., beef B) that can be compared with beef A when a predetermined time has passed since the user held beef A in his / her hand (when new item information is not received for a predetermined time). In addition, the control unit 46 can estimate that the user will make curry from the items placed in the basket, for example, and can analyze that the user may need information on the amount of beef required to make curry (Xg for 4 servings, Yg for 8 servings, etc.). In addition, the control unit 46 can analyze that the user may need information on bargains on side dishes for curry (e.g., pickled vegetables, pork cutlet, dessert, etc.) and campaign information for stores (e.g., if you buy Z yen more, you can win a lottery).

[0139] Next, in step S156, the control unit 46 judges whether or not there is information to be notified to the user as a result of the analysis in step S154. If the judgment here is negative, the process returns to step S150, but if the judgment here is positive, the process proceeds to step S158. Note that, in cases where the user touches the same item multiple times, or touches an item similar to an item previously touched, the same information as the information previously notified may be judged to be information to be notified to the user. Therefore, the control unit 46 may restrict the notification of information that is the same as information previously notified, by determining that it is not information to be notified to the user. That is, the control unit 46 may restrict the information to be notified when the same item is touched or seen multiple times, or when an item similar to an item previously touched or seen is touched or seen. In this way, it is possible to prevent the same information from being repeatedly notified to the same user.

[0140] The control unit 46 may change the information to be notified depending on the number of times the same item is touched or seen. For example, the control unit 46 may notify information about the item itself the first time, information about an item to be compared with the item the second time, and discount information related to the item the third time. In this way, it is possible to increase the possibility that a customer who is unsure about purchasing an item will purchase the item. Alternatively, the control unit 46 may notify the user of information that has not been notified to the user from the second time onwards depending on the time the user touched the same item.

[0141] If the determination in step S156 is positive, the control unit 46 proceeds to step S158 and transmits information to be notified to the user to the basket device 20'. If the process of step S158 is performed, the determination in step S144 in the process of FIG. 12 is positive, and the process of step S146 (process of transmitting information received from the server to the mounting device 10') is performed. If the process of step S146 is performed in FIG. 12, the determination in step S116 in FIG. 11(a) is positive, and the process of step S118 (process of outputting received information) is performed. In step S118, the control unit 17 uses the display unit 12 to display information to be notified to the user, or uses the speaker 18 to output information to be notified to the user. After step S158, the process returns to step S150, and the processes of steps S150 to S158 are repeated.

[0142] The control unit 46 may perform various analyses based on the shopping DB (for example, FIG. 14(b)) of the user who has finished shopping. In this case, the control unit 46 may compare curry roux A and curry roux B, and beef A and beef B, and analyze that curry roux B and beef B were purchased, for example, based on FIG. 14(b). The control unit 46 may also analyze the order in which people walk around the store and the order of purchases. The control unit 46 may also identify and analyze items that were looked at but not purchased, and items that were picked up but not purchased. The control unit 46 may provide the analysis results to the manufacturer, distributor, etc. of the product.

[0143] As described above in detail, according to the second embodiment, the control unit 46 inputs the item information of the item in response to at least one of the user picking up the item and the item entering a predetermined range within the user's field of vision (S150: Yes), and notifies the user of the input item information (S158). Therefore, in the second embodiment, it is possible to notify the user of useful information related to the item the user touched or saw. This can improve usability for the user.

[0144] In the second embodiment, the control unit 46 inputs information on the product that the user plans to purchase and that is placed in the basket 120 as the item information, and issues a notification according to the input product. This makes it possible to notify the user of useful information related to the product that the user plans to purchase. In addition, since the control unit 46 inputs information on the product that the user plans to purchase from the basket device 20' used when shopping, it is possible to input accurate information as information on the product that the user plans to purchase.

[0145] Furthermore, in the second embodiment, information is notified to the user through the display unit 12 of the wearing device 10' and the directional speaker 18, so that notification can be given to each user according to the purchasing behavior of that user without being known to other users.

[0146] In the second embodiment, when a user is holding an item in his / her hand and looking at another item, the control unit 46 may determine that the user is comparing and considering both products.

[0147] In the above second embodiment, the electronic device system 100' is used in a supermarket. However, the electronic device system 100' may be used in a clothing store or the like. For example, when a user picks up clothes, the control unit 46 can notify only a specific user (a frequent customer such as a member of the store) without being known to other users by using the display unit 12 and the speaker 18 to inform the user of information such as availability of different colors, different sizes, etc., information on whether similar products are on sale, discount information such as discounts for buying one more item, discounts for buying 2000 yen or more, etc. When a user who is a member of the store visits the store on his / her birthday, the server 40' may transmit discount information, gift information, and a congratulatory message to the wearing device 10'.

[0148] In the above second embodiment, the control unit 17 transmits the article information acquired in the mounting device 10' to the server 40' via the basket device 20', but the present invention is not limited to this. For example, if the mounting device 10' has a communication unit capable of wireless communication, the information may be transmitted from the mounting device 10' to the server 40' via wireless communication.

[0149] In the above second embodiment, the case where the article device 30' has the light-emitting unit 34 has been described, but the present invention is not limited to this, and the article device 30' does not need to have the light-emitting unit 34. In this case, the control unit 17 may identify the article viewed by the user by analyzing the image captured by the imaging unit 11. For example, the control unit 17 may identify the article viewed by the user by recognizing characters included in the display (for example, characters on a product tag or a POP advertisement).

[0150] In the first and second embodiments, the control unit 46 may take the type of user into consideration when analyzing information. If the user is a member of the store, the type of user can be determined from registered information (name, date of birth, address, shopping history). In this case, the purchasing behavior of the user may be analyzed according to the user's gender and age group.

[0151] The above-described embodiments are preferred examples of the present invention and can be combined as appropriate. However, the present invention is not limited to these, and various modifications can be made without departing from the scope of the present invention. [Explanation of symbols]

[0152] 10. Mounting device 11 Imaging unit 12 Display section 16 Communications Department 17 Control Unit 18 Speaker 20 Cage device 42 Communications Department 46 Control Unit 110 Frames 120 Basket

Claims

[Claim 1] A detection unit that detects entry of a user into a store based on information from a device carried by the user; an image recognition unit that receives image information captured within the store and recognizes items within the store; an acquisition unit that acquires information about the item recognized by the image recognition unit; A transmitting unit that transmits information regarding the item acquired by the acquiring unit to be displayed on the possessed device of the user.

Images