Information analysis system

Abstract

This information analysis system comprises: a first movement information acquisition unit that acquires first movement information, which has been created, by using a first sensor attached to the head of a person, to represent movement of the head; a first position information acquisition unit that acquires first position information, which is position information of the person that has been created by using a position information acquisition unit attached to the person; a second movement information acquisition unit that acquires second movement information, which has been created, by using a second sensor different from the first sensor, to represent the movement of the head; a second position information acquisition unit that acquires second position information, which is position information that has been created by using the second sensor different from the first sensor; a movement information correction unit that corrects the first movement information by using the second movement information; and a position information correction unit that corrects the first position information by using the second position information.

Description

Information Analysis System

[0001] The present disclosure relates to an information analysis system.

[0002] Conventionally, attempts have been made to detect a person's state, such as the direction of their gaze, in order to provide services, for example. For example, in Patent Document 1, the direction of a person's gaze is detected in order to provide audio guidance of an object that the person is gazing at.

[0003] JP 2015-079310 A

[0004] However, when detecting the direction of a person's face using a sensor attached to the head, errors are likely to occur depending on how the sensor is attached.

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to one aspect of the present disclosure, there is provided an information analysis system. The information analysis system includes a first motion information acquisition unit that acquires first motion information representing a head motion created using a first sensor worn on a person's head, a first position information acquisition unit that acquires first position information representing the person's position created using a position information acquisition unit worn by the person, a second motion information acquisition unit that acquires second motion information representing the head motion created using a second sensor different from the first sensor, a second position information acquisition unit that acquires second position information representing the position information created using the second sensor different from the first sensor, a motion information correction unit that corrects the first motion information using the second motion information, and a position information correction unit that corrects the first position information using the second position information. According to this aspect, it is possible to improve the accuracy of the first motion information and the first position information. (2) In the information analysis system of the above aspect, the second sensor used to create the second motion information may be the same image sensor as the second sensor used to create the second position information, and the second motion information and the second position information may each be information created using an image including the person captured by an imaging device having the second sensor, the installation location of which is known. According to this aspect, by using an image captured by an imaging device having a known installation location, the second motion information and the second position information, which have less error, can be used for correction. Thus, the accuracy of the first motion information and the first position information can be improved. (3) In the information analysis system of the above aspect, the system may further include a creation unit that uses the first motion information and the first position information to identify an object toward which the person is facing and create object information including the object. According to this aspect, the creation unit can create the object information. (4) In the information analysis system of the above aspect, the creation unit may create at least one of (a) object history information in which the objects are arranged in chronological order in the order in which the person turns their face, and (b) object history information in which the objects are arranged in descending order of the length of time the person looks at the objects. According to this aspect, the creation unit can create the object history information.(5) In the information analysis system of the above aspect, the object may be a product, and the information analysis system may further include an information addition unit that adds identification information of the product to the object information. According to this aspect, the additional information unit can add the identification information of the product to the object information. The object information to which the identification information of the product has been added can be used to analyze a person's behavior when selecting a product. (6) In the information analysis system of the above aspect, the object may be an advertisement, and the information analysis system may further include an information addition unit that adds identification information of the advertisement to the object information. According to this aspect, the additional information unit can add identification information of the advertisement to the object information. The object information to which the identification information of the advertisement has been added can be used to analyze the impact of the advertisement on a person. (7) In the information analysis system of the above aspect, the first sensor may be a motion sensor built into the ear-worn device. According to this aspect, a motion sensor built into the ear-worn device can be used as the first sensor. (8) The information analysis system of the above aspect may further include a psychological information acquisition unit that acquires psychological information representing the psychological state of the person, and an information addition unit that adds the psychological information to the object information. According to this aspect, the information addition unit can add the psychological information to the object information. (9) In the information analysis system of the above aspect, the psychological information may include at least one of the person's heart rate and information on the person's facial color. According to this aspect, either the person's heart rate or information on the person's facial color can be used as the psychological information. (10) In the information analysis system of the above aspect, the information addition unit may use the psychological information to calculate an evaluation value that indicates the person's level of interest in the object, and add the evaluation value to the object information. According to this aspect, the evaluation value can make it easier to analyze the impact of a product on the person. (11) The information analysis system of the above aspect may further include a sound information acquisition unit that acquires sound information estimated to be heard by the person, and an information addition unit that adds the sound information to the object history information. According to this aspect, the information addition unit can add sound information to the object history information.The present disclosure can be realized in various forms, and in addition to the information analysis system described above, can be realized in the form of, for example, an information analysis method, an information analysis program, or the like.

[0007] FIG. 1 is a block diagram showing the configuration of an information analysis system. FIG. 1 is a diagram explaining a purchaser carrying an ear-worn device and a mobile terminal device. FIG. 2 is a block diagram showing the configuration of an ear-worn device and the configuration of a mobile terminal device. FIG. 3 is a diagram explaining detection by a motion sensor. FIG. 4 is a block diagram showing the configuration of an image analysis device. FIG. 5 is a diagram explaining information correction processing. A flowchart showing the procedure of information correction processing. FIG. 6 is a diagram explaining correction of first action information. FIG. 7 is a block diagram showing the configuration of an information analysis system according to a second embodiment. A flowchart showing the procedure of purchase information creation processing. A diagram showing purchase information. A flowchart showing the procedure of information addition processing. A diagram showing product information.

[0008] A. First Embodiment: A1. Configuration of Information Correction System: Fig. 1 is a block diagram showing the configuration of an information analysis system 1. Fig. 2 is a diagram illustrating a purchaser PU carrying an ear-worn device 80 and a mobile terminal device 90 that are the targets of processing by the information analysis system 1. As will be described later, the ear-worn device 80 has a built-in motion sensor 81 as a first sensor. The information analysis system 1 corrects errors in, for example, first movement information 31, which is created using detection values ​​output by the motion sensor 81. This can improve the accuracy of analysis of the movement of the purchaser PU, for example, using the first movement information 31.

[0009] As shown in FIG. 1 , the information analysis system 1 is configured as a computer in which a processor 20, a storage unit 30, a communication interface 40, and an input / output interface 50 are interconnected by a bus. A display unit 51, such as a liquid crystal display, and an input device 52, such as a keyboard or a mouse, are connected to the input / output interface 50. The processor 20 performs various processes by executing a program stored in the storage unit 30. The processor 20 includes, as functional units, an information creation unit 21, a start determination unit 22, a first motion information acquisition unit 23, a first position information acquisition unit 24, a second motion information acquisition unit 25, a second position information acquisition unit 26, a motion information correction unit 27, and a position information correction unit 28. Each functional unit is realized by executing a program stored in the storage unit 30. The motion information correction unit 27 corrects the first motion information 31 using the second motion information 33. The position information correction unit 28 corrects the first position information 32 using the second position information 34.

[0010] The storage unit 30 is realized by a memory such as RAM or ROM. The storage unit 30 stores first motion information 31, first position information 32, second motion information 33, and second position information 34. The communication interface 40 communicates with external devices such as a mobile terminal device 90 and an image analysis device 60 (described later) via a network such as the Internet. The first motion information 31 is information representing head motion generated using a motion sensor 81 (a first sensor) worn on the head of a person, specifically, a purchaser PU. The first position information 32 is position information of the purchaser PU generated using a position information acquisition unit 94 worn on the person, specifically, the purchaser PU. In this disclosure, "worn by a person" includes a state in which the device is fixed to the person's body and a state in which the device is carried by the person. Specific examples of "carried by a person" include a state in which the device is held in the person's hand or placed in a bag worn by the person. As described below, in this embodiment, the motion sensor 81 is built into the ear-worn device 80. The position information acquisition unit 94 is built into the mobile terminal device 90 carried by the purchaser PU. The second operation information 33 is information representing the operation of the purchaser PU created using an image sensor 71 as a second sensor different from the motion sensor 81. The second position information acquisition unit 26 is the position information of the purchaser PU created using an image sensor 71 as a second sensor different from the motion sensor 81.

[0011] The information analysis system 1 communicates with each of the mobile terminal device 90 and the image analysis device 60. As shown in Fig. 2, the mobile terminal device 90 is carried by a purchaser PU. The purchaser PU carrying the mobile terminal device 90 also wears an ear-worn device 80 on his or her ear.

[0012] Fig. 3 is a block diagram showing the configuration of the ear-worn device 80 and the configuration of the mobile terminal device 90. The ear-worn device 80 has a pair of housings 80a shown in Fig. 2 each equipped with a motion sensor 81, a sound conversion unit 82, a microphone 83, and a communication interface 84 shown in Fig. 3. The motion sensor 81 and the microphone 83 may be equipped in both of the pair of housings 80a, or may be equipped in either one of the housings 80a.

[0013] The motion sensor 81 detects the movement of the head of the purchaser PU. Specifically, the motion sensor 81 is a general-purpose so-called six-axis motion sensor. Specifically, the motion sensor 81 includes a three-axis acceleration sensor and a three-axis angular velocity sensor. The motion sensor 81 detects the acceleration [m / s ２ ] and the angular velocity [deg / s] of each of the three axes.

[0014] FIG. 4 is a diagram illustrating detection by the motion sensor 81. As shown in FIG. 4, the three axes along which the motion sensor 81 detects acceleration and angular velocity are, specifically, the yaw axis, roll axis, and pitch axis, which are perpendicular to each other. The yaw axis is an axis parallel to the up-down direction relative to the customer PU. The roll axis is an axis parallel to the front-back direction relative to the customer PU. The pitch axis is an axis parallel to the left-right direction relative to the customer PU. With respect to the acceleration along the yaw axis, the motion sensor 81 outputs a positive value when the customer PU moves their head up, and a negative value when the customer PU moves their head down. With respect to the acceleration along the roll axis, the motion sensor 81 outputs a positive value when the customer PU moves their head forward, and a negative value when the customer PU moves their head back. With respect to the acceleration along the pitch axis, the motion sensor 81 outputs a positive value when the customer PU moves their head to the left, and a negative value when the customer PU moves their head to the right. The motion sensor 81 outputs a positive value for the angular velocity of the yaw axis when the consumer PU turns his / her face to the right, and a negative value when the consumer PU turns his / her face to the left. The motion sensor 81 outputs a positive value for the angular velocity of the roll axis when the consumer PU tilts his / her head to the left, and a negative value when the consumer PU tilts his / her head to the right. The motion sensor 81 outputs a positive value for the angular velocity of the pitch axis when the consumer PU turns his / her face down, and a negative value when the consumer PU turns his / her face up. Therefore, head movement can be detected from changes in acceleration and angular velocity on each of the three axes.

[0015] The sound conversion unit 82 shown in Figure 3 converts sound data, which is an electrical signal, into sound and emits it toward the ear of the purchaser PU. The microphone 83 converts the sound into sound data, which is an electrical signal. The microphone 83 outputs the acquired sound data to the communication interface 84. The communication interface 84 performs short-range communication with the mobile terminal device 90. In this embodiment, the communication interface 84 and the mobile terminal device 90 communicate in accordance with Bluetooth (registered trademark). The communication interface 84 is realized by an electronic circuit or the like.

[0016] In this embodiment, the mobile terminal device 90 is a smartphone. The mobile terminal device 90 includes a processor 91, a storage unit 92, a display operation unit 93, a position information acquisition unit 94, a sensor unit 95, and a communication interface 96.

[0017] The processor 91 performs various processes by executing programs stored in the storage unit 92. The storage unit 92 is realized by a memory such as a RAM or a ROM. An information provision application program 97 is stored in the storage unit 92. The storage unit 92 also stores first motion information 31 and first position information 32. The display operation unit 93 has a touch panel. The display operation unit 93 displays information on the touch panel and accepts operations input to the touch panel.

[0018] The position information acquisition unit 94 has a GNSS (Global Navigation Satellite System) receiver (not shown) and receives signals from positioning satellites under the control of the processor 91. In this embodiment, the position information acquisition unit 94 uses the GPS (Global Positioning System) as the GNSS. The position information acquisition unit 94 may acquire position information using other satellite positioning systems such as the Quasi-Zenith Satellite System. The sensor unit 95 includes a geomagnetic sensor and an acceleration sensor. The position information acquisition unit 94 may correct the position information using a detection value from the sensor unit 95, radio waves from a base station of a mobile communication system, radio waves from an access point of a wireless LAN (Local Area Network), etc.

[0019] The communication interface 96 performs short-range communication with the communication interface 84 of the ear-worn device 80. The communication interface 96 also performs communication with the communication interface 40 of the information analysis system 1 using a mobile communication system such as 5G (5th Generation Mobile Communication System). The communication method between the communication interface 96 and the communication interface 40 of the information analysis system 1 is not limited to a mobile communication system, but may also be a method using a wireless LAN, or a combination of these. The communication between the ear-worn device 80 and the mobile terminal device 90 is not limited to wireless communication, but may also be wired communication.

[0020] The processor 91 of the mobile terminal device 90 executes the information provision application program 97 to sequentially transmit the first movement information 31 acquired by the motion sensor 81 and the first position information 32 acquired by the position information acquisition unit 94 to the information analysis system 1. In other words, the first movement information 31 is information representing the movement of the customer PU, i.e., the person's head. In this embodiment, the first movement information 31 includes the detection value of the motion sensor 81 and a current angle from a zero reference position created by the information provision application program 97 using the detection value of the motion sensor 81. The first position information is the position information of the person, specifically the customer PU. The information analysis system 1 stores the received first movement information 31 and first position information 32 in the storage unit 30.

[0021] FIG. 5 is a block diagram showing the configuration of the image analysis device 60. As shown in FIG. 2, in this embodiment, the image analysis device 60 is installed in a store ST in which an imaging device 70 is also installed. Product shelves SH are arranged in the store ST. Products ME are displayed on the product shelves SH. Specifically, the imaging device 70 is a camera having an image sensor 71. The image sensor 71 is a sensor that converts light into an electrical signal, such as a CCD (Charge-Coupled Device) image sensor. The imaging device 70 is installed so as to be able to capture images of the products ME and the purchaser PU. Note that while FIG. 2 shows only one imaging device 70, multiple imaging devices 70 may be installed in the store ST.

[0022] 5 , the image analysis device 60 is configured as a computer in which a processor 61, a storage unit 63, and a communication interface 66 are interconnected by a bus. The processor 61 performs various processes by executing programs stored in the storage unit 63. The processor 61 has an image analysis unit 62 as a functional unit. The image analysis unit 62 is realized by executing the programs stored in the storage unit 63.

[0023] The storage unit 63 is realized by a memory such as a RAM or a ROM. The second operation information 33 and the second position information 34 are stored in the storage unit 63. The communication interface 66 and the imaging device 70 are connected via a video cable. The communication interface 66 communicates with the communication interface 40 of the information analysis system 1 using a mobile communication system such as 5G. The communication method between the communication interface 66 and the communication interface 40 of the information analysis system 1 is not limited to a mobile communication system, and may be a method using a wireless LAN, or a combination of these.

[0024] The image analysis unit 62 generates second movement information 33 and second position information 34 using images including the customer PU captured using an imaging device 70 whose installation location is known. Specifically, the image analysis unit 62 analyzes the image of the customer PU captured by the imaging device 70 and generates second movement information 33 representing the head movement of the customer PU. In this embodiment, the second movement information 33 is information equivalent to the detection value of the motion sensor 81 or information that can be converted into angle information calculated from the angular velocity output by the motion sensor 81. The image analysis unit 62 analyzes the image of the customer PU captured by the imaging device 70 and generates second position information 34 representing the position information of the customer PU. Date and time information is associated with the movement information in the second movement information 33. Date and time information is associated with the position information in the second position information 34. As described above, when multiple imaging devices 70 are installed, the image analysis unit 62 may analyze multiple images captured by the multiple imaging devices 70.

[0025] The image analysis unit 62 sequentially transmits the created second movement information 33 and second position information 34 to the information analysis system 1. The information analysis system 1 stores the received second movement information 33 and second position information 34 in the storage unit 30.

[0026] A2. Information Correction Process: Fig. 6 is a diagram illustrating the information correction process. Fig. 6 is a map showing the layout of a plurality of stores ST. Among the plurality of stores ST, the store ST in which the imaging device 70 and the image analysis device 60 are located is referred to as the target store OST.

[0027] The zero reference position of the motion sensor 81 built into the ear-worn device 80 worn by the consumer PU is adjusted using the mobile terminal device 90 when the consumer PU wears the ear-worn device 80. As described above, the information provision application program 97 uses the detected angular velocity value from the motion sensor 81 to determine the current angle relative to the zero reference position and create the first motion information 31. The information provision application program 97 then adjusts the angle of each of the three axes of the motion sensor 81 to "zero" when the consumer PU is not tilting his / her head. However, depending on the wearing state of the ear-worn device 80, for example, the angles of each of the three axes may not be "zero" even when the consumer PU is not tilting his / her head. In this case, an error occurs between the actual posture of the consumer PU's head and the angle determined from the detected value of the motion sensor 81. Furthermore, an error may occur between the first position information 32 and the actual current position. Therefore, the information analysis system 1 performs an information correction process to correct the first movement information 31 and the first position information 32. This can improve the accuracy of the first movement information 31 obtained using the motion sensor 81 relative to the actual posture. Also, it can improve the accuracy of the first position information 32 relative to the actual current position.

[0028] Furthermore, by making the correction, it is possible to improve the accuracy of the purchase information 35 described in the second embodiment when the purchaser PU enters a store ST that is not the target store OST. Furthermore, by making the correction, it is possible to improve the quality of services, such as providing guidance on an object that the purchaser PU is presumed to be looking at.

[0029] 7 is a flowchart showing the steps of the information correction process. The start determination unit 22 determines whether the purchaser PU has entered the target store OST. Specifically, if it is determined using the first location information 32 that the current location of the purchaser PU is within a predetermined reference location range, it is determined that the purchaser PU has entered the target store OST. Here, the reference location range is a location range that indicates the area within the target store OST. When the start determination unit 22 determines that the purchaser PU has entered the target store OST, the information correction process is initiated.

[0030] In step S1, the motion information correcting unit 27 determines whether the first motion information 31 matches the second motion information 33 with respect to the angle of the yaw axis. In step S1, the motion information correcting unit 27 calculates a first angular difference D1 using the following formula (1): D1 = θ1a - θ1b (1) "θ1a" in formula (1) is the angle of the yaw axis of the first motion information 31. That is, "θ1a" in formula (1) is the angle of the yaw axis calculated from the detection value of the motion sensor 81. "θ1b" in formula (1) is the angle of the yaw axis of the second motion information 33. That is, "θ1b" in formula (1) is the angle of the yaw axis calculated from the image captured by the imaging device 70. If the first angular difference D1 is within a predetermined first reference range, the motion information correcting unit 27 determines that the first motion information 31 matches the second motion information 33. On the other hand, when the first angular difference D1 is outside the first reference range, the motion information correcting unit 27 determines that the first motion information 31 does not match the second motion information 33. The first reference range is a range determined by an experiment or the like.

[0031] If it is determined in step S1 that the first movement information 31 does not match the second movement information 33 with respect to the angle of the yaw axis, then in step S3, the movement information correcting unit 27 corrects the zero reference position of the yaw axis in the first movement information 31. Specifically, the movement information correcting unit 27 sends a command signal to the mobile terminal device 90 to set a position obtained by subtracting the first angular difference D1 from the current reference position as a new zero reference position. Upon receiving the command signal, the mobile terminal device 90 uses the first angular difference D1 to correct the zero reference position of the first movement information 31, which is created using the detection value from the motion sensor 81.

[0032] 8 is a diagram illustrating step S3. In step S3, the zero reference position is corrected to a position indicated by a dashed line, rotated by a first angular difference D1, which is the angular difference between the yaw axis angle θ1a of the first movement information 31 and the yaw axis angle θ1b of the second movement information 33. In other words, the dashed arrow in FIG. 8 indicates the new zero reference position. By correcting the zero reference position in step S3, the accuracy of the yaw axis angle in the first movement information 31 can be improved thereafter.

[0033] In step S1 of Figure 7, if it is determined that the first movement information 31 matches the second movement information 33 with respect to the yaw axis angle, no correction is necessary, so the movement information correction unit 27 proceeds to step S5.

[0034] In step S5, the motion information correcting unit 27 determines whether or not the first motion information 31 matches the second motion information 33 with respect to the angle of the pitch axis. Step S5 is performed in the same manner as step S1, with the pitch axis as the processing target. If it is determined in step S5 that the first motion information 31 does not match the second motion information 33 with respect to the angle of the pitch axis, then in step S7 the motion information correcting unit 27 corrects the zero reference position of the pitch axis in the first motion information 31. Step S7 is performed in the same manner as step S3, with the pitch axis as the processing target. If it is determined in step S5 that the first motion information 31 matches the second motion information 33 with respect to the angle of the pitch axis, the motion information correcting unit 27 proceeds to step S9.

[0035] In step S9, the motion information correcting unit 27 determines whether the first motion information 31 matches the second motion information 33 with respect to the angle of the roll axis. Step S9 is performed in the same manner as step S1, with the roll axis as the processing target. If it is determined in step S9 that the first motion information 31 does not match the second motion information 33 with respect to the angle of the roll axis, then in step S11, the motion information correcting unit 27 corrects the zero reference position of the roll axis in the first motion information 31. Step S11 is performed in the same manner as step S3, with the roll axis as the processing target. If it is determined in step S9 that the first motion information 31 matches the second motion information 33 with respect to the angle of the roll axis, the motion information correcting unit 27 proceeds to step S13.

[0036] In step S13, the position information correcting unit 28 determines whether the first position information 32 matches the second position information 34. If the position information is expressed in an XYZ coordinate system, for example, the position information correcting unit 28 determines whether the first position information 32 matches the second position information 34 for each of the X coordinate, the Y coordinate, and the Z coordinate. If at least one of the X coordinate, the Y coordinate, and the Z coordinate does not match, the position information correcting unit 28 determines that the first position information 32 does not match the second position information 34. If it is determined in step S13 that the first position information 32 does not match the second position information 34, in step S15, the position information correcting unit 28 corrects the first position information 32 so that the first position information 32 matches the second position information 34. In step S15, specifically, position information corrector 28 transmits a command signal to portable terminal device 90 to set the corrected coordinates to be values ​​obtained by subtracting the difference between first position information 32 and second position information 34 from first position information 32, as in the case of first operation information 31. Upon receiving the command signal, portable terminal device 90 corrects first position information 32.

[0037] If it is determined in step S13 that the first position information 32 matches the second position information 34, or after step S15 has been performed, the motion information correcting unit 27 ends this processing routine.

[0038] According to the first embodiment described above, the information analysis system 1 includes a first movement information acquisition unit 23, a first position information acquisition unit 24, a second movement information acquisition unit 25, a second position information acquisition unit 26, a movement information correction unit 27, and a position information correction unit 28. The movement information correction unit corrects the first movement information 31 using the second movement information 33. The position information correction unit 28 corrects the first position information 32 using the second position information 34. This makes it possible to improve the accuracy of the first movement information 31 and the first position information 32.

[0039] Furthermore, each of the second operation information 33 and the second position information 34 is information created using an image including a person captured by the imaging device 70 whose installation position is known. By using an image captured by the imaging device 70 whose installation position is known, the second operation information 33 and the second position information 34 with less error can be used for correction.

[0040] B. Second Embodiment: Fig. 9 is a block diagram showing the configuration of an information analysis system 1 according to this embodiment. As shown in Fig. 9, the processor 20 differs from the first embodiment in that it has, as functional units, a purchase information creation unit 121 as a creation unit and an information addition unit 122. The memory unit 30 differs from the first embodiment in that it stores purchase information 35 as object information and object history information, and product information 36. The same configurations and processing steps as those in the first embodiment are denoted by the same reference numerals, and detailed descriptions thereof will be omitted where appropriate.

[0041] The purchase information creation unit 121 uses the first action information 31 and the first position information 32 to identify the object at which the person turns their head, and creates purchase information 35 as object information including the object. In this embodiment, the object is specifically a product ME displayed on the product shelf SH shown in FIG. 2. In this embodiment, the purchase information 35 also serves as object history information that chronologically arranges the objects in the order in which the person turns their head. The purchase information 35 makes it possible to know which position on the product shelf SH the consumer PU shown in FIG. 2 looked at when selecting a product ME in the store ST. The purchase information 35 can be used as information for considering the purchase of the product ME or the location of the product ME.

[0042] In this embodiment, in the purchase information creation process described later, the object that the purchaser PU is facing is identified as the position information of the product shelf SH. Then, the information addition unit 122 adds a product ID as identification information of the product ME to the purchase information 35 using the position information of the product shelf SH and the product information 36.

[0043] FIG. 11 is a diagram showing purchase information 35. The purchase information 35 is data in a table format in which the following items are grouped together: "date and time," "location," "visual inspection time," and "product ID." The "location" item further includes the items "shelf ID," "tier," and "X position." The "shelf ID" item is entered with an identifier that has been assigned in advance to the product shelf SH of the store ST shown in FIG. 2. The "tier" item is entered with a number indicating the tier from the bottom of the product shelf SH. The "X position" item is entered with the dimension based on the left edge, which is the position on the same tier. The other items will be described later.

[0044] FIG. 10 is a flowchart showing the steps of the purchase information creation process. When the start determination unit 22 determines that the purchaser PU has entered a store ST, the purchase information creation process shown in FIG. 10 is started. In step S31, the start determination unit 22 determines whether the store ST that the purchaser PU has entered is a target store OST. When the start determination unit 22 determines that the store ST that the purchaser PU has entered is a target store OST, information correction processing is performed in step S33. The information correction processing is performed in the same manner as in the first embodiment. When the start determination unit 22 determines that the store ST that the purchaser PU has entered is not a target store OST, processing proceeds to step S35.

[0045] In step S35, the purchase information creation unit 121 determines whether the detected speed Sm, which is obtained from the acceleration detection value of the motion sensor 81, is equal to or less than a reference speed Smth. The reference speed Smth is a speed used to determine whether the purchaser PU has stopped to select a product ME, and is determined in advance through experiments, etc. In step S35, if the purchase information creation unit 121 determines that the detected speed Sm is not equal to or less than the reference speed Smth, the process proceeds to step S47.

[0046] If it is determined in step S35 that the detected speed Sm is equal to or less than the reference speed Smth, then in step S37 the purchasing information creation unit 121 determines whether the detected angular speed Sa detected by the motion sensor 81 is equal to or less than the reference angular speed Sath. The reference angular speed Sath is an angular speed used to determine whether the purchaser PU is gazing at one product ME in order to select a product ME, and is determined in advance through experiments, etc. In step S37, if all of the angular velocities about the three axes are equal to or less than the reference angular speed Sath, the purchasing information creation unit 121 determines that the detected angular speed Sa is equal to or less than the reference angular speed Sath.

[0047] If it is determined in step S37 that the detected angular velocity Sa is not equal to or less than the reference angular velocity Sath, the purchase information creation unit 121 proceeds to step S47.

[0048] If it is determined in step S37 that the detected angular velocity Sa is equal to or less than the reference angular velocity Sath, then in step S39, the purchase information creation unit 121 identifies the location where the customer PU is facing, i.e., the location where the customer PU is presumed to be looking. Specifically, the purchase information creation unit 121 identifies the location where the customer PU is facing, using the first action information 31, the first position information 32, the facility information of the store ST, and the physical information of the customer PU. The facility information of the store ST is information such as the shelf ID, position, dimensions, and number of shelves of the product shelf SH in the store ST. Note that the "shelf ID" is identification information that is assigned in advance to the product shelf SH in the store ST and identifies the product shelf SH. The physical information of the customer PU is the height of the customer PU, etc. The facility information of the store ST and the physical information of the customer PU are pre-stored in the storage unit 30. The location where the face is turned is specified by the values ​​of the "shelf ID", "tier" and "X position" items in the purchase information 35 described above.

[0049] In step S41, the purchase information creation unit 121 determines whether the detected angular velocity Sa is greater than the reference angular velocity Sath. Step S41 is a processing step for determining whether the consumer PU has shifted their gaze to a different product ME from the product ME they had been gazing at. In step S41, if the angular velocity of at least one of the three axes is greater than the reference angular velocity Sath, the purchase information creation unit 121 determines that the detected angular velocity Sa is greater than the reference angular velocity Sath.

[0050] In step S41, if it is determined that the detected angular velocity Sa is not greater than the reference angular velocity Sath, the purchase information creation unit 121 repeatedly executes step S41 at predetermined time intervals until it determines that the detected angular velocity Sa is greater than the reference angular velocity Sath. The predetermined time is, for example, about several milliseconds.

[0051] If it is determined in step S41 that the detected angular velocity Sa is greater than the reference angular velocity Sath, then in step S43 the purchasing information creation unit 121 adds information to the purchasing information 35. Specifically, the purchasing information creation unit 121 inputs values ​​into each field of the purchasing information 35 to add one row. The date and time at which it was determined in step S37 that the detected angular velocity Sa is equal to or less than the reference angular velocity Sath is input into the "date and time" field of the purchasing information 35. The time from the time at which the purchasing information creation unit 121 determined in step S37 that the detected angular velocity Sa is equal to or less than the reference angular velocity Sath to the time at which it determined in step S41 that the detected angular velocity Sa is greater than the reference angular velocity Sath is input into the "visual observation time" field of the purchasing information 35. The value identified in step S39 is input into the "location" field of the purchasing information 35. A null value is input into the "product ID" field of the purchasing information 35. That is, the purchase information 35 is history information in which locations as objects are arranged in chronological order in the order that the purchaser PU turned his / her face.

[0052] In step S45, the purchase information creation unit 121 determines whether the detected speed Sm, which is calculated from the acceleration detection value of the motion sensor 81, is greater than the reference speed Smth. Step S45 is a processing step for determining whether the consumer PU has moved. If it is determined in step S45 that the detected speed Sm is not greater than the reference speed Smth, the purchase information creation unit 121 returns the process to step S37. This is because in this case, the consumer PU is likely to look at products ME displayed on the same product shelf SH without moving.

[0053] If it is determined in step S45 that the detected speed Sm is greater than the reference speed Smth, then in step S47 the purchase information creation unit 121 determines whether the purchaser PU has left the store ST. If it is determined in step S47 that the purchaser PU has not left the store ST, the purchase information creation unit 121 returns the process to step S35. This is because in this case, there is a high possibility that the purchaser PU will move around the store and look at a product ME on another product shelf SH.

[0054] In step S47, if it is determined that the purchaser PU has left the store ST, the purchase information creation unit 121 ends this processing routine. This processing routine creates purchase information 35, which is a history of the places that the purchaser PU has visited from the time they entered the store ST until they left.

[0055] FIG. 12 is a flowchart showing the procedure for the information addition process. The purchase information 35 includes information about the location where the purchaser PU is presumed to have viewed the product. The information addition unit 122 performs the information addition process to add information about the product ME that was displayed in the location where the purchaser PU is presumed to have viewed the product. FIG. 13 is a diagram showing product information 36 used by the information addition unit 122. As shown in FIG. 13, the product information 36 is data in which the fields "Location" and "Product ID" are paired. The "Location" field of the product information 36 is the same as the "Location" field of the purchase information 35. An identifier is entered in the "Product ID" field as identification information for identifying the product. In the case of a target store OST, the product information 36 is created by the image analysis device 60. In the case of a store ST, i.e., a store ST where the image analysis device 60 is not installed, the product information 36 is created manually.

[0056] When the purchasing information creation unit 121 finishes creating the purchasing information 35, the information addition unit 122 performs the information addition process. When the information addition unit 122 starts the information addition process, it sets a variable n used in the process to an initial value of zero.

[0057] In step S61 of FIG. 12 , the information addition unit 122 increments the variable n. In step S63, the information addition unit 122 sets the nth row of the purchasing information 35 as the processing target. In step S65, the information addition unit 122 uses the product information 36 to identify a product ID corresponding to the value of "location" in the nth row of the purchasing information 35, and inputs the identified identifier into "product ID" in the nth row. In step S67, the information addition unit 122 determines whether processing has been completed for all rows. If it is determined in step S67 that processing has not been completed, the information addition unit 122 returns the processing to step S61. If it is determined in step S67 that processing has been completed, the information addition unit 122 ends this processing routine.

[0058] According to the second embodiment described above, the purchase information creation unit 121 identifies the position of the object to which the purchaser PU is facing, and creates the purchase information 35. The information addition unit 122 adds product ID information to the purchase information 35. The purchase information 35 makes it possible to analyze the behavior of people when selecting a product ME.

[0059] In the second embodiment, the information addition unit 122 adds identification information of the product ME to the purchase information 35. The identification information added to the purchase information 35 may be identification information of an advertisement instead of the product ME. An example of an advertisement is a point-of-purchase (POP) advertisement. When the object that the purchaser PU turns his / her face to is an advertisement, the created object information can be used to analyze the influence of the advertisement on a person's purchase.

[0060] C. Third Embodiment: This embodiment differs from the second embodiment in that the image analysis unit 62, which serves as a psychological information acquisition unit, acquires the psychological state of the purchaser PU using an image including the purchaser PU captured by the imaging device 70. Differences from the second embodiment will be described, and other explanations will be omitted. The same configurations and processing steps as those in the above embodiments will be assigned the same reference numerals, and detailed explanations will be omitted as appropriate.

[0061] The psychological state, in detail, refers to the heart rate and facial color of the purchaser PU. The image analysis unit 62 analyzes the image and creates psychological information by associating the acquired heart rate and facial color with the date and time. The information addition unit 122 adds the acquired heart rate and facial color information to the purchase information 35 in association with the date and time of the purchase information 35. Specifically, "facial color" refers to the purchaser PU's facial expressions, such as a smile or a stern look, and whether or not the person has a healthy complexion. Furthermore, the information addition unit 122 uses the psychological information to calculate an evaluation value indicating the person's level of interest in the object and adds the calculated evaluation value to the purchase information 35. Specifically, in the case of the heart rate, the information addition unit 122 lowers the evaluation value if the heart rate is lower than normal. The information addition unit 122 raises the evaluation value if the purchaser PU's heart rate is higher than normal. Specifically, in the case of facial color, the information addition unit 122 raises the evaluation value if the purchaser PU has a healthy complexion or appears to be smiling. The information addition unit 122 lowers the evaluation value if the person has a poor complexion or a stern look. More specifically, the evaluation value is ranked as "A," "B," or "C" in descending order of interest.

[0062] According to the third embodiment, the image analysis unit 62 and the information addition unit 122 add psychological information of the purchaser PU and an evaluation value indicating the degree of interest to the purchase information 35. By adding the psychological information and the evaluation value to the purchase information 35, it becomes easier to use the purchase information 35 to analyze the influence that the product ME has on people.

[0063] D. Fourth Embodiment: This embodiment differs from the second embodiment in that the processor 20 of the information analysis system 1 further includes a sound information acquisition unit as a functional unit. Differences from the second embodiment will be described, and other descriptions will be omitted. The same configurations and processing steps as those in the above embodiments will be assigned the same reference numerals, and detailed descriptions will be omitted as appropriate.

[0064] The sound information acquisition unit acquires sound information collected by the microphone 83 built into the ear-worn device 80. The sound information collected by the microphone 83 is, in other words, sound information that is presumed to be heard by the purchaser PU. Examples of sound information include announcements played in the store ST. The information addition unit 122 adds the acquired sound information to the purchase information 35 in association with the date and time of the purchase information 35.

[0065] According to the fourth embodiment, the sound information acquisition unit and the information addition unit 122 add sound information that is presumed to be heard by the purchase information 35. Therefore, using the purchase information 35, it is possible to analyze the influence that sounds such as announcements made to the purchaser PU have on the behavior of the purchaser PU.

[0066] E. Other Embodiments: (E1) In the first embodiment, the motion information correcting unit 27 instructs the mobile terminal device 90 to correct the zero reference position of the first motion information 31. Similarly, the position information correcting unit 28 instructs the mobile terminal device 90 to correct the first position information 32. As a result, the first motion information 31 indicating the current motion and the first position information 32 indicating the current position, which are created by the mobile terminal device 90, are corrected. In another embodiment, as in the second embodiment, when it is desired to use the purchase information 35, the correction does not necessarily need to be performed in real time. Therefore, after the purchase information 35 is created, an information correction process may be performed to correct the location to which the purchaser PU in the purchase information 35 was facing.

[0067] (E2) In the first embodiment, the device equipped with the motion sensor 81 is an ear-worn device 80. The device equipped with the motion sensor 81 is not limited to a device worn on the ear of the purchaser PU, but may also be a device worn on the head of the purchaser PU. Specifically, the device equipped with the motion sensor 81 may be a device worn on the head of the purchaser PU or a device worn so as to cover the eyes of the purchaser PU.

[0068] (E3) In the second embodiment, the purchase information 35 is created in which the locations to which the purchaser PU turned their face are listed in chronological order. In another embodiment, the purchase information creation unit 121 may create object information that includes at least one location to which the purchaser PU turned their face. By including at least one location to which the purchaser PU turned their face in the object information, the object information can be used to consider the placement location of the product ME, for example.

[0069] (E4) In the first embodiment, the second sensor used to generate the second operation information 33 and the second position information 34 is the image sensor 71. The second sensor is not limited to the image sensor 71. The second sensor may be, for example, a sensor that detects infrared or far-infrared rays instead of visible light to detect the body temperature of the purchaser PU. Furthermore, in the first embodiment, the imaging device 70 is fixed to the store ST, but it may also be an unfixed imaging device. An example of an unfixed imaging device is an imaging device owned by someone other than the purchaser PU.

[0070] (E5) In the third embodiment, the psychological information of the purchaser PU is acquired by the image analysis unit 62. In another embodiment, the psychological information acquisition unit may be a heart rate monitor worn by the purchaser PU. Specifically, the psychological information acquisition unit may be realized by a heart rate monitor built into a wristwatch worn on the wrist of the purchaser PU.

[0071] (E6) In the second embodiment, the purchase information creation unit 121 creates the purchase information 35 as object history information. As another embodiment, the purchase information creation unit 121 may create the purchase information 35 as object history information in which products ME as objects are arranged in descending order of the length of time that a person has looked at them. When creating object history information in which the products ME are arranged in descending order of the length of time that a person has looked at them, the purchase information creation unit 121 creates the object history information by rearranging the "product IDs" in the created purchase information 35 in descending order of the "looking time." This object history information can provide information about products ME that are attracting a lot of attention from people.

[0072] The present disclosure is not limited to the above-described embodiments and can be realized in various configurations without departing from the spirit thereof. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention section can be appropriately replaced or combined to solve some or all of the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0073] 1...information analysis system, 20...processor, 21...information creation unit, 22...start determination unit, 23...first operation information acquisition unit, 24...first position information acquisition unit, 25...second operation information acquisition unit, 26...second position information acquisition unit, 27...operation information correction unit, 28...position information correction unit, 30...storage unit, 31...first operation information, 32...first position information, 33...second operation information, 34...second position information, 35...purchase information, 36...product information, 40...communication interface, 50...input / output interface, 51...display unit, 52...input device, 60...image analysis device, 61...processor, 62...image analysis unit, 6 3...storage unit, 66...communication interface, 70...imaging device, 71...image sensor, 80...ear-worn device, 80a...housing, 81...motion sensor, 82...sound conversion unit, 83...microphone, 84...communication interface, 90...mobile terminal device, 91...processor, 92...storage unit, 93...display operation unit, 94...position information acquisition unit, 95...sensor unit, 96...communication interface, 97...information provision application program, 121...purchase information creation unit, 122...information addition unit, ME...product, OST...target store, PU...purchaser, SH...product shelf, ST...store

Claims

1. A first motion information acquisition unit that acquires first motion information representing the motion of the head created using a first sensor worn on a person's head; a first position information acquisition unit that acquires first position information, which is the position information of the person, created using a position information acquisition unit worn on the person; a second motion information acquisition unit that acquires second motion information representing the motion of the head created using a second sensor different from the first sensor; a second position information acquisition unit that acquires second position information, which is the position information, created using a second sensor different from the first sensor; a motion information correction unit that corrects the first motion information using the second motion information; and a position information correction unit that corrects the first position information using the second position information. An information analysis system comprising the above components.

2. The information analysis system according to claim 1, wherein the second sensor used for creating the second motion information is the same image sensor as the second sensor used for creating the second position information, and each of the second motion information and the second position information is information created using an image of the person including the second sensor, which is an imaging device with a known installation position. An information analysis system.

3. The information analysis system according to claim 1, further comprising a creation unit that uses the first motion information and the first position information to identify an object the person is facing and creates object information including the object. An information analysis system.

4. The information analysis system according to claim 3, wherein the creation unit creates at least one of (a) object history information in which the objects are arranged in chronological order in the order in which the person faces them, and (b) object history information in which the objects are arranged in the order of the longest viewing time while the person is facing them. An information analysis system.

5. The information analysis system according to claim 3 or 4, wherein the object is a product, and the information analysis system further comprises an information addition unit that adds identification information of the product to the object information. An information analysis system.

6. The information analysis system according to claim 3 or 4, wherein the object is an advertisement, and the information analysis system further comprises an information addition unit that adds identification information of the advertisement to the object information. An information analysis system.

7. The information analysis system according to claim 1, wherein the first sensor is a motion sensor built in an ear-mounted device.

8. The information analysis system according to claim 3 or 4, further comprising a psychological information acquisition unit that acquires psychological information representing the psychological state of the person, and an information addition unit that adds the psychological information to the object information.

9. The information analysis system according to claim 8, wherein the psychological information includes at least one of the person's heart rate and information on the person's facial color.

10. The information analysis system according to claim 8, wherein the information addition unit obtains an evaluation value indicating the degree of interest of the person in the object using the psychological information, and adds the evaluation value to the object information.

11. The information analysis system according to claim 4, further comprising a sound information acquisition unit that acquires sound information estimated to be being listened to by the person, and an information addition unit that adds the sound information to the object history information.

Images