Information analysis system

Abstract

This invention relates to an information analysis system. An information analysis system includes: a first motion information acquisition unit that acquires first motion information representing head movements generated using a first sensor worn on a person's head; a first position information acquisition unit that acquires position information of a person, i.e., first position information, generated using a position information acquisition unit worn on a person; a second motion information acquisition unit that acquires second motion information representing head movements generated using a second sensor different from the first sensor; a second position information acquisition unit that acquires position information, i.e., second position information, generated using a second sensor different from the first sensor; a motion information correction unit that uses the second motion information to correct the first motion information; and a position information correction unit that uses the second position information to correct the first position information.

Description

Technical Field

[0001] This invention relates to an information analysis system. Background Technology

[0002] In the past, attempts have been made to detect the direction of a person's gaze, for example, in order to provide services. For example, in Patent Document 1, the direction of a person's gaze was detected in order to provide voice guidance on the object being looked at.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2015-079310

[0004] However, when using head-mounted sensors to detect a person's facial orientation, errors can easily occur depending on how the sensors are worn. Summary of the Invention

[0005] The present invention can be implemented in the following manner.

[0006] (1) According to one aspect of the present invention, an information analysis system is provided. The information analysis system comprises: a first motion information acquisition unit that acquires first motion information representing a head movement generated using a first sensor worn on a person's head; a first position information acquisition unit that acquires the person's position information, i.e., first position information, generated using a position information acquisition unit worn on the person; a second motion information acquisition unit that acquires second motion information representing the head movement generated using a second sensor different from the first sensor; a second position information acquisition unit that acquires the position information, i.e., second position information, generated using a second sensor different from the first sensor; a motion information correction unit that corrects the first motion information using the second motion information; and a position information correction unit that corrects the first position information using the second position information. According to this method, the accuracy of the first motion information and the first position information can be improved.

[0007] (2) In the information analysis system described above, the second sensor used to generate the second motion information and the second sensor used to generate the second position information can also be the same image sensor. The second motion information and the second position information can each be generated using an image of the person captured by a camera with a known location and equipped with the second sensor. According to this method, by using an image captured by a camera with a known location, the second motion information and second position information with smaller errors can be used for correction. Therefore, the accuracy of the first motion information and the first position information can be improved.

[0008] (3) In the above-described information analysis system, a manufacturing unit may also be included, which uses the first action information and the first position information to determine the object toward which the person's face is turned, and manufactures object information including the object. According to this method, the manufacturing unit is able to manufacture object information.

[0009] (4) In the above-described information analysis system, the manufacturing unit may also produce at least one of the following object history information: (a) object history information arranged in chronological order according to the direction the person faces, and (b) object history information arranged in descending order of viewing time the person faces. According to this method, the manufacturing unit is able to produce object history information.

[0010] (5) In the information analysis system described above, the object can also be a commodity, and the information analysis system may also include an information attachment unit that attaches commodity identification information to the object information. According to this method, the information attachment unit can attach commodity identification information to the object information. The object information with attached commodity identification information can be used to analyze a person's behavior when selecting a commodity.

[0011] (6) In the information analysis system described above, the object can also be an advertisement, and the information analysis system may also include an information appending unit that appends the advertisement's identification information to the object information. According to this method, the information appending unit can append the advertisement's identification information to the object information. The object information with the appended advertisement identification information can be used to analyze the impact of the advertisement on people.

[0012] (7) In the above-described information analysis system, the first sensor may also be a motion sensor built into the ear-worn device. According to this method, a motion sensor built into the ear-worn device can be used as the first sensor.

[0013] (8) The information analysis system described above may further include: a psychological information acquisition unit that acquires psychological information representing the psychological state of the person; and an information appending unit that appends the psychological information to the object information. According to this method, the information appending unit can append psychological information to the object information.

[0014] (9) In the above-described information analysis system, the psychological information may also include at least one of the person's heart rate and facial expression information. According to this method, either the person's heart rate or facial expression information can be used as psychological information.

[0015] (10) In the information analysis system described above, the information appending unit can also use the psychological information to calculate an evaluation value representing the degree of concern of the person for the object, and append the evaluation value to the object information. According to this method, the impact of the product on the person can be more easily analyzed through the evaluation value.

[0016] (11) The information analysis system described above may further include: a sound information acquisition unit that acquires sound information that is inferred to be being listened to by the person; and an information appending unit that appends the sound information to the object's history information. According to this method, the information appending unit can append sound information to the object's history information.

[0017] This invention can be implemented in various ways, in addition to the information analysis system described above, such as information analysis methods and information analysis programs. Attached Figure Description

[0018] Figure 1 It is a block diagram representing the structure of an information analysis system.

[0019] Figure 2 This diagram illustrates the purpose of purchasing ear-worn devices and portable terminal devices.

[0020] Figure 3 This is a block diagram illustrating the structure of an ear-worn device and a portable terminal device.

[0021] Figure 4 This diagram illustrates the detection performed by a motion sensor.

[0022] Figure 5 This is a block diagram representing the structure of an image resolution device.

[0023] Figure 6 This diagram illustrates the information correction process.

[0024] Figure 7 This is a flowchart illustrating the steps involved in the information correction process.

[0025] Figure 8 This diagram illustrates the correction of the first action information.

[0026] Figure 9 This is a block diagram illustrating the structure of the information analysis system involved in the second embodiment.

[0027] Figure 10 This is a flowchart illustrating the steps involved in processing purchase information.

[0028] Figure 11 This is a diagram representing purchase information.

[0029] Figure 12 This is a flowchart illustrating the steps involved in information appending processing.

[0030] Figure 13 It is a diagram that represents product information. Detailed Implementation

[0031] A. First implementation method:

[0032] A1. Structure of the Information Correction System:

[0033] Figure 1 This is a block diagram representing the structure of information analysis system 1. Figure 2 This diagram illustrates a purchaser PU of a wearable earpiece 80 and a portable terminal device 90, which are processed by the information analysis system 1. As described later, the earpiece 80 has a built-in motion sensor 81 as a first sensor. The information analysis system 1 performs calibration to reduce the error of the first motion information 31, etc., generated using the detection values ​​output by the motion sensor 81. This improves the accuracy of analyzing the actions of, for example, the purchaser PU, using the first motion information 31.

[0034] like Figure 1 As shown, the information analysis system 1 is configured as a computer in which a processor 20, a storage unit 30, a communication interface 40, and an input / output interface 50 are interconnected via a bus. The input / output interface 50 is connected to a display unit 51 such as a liquid crystal display and input devices 52 such as a keyboard and mouse. The processor 20 performs various processes by executing programs stored in the storage unit 30. The processor 20, as a functional unit, includes: an information generation unit 21, a start determination unit 22, a first motion information acquisition unit 23, a first position information acquisition unit 24, a second motion information acquisition unit 25, a second position information acquisition unit 26, a motion information correction unit 27, and a position information correction unit 28. Each functional unit is implemented by executing programs stored in the storage unit 30. The motion information correction unit 27 uses second motion information 33 to correct first motion information 31. The position information correction unit 28 uses second position information 34 to correct first position information 32.

[0035] The storage unit 30 is implemented using a memory such as RAM or ROM. The storage unit 30 stores first motion information 31, first position information 32, second motion information 33, and second position information 34. The communication interface 40 communicates with external devices such as the portable terminal device 90 and the image analysis device 60 (described later) via a network such as the Internet. The first motion information 31 is information representing head movements generated using a motion sensor 81, which is worn on the head of a person, specifically the purchaser PU, as a first sensor. The first position information 32 is the purchaser PU's position information generated using a position information acquisition unit 94 worn on the head of the purchaser PU. Furthermore, in this invention, "worn on a person" includes a state fixed to a person's body and a state carried by a person. Specifically, "carried by a person" includes: a state held in a person's hand, a state placed in a bag carried on a person's back, etc. As described later, in this embodiment, the motion sensor 81 is built into the ear-worn device 80. The position information acquisition unit 94 is built into the portable terminal device 90 carried by the purchaser PU. The second motion information 33 is information representing the actions of the purchaser PU, generated using an image sensor 71 (different from the motion sensor 81) as the second sensor. The second position information acquisition unit 26 acquires position information of the purchaser PU generated using an image sensor 71 (different from the motion sensor 81) as the second sensor.

[0036] The information analysis system 1 communicates with both the portable terminal device 90 and the image analysis device 60. For example... Figure 2 As shown, the portable terminal device 90 is carried by the purchaser PU. In addition, the purchaser PU, who carries the portable terminal device 90, wears an ear-worn device 80 on his ear.

[0037] Figure 3 This is a block diagram showing the structure of the ear-worn device 80 and the portable terminal device 90. The ear-worn device 80... Figure 2 Each of the pair of housings 80a shown has a built-in Figure 3 The motion sensor 81, sound converter 82, microphone 83, and communication interface 84 are shown. Furthermore, the motion sensor 81 and microphone 83 can be housed separately in either of a pair of housings 80a, or in either housing 80a.

[0038] Motion sensor 81 detects the head movements of the purchaser (PU). Specifically, motion sensor 81 is a general-purpose so-called six-axis motion sensor. Specifically, motion sensor 81 includes a three-axis accelerometer and a three-axis angular velocity sensor. Motion sensor 81 detects the acceleration [m / s²] and angular velocity [deg / s] of each of the three axes.

[0039] Figure 4This diagram illustrates the detection performed by motion sensor 81. (As shown...) Figure 4 As shown, the three axes used by motion sensor 81 to detect acceleration and angular velocity are specifically the mutually orthogonal yaw axis, roll axis, and pitch axis. The yaw axis is parallel to the vertical direction relative to the buyer PU. The roll axis is parallel to the forward / backward direction relative to the buyer PU. The pitch axis is parallel to the left / right direction relative to the buyer PU. For yaw acceleration, motion sensor 81 outputs a positive value when the buyer PU moves their head upward and a negative value when the buyer PU moves their head downward. For roll acceleration, motion sensor 81 outputs a positive value when the buyer PU moves their head forward and a negative value when the buyer PU moves their head backward. For pitch acceleration, motion sensor 81 outputs a positive value when the buyer PU moves their face to the left and a negative value when the buyer PU moves their head to the right. Motion sensor 81 outputs a positive value for the angular velocity along the yaw axis when the user (PU) turns their face to the right and a negative value when they turn their face to the left. Similarly, it outputs a positive value for the angular velocity along the roll axis when the user tilts their head to the left and a negative value when they tilt their head to the right. Finally, it outputs a positive value for the angular velocity along the pitch axis when the user (PU) turns their face downwards and a negative value when they turn their face upwards. Therefore, head movements can be detected based on the changes in acceleration and angular velocity along each of the three axes.

[0040] Figure 3 The sound conversion unit 82 shown converts sound data, which is an electrical signal, into sound and emits it toward the ear of the purchaser PU. The microphone 83 converts sound into sound data, which is an electrical signal. The microphone 83 outputs the acquired voice data to the communication interface 84. The communication interface 84 communicates with the portable terminal device 90 over a short distance. In this embodiment, the communication interface 84 communicates with the portable terminal device 90 via Bluetooth (a registered trademark). The communication interface 84 is implemented using electronic circuitry or the like.

[0041] In this embodiment, the portable terminal device 90 is a smartphone. The portable terminal device 90 includes a processor 91, a storage unit 92, a display operation unit 93, a location information acquisition unit 94, a sensor unit 95, and a communication interface 96.

[0042] The processor 91 performs various processes by executing programs stored in the storage unit 92. The storage unit 92 is implemented using memory such as RAM or ROM. The storage unit 92 stores an information providing application program 97. In addition, the storage unit 92 stores first action information 31 and first position information 32. The display operation unit 93 has a touch panel. The display operation unit 93 displays information on the touch panel and accepts operations input to the touch panel.

[0043] The location information acquisition unit 94 has a GNSS (Global Navigation Satellite System) receiver (not shown) and receives signals from positioning satellites under the control of the processor 91. In this embodiment, the location information acquisition unit 94 uses GPS (Global Positioning System) as GNSS. The location information acquisition unit 94 may also use other satellite positioning systems such as the Quasi-Zenith Satellite System to acquire location information. The sensor unit 95 includes a geomagnetic sensor and an accelerometer. The location information acquisition unit 94 can use detection values ​​from the sensor unit 95, radio waves from a mobile communication system base station, radio waves from a wireless LAN (Local Area Network) access point, etc., to correct the location information.

[0044] Communication interface 96 communicates with communication interface 84 of ear-worn device 80 at close range. Furthermore, communication interface 96 communicates with communication interface 40 of information analysis system 1 using a mobile communication system such as 5G (5th Generation Mobile Communication System). Additionally, the communication method between communication interface 96 and communication interface 40 of information analysis system 1 is not limited to mobile communication systems; it can also use wireless LAN, or a combination of these methods. Furthermore, communication between ear-worn device 80 and portable terminal device 90 is not limited to wireless communication; it can also be wired communication.

[0045] The processor 91 of the portable terminal device 90 executes the information providing application 97, sequentially sending the first motion information 31 acquired by the motion sensor 81 and the first position information 32 acquired by the position information acquisition unit 94 to the information analysis system 1. That is, the first motion information 31 represents information about the head movements of the purchaser (PU), i.e., the person. In this embodiment, the first motion information 31 includes the detection value of the motion sensor 81 and the current angle relative to the zero reference position, calculated by the information providing application 97 using the detection value of the motion sensor 81. The first position information is the position information of the person, specifically the purchaser (PU). The information analysis system 1 stores the received first motion information 31 and first position information 32 in the storage unit 30.

[0046] Figure 5 This is a block diagram showing the structure of the image resolution device 60. For example... Figure 2As shown, in this embodiment, the image analysis device 60 is installed in a store ST equipped with a shooting device 70. The store ST is equipped with a merchandise shelf SH. Merchandise ME is displayed on the merchandise shelf SH. Specifically, the shooting device 70 is a camera with an image sensor 71. The image sensor 71 is, for example, a CCD (Charge-Coupled Device) image sensor, a sensor that converts light into an electrical signal. The shooting device 70 is configured to capture images of the merchandise ME and the buyer PU. Furthermore, Figure 2 Only one camera device 70 is shown in the image, but multiple camera devices 70 can be installed in the store ST.

[0047] like Figure 5 As shown, the image parsing apparatus 60 is configured as a computer in which a processor 61, a storage unit 63, and a communication interface 66 are interconnected via a bus. The processor 61 performs various processes by executing programs stored in the storage unit 63. The processor 61 includes an image parsing unit 62 as a functional unit. The image parsing unit 62 is implemented by executing programs stored in the storage unit 63.

[0048] The storage unit 63 is implemented using memory such as RAM or ROM. The storage unit 63 stores second action information 33 and second position information 34. The communication interface 66 is connected to the imaging device 70 via a video cable. Furthermore, the communication interface 66 communicates with the communication interface 40 of the information analysis system 1 using a mobile communication system such as 5G. Additionally, the communication method between the communication interface 66 and the communication interface 40 of the information analysis system 1 is not limited to a mobile communication system; it can also use a wireless LAN method, or a combination of these methods.

[0049] The image analysis unit 62 uses an image containing the purchaser (PU) captured by the imaging device 70, whose location is known, to generate second motion information 33 and second position information 34. Specifically, it analyzes the image of the purchaser (PU) input from the imaging device 70 to generate second motion information 33 representing the head movement of the purchaser (PU). In this embodiment, the second motion information 33 is information equivalent to the detection value of the motion sensor 81, or information that can be converted into angle information obtained from the angular velocity output by the motion sensor 81. The image analysis unit 62 analyzes the image of the purchaser (PU) input from the imaging device 70 to generate second position information 34 representing the position information of the purchaser (PU). The motion information of the second motion information 33 is associated with date and time information. The position information of the second position information 34 is associated with date and time information. As described above, when multiple imaging devices 70 are configured, the image analysis unit 62 can also analyze multiple images captured by multiple imaging devices 70.

[0050] The image analysis unit 62 sequentially sends the generated second motion information 33 and second position information 34 to the information analysis system 1. The information analysis system 1 stores the received second motion information 33 and second position information 34 in the storage unit 30.

[0051] A2. Information Correction Processing:

[0052] Figure 6 This diagram illustrates the information correction process. Figure 6 This is a map showing the configuration of multiple store STs. The store ST equipped with the aforementioned camera 70 and image analysis device 60 among the multiple store STs is called the target store OST.

[0053] The zero reference position of the motion sensor 81 built into the ear-worn device 80 worn by the purchaser PU is adjusted using the portable terminal device 90 when the purchaser PU wears the ear-worn device 80. As described above, the information providing application 97 uses the detected value of the angular velocity from the motion sensor 81 to calculate the current angle relative to the zero reference position and generates the first motion information 31. Furthermore, the information providing application 97 adjusts and sets the angles of each of the three axes of the motion sensor 81 to be "zero" when the purchaser PU is in a posture with their head not tilted. However, depending on the wearing state of the ear-worn device 80, sometimes even though the purchaser PU is in a posture with their head not tilted, the angles of each of the three axes are not "zero". In this case, an error will occur between the actual posture of the purchaser PU's head and the angles calculated from the detection values ​​of the motion sensor 81. In addition, an error may also occur between the first position information 32 and the actual current position. Therefore, the information analysis system 1 corrects the first motion information 31 and the first position information 32 by performing information correction processing. This improves the accuracy of the first motion information 31 relative to the actual posture, achieved using the motion sensor 81. Furthermore, it improves the accuracy of the first position information 32 relative to the actual current position.

[0054] Furthermore, by performing corrections, the accuracy of the purchase information 35 described in the second embodiment can be improved when the buyer (PU) enters a store (ST) that is not the target store (OST). Moreover, by performing corrections, the quality of services such as providing guidance that infers the object being viewed by the buyer (PU) can be improved.

[0055] Figure 7This is a flowchart illustrating the steps of the information correction process. The start determination unit 22 determines whether the buyer (PU) has entered the target store OST. Specifically, when the first location information 32 determines that the buyer's current location is within a predetermined reference location range, the start determination unit 22 determines that the buyer has entered the target store OST. Here, the reference location range refers to the location range of the target store OST's in-store area. When the start determination unit 22 determines that the buyer has entered the target store OST, the information correction process begins.

[0056] In step S1, the motion information correction unit 27 determines whether the first motion information 31 and the second motion information 33 are consistent with each other about the yaw axis. In step S1, the motion information correction unit 27 calculates the first angle difference D1 using the following formula (1).

[0057] D1=θ1a-θ1b…(1)

[0058] In Equation (1), "θ1a" is the angle of the yaw axis of the first motion information 31. That is, "θ1a" in Equation (1) is the angle of the yaw axis calculated based on the detection value of the motion sensor 81. In Equation (1), "θ1b" is the angle of the yaw axis of the second motion information 33. That is, "θ1b" in Equation (1) is the angle of the yaw axis calculated based on the image captured by the imaging device 70. Furthermore, when the first angle difference D1 is within a predetermined first reference range, the motion information correction unit 27 determines that the first motion information 31 and the second motion information 33 are consistent. On the other hand, when the first angle difference D1 is outside the first reference range, the motion information correction unit 27 determines that the first motion information 31 and the second motion information 33 are inconsistent. The first reference range is a range determined through experiments, etc.

[0059] In step S1, when the angle of the yaw axis is determined to be inconsistent between the first motion information 31 and the second motion information 33, in step S3, the motion information correction unit 27 corrects the zero reference position of the yaw axis of the first motion information 31. Specifically, the motion information correction unit 27 sends a command signal to the portable terminal device 90 to set the position obtained by subtracting the first angle difference D1 from the current reference position as the new zero reference position. When the command signal is received, the portable terminal device 90 uses the first angle difference D1 to correct the zero reference position of the first motion information 31, which is made using the detection value from the motion sensor 81.

[0060] Figure 8 This diagram illustrates step S3. In step S3, the zero reference position is corrected to the position shown by the dashed line, which is rotated by a first angular difference D1, i.e., the angular difference between the yaw axis angle θ1a of the first motion information 31 and the yaw axis angle θ1b of the second motion information 33. In other words, Figure 8 The arrow shown by the dashed line represents the new zero reference position. By correcting the zero reference position in step S3, the accuracy of the yaw axis angle in the subsequent first action information 31 can be improved.

[0061] exist Figure 7 In step S1, when the angle of the yaw axis is determined to be consistent with the first motion information 31 and the second motion information 33, since no correction is required, the motion information correction unit 27 will proceed to step S5.

[0062] In step S5, the motion information correction unit 27 determines whether the first motion information 31 and the second motion information 33 are consistent with each other regarding the pitch axis. Step S5 is performed in the same manner as step S1, with the pitch axis as the processing object. In step S5, if the angle with respect to the pitch axis determines that the first motion information 31 and the second motion information 33 are inconsistent, in step S7, the motion information correction unit 27 corrects the zero reference position of the pitch axis of the first motion information 31. Step S7 is performed in the same manner as step S3, with the pitch axis as the processing object. In step S5, if the angle with respect to the pitch axis determines that the first motion information 31 and the second motion information 33 are consistent, the motion information correction unit 27 proceeds to step S9.

[0063] In step S9, the motion information correction unit 27 determines whether the first motion information 31 and the second motion information 33 are consistent with each other regarding the roll axis angle. Step S9 is performed in the same manner as step S1, with the roll axis as the processing object. In step S9, if the first motion information 31 and the second motion information 33 are inconsistent with each other regarding the roll axis angle, in step S11, the motion information correction unit 27 corrects the zero reference position of the roll axis for the first motion information 31. Step S11 is performed in the same manner as step S3, with the roll axis as the processing object. In step S9, if the first motion information 31 and the second motion information 33 are consistent with each other regarding the roll axis angle, the motion information correction unit 27 proceeds to step S13.

[0064] In step S13, the position information correction unit 28 determines whether the first position information 32 is consistent with the second position information 34. When the position information is represented, for example, using an XYZ coordinate system, the position information correction unit 28 determines whether the first position information 32 is consistent with the second position information 34 regarding the X, Y, and Z coordinates respectively. When at least one of the X, Y, and Z coordinates is inconsistent, the position information correction unit 28 determines that the first position information 32 is inconsistent with the second position information 34. In step S13, when it is determined that the first position information 32 is inconsistent with the second position information 34, in step S15, the position information correction unit 28 corrects the first position information 32 so that the first position information 32 is consistent with the second position information 34. In step S15, specifically, the position information correction unit 28, similarly to the first action information 31, sends a command signal to the portable terminal device 90 so that the value obtained by subtracting the difference between the first position information 32 and the second position information 34 from the first position information 32 is used as the corrected coordinate. When a command signal is received, the portable terminal device 90 corrects the first position information 32.

[0065] In step S13, when it is determined that the first position information 32 and the second position information 34 are consistent, and after performing step S15, the motion information correction unit 27 ends the processing routine.

[0066] According to the first embodiment described above, the information analysis system 1 includes a first motion information acquisition unit 23, a first position information acquisition unit 24, a second motion information acquisition unit 25, a second position information acquisition unit 26, a motion information correction unit 27, and a position information correction unit 28. The motion information correction unit uses second motion information 33 to correct the first motion information 31. The position information correction unit 28 uses second position information 34 to correct the first position information 32. Therefore, the accuracy of the first motion information 31 and the first position information 32 can be improved.

[0067] Furthermore, the second motion information 33 and the second position information 34 are each generated using an image containing a person captured by the imaging device 70 whose setting position is known. By using the image captured by the imaging device 70 whose setting position is known, the second motion information 33 and the second position information 34, which have smaller errors, can be used for correction.

[0068] B. Second implementation method:

[0069] Figure 9 This is a block diagram illustrating the structure of the information analysis system 1 according to this embodiment. For example... Figure 9As shown, the processor 20 differs from the first embodiment in that it includes a purchase information manufacturing unit 121 and an information addition unit 122, which are manufacturing units. The storage unit 30 stores purchase information 35 and product information 36, which are object information and object history information. The same reference numerals are used to denote structures and processing steps identical to those in the first embodiment, and detailed descriptions are omitted where appropriate.

[0070] The purchase information generation unit 121 uses the first motion information 31 and the first position information 32 to determine the object that the person is facing, and generates purchase information 35 as an object information containing the object. In this embodiment, the object is specifically displayed in Figure 2 The product ME on the shelf SH shown. Furthermore, in this embodiment, the purchase information 35 is also a chronological record of the items arranged in the order in which people face them. Through the purchase information 35, it is possible to know... Figure 2 The image shows where the buyer (PU) looked at on the shelf (SH) when selecting product (ME) in store (ST). Purchase information 35 can be used to study information about the inventory of product (ME) and the placement of product (ME).

[0071] In this embodiment, during the purchase information generation process described later, the object that the purchaser PU is facing is determined as the location information of the shelf SH. Furthermore, the information attachment unit 122 uses the shelf SH location information and the product information 36 to attach a product ID as identification information for the product ME to the purchase information 35.

[0072] Figure 11 This is a diagram representing Purchase Information 35. Purchase Information 35 is tabular data where "Date and Time," "Location," "Viewing Time," and "Product ID" are grouped together. The "Location" item also includes "Shelf ID," "Layer," and "X Location" items. Enter the pre-assigned... Figure 2 The identifier for the shelf SH in the store ST shown. Enter the shelf number (from the bottom) of the shelf SH in the "Layer" field. Enter the dimension relative to the left edge in the "X Position" field, representing its position within the same shelf. Other fields will be explained later.

[0073] Figure 10 This is a flowchart illustrating the steps involved in generating purchase information. When the initial determination unit 22 determines that a buyer (PU) has entered the store (ST), the process begins... Figure 10The purchase information shown is processed. In step S31, the start determination unit 22 determines whether the store ST entered by the purchaser PU is the target store OST. When the start determination unit 22 determines that the store ST entered by the purchaser PU is the target store OST, information correction processing is performed in step S33. The information correction processing is performed in the same way as in the first embodiment. When the start determination unit 22 determines that the store ST entered by the purchaser PU is not the target store OST, the process proceeds to step S35.

[0074] In step S35, the purchase information generation unit 121 determines whether the detected speed Sm, calculated based on the acceleration detection value of the motion sensor 81, is below the reference speed Smth. The reference speed Smth is the speed at which the purchaser PU stops to select product ME, and is determined in advance through experiments, etc. In step S35, if it is determined that the detected speed Sm is not below the reference speed Smth, the purchase information generation unit 121 proceeds to step S47.

[0075] In step S35, when it is determined that the detection speed Sm is below the reference speed Smth, in step S37, the purchase information generation unit 121 determines whether the detection angular velocity Sa detected by the motion sensor 81 is below the reference angular velocity Sath. The reference angular velocity Sath is the angular velocity used to determine whether the purchaser PU is looking at a product ME in order to select the product ME, and is determined in advance through experiments, etc. In step S37, when all three axes of angular velocity are below the reference angular velocity Sath, the purchase information generation unit 121 determines that the detection angular velocity Sa is below the reference angular velocity Sath.

[0076] In step S37, when it is determined that the detected angular velocity Sa is not below the reference angular velocity Sath, the purchase information production unit 121 will proceed to step S47.

[0077] In step S37, when it is determined that the detected angular velocity Sa is below the reference angular velocity Sath, in step S39, the purchase information generation unit 121 determines the position in which the purchaser PU is facing, i.e., the position inferred as being gazed at. Specifically, the purchase information generation unit 121 uses the first action information 31, the first position information 32, the equipment information of the store ST, and the body information of the purchaser PU to determine the position in which the purchaser PU is facing. The equipment information of the store ST refers to the shelf ID, position, size, number of layers, etc. of the product shelf SH in the store ST. In addition, the "shelf ID" is the identification information pre-assigned to the product shelf SH of the store ST to identify the product shelf SH. The body information of the purchaser PU refers to the height of the purchaser PU, etc. The equipment information of the store ST and the body information of the purchaser PU are pre-stored in the storage unit 30. The position in which the face is facing is determined by the values ​​of the "shelf ID", "layer", and "X position" items in the purchase information 35 mentioned above.

[0078] In step S41, the purchase information generation unit 121 determines whether the detected angular velocity Sa is greater than the reference angular velocity Sath. Step S41 is a processing step for determining whether the purchaser PU has shifted their gaze to a different product ME than the one they were previously looking at. In step S41, when the angular velocity of at least one of the three axes is greater than the reference angular velocity Sath, the purchase information generation unit 121 determines that the detected angular velocity Sa is greater than the reference angular velocity Sath.

[0079] In step S41, when it is determined that the detected angular velocity Sa is not greater than the reference angular velocity Sath, the purchase information generation unit 121 repeats step S41 at predetermined time intervals until it is determined that the detected angular velocity Sa is greater than the reference angular velocity Sath. The predetermined time is, for example, about a few milliseconds.

[0080] In step S41, when it is determined that the detected angular velocity Sa is greater than the reference angular velocity Sath, in step S43, the purchase information generation unit 121 adds information to the purchase information 35. Specifically, the purchase information generation unit 121 inputs values ​​into each item of the purchase information 35 and adds a line. In the "Date and Time" field of the purchase information 35, the date and time at which it was determined in step S37 that the detected angular velocity Sa is less than or equal to the reference angular velocity Sath are entered. In the "Visual Time" field of the purchase information 35, the time from the moment in step S37 when the purchase information generation unit 121 determined that the detected angular velocity Sa is less than or equal to the reference angular velocity Sath are entered, until the moment in step S41 when the purchase information generation unit 121 determined that the detected angular velocity Sa is greater than or equal to the reference angular velocity Sath are entered. In the "Location" field of the purchase information 35, the value determined in step S39 is entered. In addition, a blank value is entered in the "Product ID" field of the purchase information 35. In other words, the purchase information 35 is a resume information that arranges the locations of the objects in chronological order according to the direction the purchaser PU faces.

[0081] In step S45, the purchase information generation unit 121 determines whether the detected speed Sm, calculated based on the acceleration detected by the motion sensor 81, is greater than the reference speed Smth. Step S45 is a processing step for determining whether the purchaser PU has moved. In step S45, if it is determined that the detected speed Sm is not greater than the reference speed Smth, the purchase information generation unit 121 returns the processing to step S37. This is because, in this case, there is a higher probability that the purchaser PU has not moved and is looking at the product ME displayed on the same shelf SH.

[0082] In step S45, when it is determined that the detection speed Sm is greater than the reference speed Smth, in step S47, the purchase information generation unit 121 determines whether the buyer PU has left the store ST. In step S47, if it is determined that the buyer PU has not left the store ST, the purchase information generation unit 121 returns the process to step S35. This is because, in this case, there is a higher probability that the buyer PU is moving around in the store and looking at the product ME on other shelves SH.

[0083] In step S47, when it is determined that the buyer PU has left the store ST, the purchase information generation unit 121 terminates this processing routine. Through this processing routine, purchase information 35 is generated as the history of the location viewed by the buyer PU from entering the store ST to leaving.

[0084] Figure 12 This is a flowchart illustrating the steps of information appending processing. Purchase information 35 includes information about locations that are inferred to have been viewed by the purchaser (PU). The information appending unit 122 appends information about the merchandise (ME) displayed in the locations inferred to have been viewed by the purchaser (PU) through information appending processing. Figure 13 This is a diagram showing the product information 36 used by the information attachment section 122. (Example) Figure 13 As shown, product information 36 is data where the "location" and "product ID" items are grouped together. The "location" item in product information 36 is the same as the "location" item in purchase information 35. An identifier is entered in the "product ID" item as identification information for the product. Product information 36 is generated by image analysis device 60 in the case of the target store OST. In the case of store ST, i.e., store ST without image analysis device 60, it is generated by a person.

[0085] When the purchase information production unit 121 completes the production of purchase information 35, the information attachment unit 122 performs information attachment processing. When the information attachment processing begins, the information attachment unit 122 sets the variable n used in the processing to an initial value of zero.

[0086] exist Figure 12 In step S61, the information appending unit 122 increments the variable n. In step S63, the information appending unit 122 sets the nth row of the purchase information 35 as the processing target. In step S65, the information appending unit 122 uses the product information 36 to determine the product ID corresponding to the value of the "position" in the nth row of the purchase information 35, and inputs the determined identifier into the "product ID" field in the nth row. In step S67, the information appending unit 122 determines whether processing has been completed for all rows. In step S67, if it is determined that processing is not complete, the information appending unit 122 returns the processing to step S61. In step S67, if it is determined that processing is complete, the information appending unit 122 ends the current processing routine.

[0087] According to the second embodiment described above, the purchase information generation unit 121 determines the position of the object that the purchaser PU is facing and generates purchase information 35. The information attachment unit 122 attaches product ID information to the purchase information 35. Through the purchase information 35, it is possible to analyze the behavior of the person when selecting product ME.

[0088] In the second embodiment, the information attachment unit 122 attaches identification information of the product ME to the purchase information 35. The identification information attached to the purchase information 35 may also be advertising identification information, rather than the product ME. For example, a POP (Point of Purchase) advertisement can be cited. When the object that the buyer PU is facing is an advertisement, the information about the object can be used to analyze the impact of the advertisement on the buyer's purchase.

[0089] C. Third implementation method:

[0090] In this embodiment, the difference from the second embodiment is that the image analysis unit 62, which is the psychological information acquisition unit, uses an image containing the purchaser PU captured by the imaging device 70 to acquire the purchaser PU's psychological state. The parts that differ from the second embodiment will be described, and other descriptions will be omitted. The same reference numerals are used for structures and processing steps that are the same as in the above embodiments, and detailed descriptions are omitted where appropriate.

[0091] Specifically, the psychological state refers to the buyer PU's heart rate and facial expression. The image analysis unit 62 analyzes the image and generates psychological information that associates the acquired heart rate and facial expression with the date and time. The information attachment unit 122 associates the acquired heart rate and facial expression information with the purchase information 35 and attaches it to the purchase information 35. "Facial expression" specifically refers to the buyer PU's facial expressions, such as a smile or a serious expression, as well as the quality of their complexion. Furthermore, the information attachment unit 122 uses the psychological information to calculate an evaluation value representing the degree of concern a person has for an object, and attaches the calculated evaluation value to the purchase information 35. Specifically, regarding heart rate, when the heart rate is lower than normal, the information attachment unit 122 lowers the evaluation value. When the buyer PU's heart rate is higher than normal, the information attachment unit 122 raises the evaluation value. Specifically, regarding facial expression, when the buyer PU has a good complexion or a visible smile, the information attachment unit 122 raises the evaluation value. When the complexion is poor or the expression is serious, the information attachment unit 122 lowers the evaluation value. More specifically, the evaluation values ​​are assigned grades of "A", "B", and "C" in descending order of importance.

[0092] According to the third embodiment described above, the image analysis unit 62 and the information addition unit 122 add psychological information of the buyer PU and an evaluation value indicating the degree of concern to the purchase information 35. By adding psychological information and evaluation values ​​to the purchase information 35, it is possible to more easily analyze the impact of the product ME on people using the purchase information 35.

[0093] D. Fourth Implementation Method:

[0094] In this embodiment, the difference from the second embodiment is that the processor 20 of the information analysis system 1 also includes a sound information acquisition unit as a functional unit. The parts that differ from the second embodiment will be described, while other descriptions are omitted. The same reference numerals are used to denote structures and processing steps identical to those in the above embodiments, and detailed descriptions are omitted where appropriate.

[0095] The sound information acquisition unit acquires sound information collected by the microphone 83 built into the ear-worn device 80. The sound information collected by the microphone 83 is inferred to be the sound information that the purchaser PU is listening to. Examples of sound information include, for example, notifications played in the store ST. The information attachment unit 122 associates the information with the date and time of the purchase information 35 and attaches the acquired sound information to the purchase information 35.

[0096] According to the fourth embodiment described above, the sound information acquisition unit and the information attachment unit 122 attach sound information inferred to be being listened to by the purchase information 35. Therefore, by using the purchase information 35, it is possible to analyze the impact of sounds such as notifications issued to the purchaser PU on the purchaser PU's actions.

[0097] E. Other implementation methods:

[0098] (E1) In the first embodiment described above, the motion information correction unit 27 issues a command to the portable terminal device 90 to correct the zero reference position of the first motion information 31. Similarly, the position information correction unit 28 issues a command to the portable terminal device 90 to correct the first position information 32. Thus, the first motion information 31, which indicates the current action, and the first position information 32, which indicates the current position, generated by the portable terminal device 90, are corrected. As another embodiment, when it is desired to utilize the purchase information 35 as in the second embodiment, the correction does not necessarily need to be performed in real time. Therefore, it is also possible to correct the position in which the purchaser PU of the purchase information 35 had their face turned by performing information correction processing after the purchase information 35 is generated.

[0099] (E2) In the first embodiment described above, the device equipped with the motion sensor 81 is an ear-worn device 80. The device equipped with the motion sensor 81 is not limited to a device worn on the buyer's PU ear, but may also be a device worn on the buyer's PU head. Specifically, the device equipped with the motion sensor 81 may also be a device worn on the buyer's PU head, or a device worn in a manner that covers the buyer's PU eyes.

[0100] (E3) In the second embodiment described above, purchase information 35 is generated, which arranges the positions in which the purchaser (PU) faces in a time sequence. Alternatively, the purchase information generation unit 121 may generate object information that includes the positions in which at least one purchaser (PU) faces. By including the positions in which at least one purchaser (PU) faces in the object information, it can be used to study the placement of goods (ME), etc.

[0101] (E4) In the first embodiment described above, the second sensor used to generate the second motion information 33 and the second position information 34 is an image sensor 71. The second sensor is not limited to an image sensor 71. For example, the second sensor could also be a sensor that detects infrared or far-infrared light instead of visible light to detect the body temperature of the purchaser (PU). Furthermore, in the first embodiment, the shooting device 70 is fixed to the store ST, but it could also be a non-fixed shooting device. Examples of non-fixed shooting devices include those held by a person other than the purchaser (PU).

[0102] (E5) In the third embodiment described above, the psychological information of the purchaser PU is acquired by the image analysis unit 62. Alternatively, the psychological information acquisition unit may be a heart rate monitor worn by the purchaser PU. Specifically, the psychological information acquisition unit may also be a heart rate monitor built into a watch worn on the purchaser PU's arm.

[0103] (E6) In the second embodiment described above, the purchase information generation unit 121 generates purchase information 35 as object history information. Alternatively, the purchase information generation unit 121 may generate purchase information 35 as object history information, which arranges the product ME (object) in descending order of viewing time (when a person's face is turned towards it). When generating this object history information arranged in descending order of viewing time, the purchase information generation unit 121 rearranges the "product ID" of the generated purchase information 35 in descending order of "viewing time" to generate the object history information. Based on this object history information, information about products ME that attract high human attention can be provided.

[0104] This invention is not limited to the embodiments described above, and can be implemented in various structures without departing from its spirit. For example, the technical features of the embodiments corresponding to the technical features in the various methods described in the summary of the invention can be appropriately replaced or combined to solve some or all of the above-described problems or to achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0105] Explanation of reference numerals in the attached figures

[0106] 1…Information analysis system, 20…Processor, 21…Information production unit, 22…Start determination unit, 23…First motion information acquisition unit, 24…First position information acquisition unit, 25…Second motion information acquisition unit, 26…Second position information acquisition unit, 27…Motion information correction unit, 28…Position information correction unit, 30…Storage unit, 31…First motion information, 32…First position information, 33…Second motion information, 34…Second position information, 35…Purchase information, 36…Product information, 40…Communication interface, 50…Input / output interface, 51…Display unit, 52…Input device, 60…Image analysis device, 61…Processor 62…Image analysis unit, 63…Storage unit, 66…Communication interface, 70…Shooting device, 71…Image sensor, 80…Ear-worn device, 80a…House, 81…Motion sensor, 82…Sound conversion unit, 83…Microphone, 84…Communication interface, 90…Portable terminal device, 91…Processor, 92…Storage unit, 93…Display operation unit, 94…Location information acquisition unit, 95…Sensor unit, 96…Communication interface, 97…Information providing application, 121…Purchase information production unit, 122…Information attachment unit, ME…Product, OST…Target store, PU…Buyer, SH…Shelf, ST…Store.

Claims

1. An information analysis system, comprising: The first motion information acquisition unit acquires first motion information representing the aforementioned head motion, generated using a first sensor worn on a person's head. The first location information acquisition unit acquires the location information of the person, i.e., the first location information, which is made by using the location information acquisition unit worn by the person. The second motion information acquisition unit acquires second motion information representing the head motion, which is generated using a second sensor different from the first sensor described above. The second location information acquisition unit acquires the location information, i.e., the second location information, generated using a second sensor that is different from the first sensor described above. The motion information correction unit uses the second motion information to correct the first motion information; and The position information correction unit uses the second position information to correct the first position information.

2. The information analysis system according to claim 1, wherein, The second sensor used to generate the second motion information is the same image sensor as the second sensor used to generate the second position information. The aforementioned second action information and the aforementioned second location information are each generated using an image of the person captured by a camera device whose location is known and which is equipped with the aforementioned second sensor.

3. The information analysis system according to claim 1, further comprising: The manufacturing unit uses the first action information and the first position information to determine the object toward which the person's face is turned, and manufactures object information containing the object.

4. The information analysis system according to claim 3, wherein, The manufacturing unit produces at least one of the following object history information: (a) object history information arranged in chronological order according to the direction the person faces, and (b) object history information arranged in order from longest to shortest viewing time the person faces.

5. The information analysis system according to claim 3 or 4, wherein, The aforementioned objects are commodities. The aforementioned information analysis system also includes an information attachment unit, which attaches the identification information of the aforementioned goods to the aforementioned object information.

6. The information analysis system according to claim 3 or 4, wherein, The aforementioned object is an advertisement. The aforementioned information analysis system also includes an information appending unit, which appends the identification information of the aforementioned advertisement to the aforementioned object information.

7. The information analysis system according to claim 1, wherein, The first sensor mentioned above is a motion sensor built into an ear-worn device.

8. The information analysis system according to claim 3 or 4, further comprising: The psychological information acquisition unit acquires psychological information representing the psychological state of the aforementioned person; and The information attachment section adds the aforementioned psychological information to the aforementioned object information.

9. The information analysis system according to claim 8, wherein, The aforementioned psychological information includes at least one of the person's heart rate and facial expression.

10. The information analysis system according to claim 8, wherein, The aforementioned information appending unit uses the aforementioned psychological information to calculate an evaluation value representing the degree of concern the aforementioned person has for the aforementioned object, and appends the evaluation value to the aforementioned object information.

11. The information analysis system according to claim 4, further comprising: The sound information acquisition unit acquires and infers that the aforementioned person is listening to sound information; and The information appending unit appends the aforementioned audio information to the aforementioned object's history information.

Images