Behavioral data management system, behavioral data management method

Abstract

To increase the value of marketing data related to the products that customers have purchased. [Solution] The system includes an action determination unit that determines whether there is a correspondence between a first timing when an action of removing the product from the shelf is detected based on a camera image of an area including a shelf on which the product is displayed, and a second timing when the RFID tag attached to the product is no longer read by a reader set to read the vicinity of the shelf. If the action determination unit determines that there is a correspondence, the system includes a writing unit that associates attribute data of the person estimated based on the image of the person included in the camera image with product information corresponding to the RFID tag attached to the product and writes it to the storage unit.

Description

Technical Field

[0005]

[0001] The present invention relates to a behavior data management system and a behavior data management method.

Background Art

[0002] There is a system in which products attached with RFID (Radio Frequency Identification) tags are displayed on shelves, and the RFID tags are read at regular intervals by a tag reader, and based on whether the RFID tags can be read or not, it is managed whether a customer has picked up a product (for example, Patent Document 1). By referring to the identification information of the RFID tag, it is possible to grasp which products a customer has picked up, and such data can be utilized as marketing data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, although it is possible to grasp which products a customer is interested in, it is desirable to obtain more valuable marketing data.

[0005] The present invention has been made in view of such circumstances, and its object is to provide a behavior data management system and a behavior data management method that can enhance the value of marketing data regarding products picked up by a customer.

Means for Solving the Problems

[0006] To solve the above-mentioned problems, one aspect of the present invention is an action data management system having an action determination unit that determines whether there is a correspondence between a first timing in which an action of removing the product from a shelf is detected based on a camera image taken by a camera of an area including a shelf on which the product is displayed, and a second timing in which the RFID tag attached to the product is no longer read by a reader set to read the vicinity of the shelf, and if the action determination unit determines that there is a correspondence, a writing unit that associates attribute data of the person estimated based on the image of the person included in the camera image with product information corresponding to the RFID tag attached to the product and writes it to a storage unit.

[0007] Furthermore, one aspect of the present invention is a behavioral data management method performed by a computer, which includes determining whether there is a correspondence between a first timing when an action of removing a product from a shelf is detected and a second timing when an RFID tag attached to the product is no longer read by a reader set to read the vicinity of the shelf, based on a camera image taken by a camera of an area including a shelf on which products are displayed, and if such a correspondence is determined, writing to a storage unit an association between attribute data of a person estimated based on an image of the person included in the camera image and product information corresponding to the RFID tag attached to the product. [Effects of the Invention]

[0008] As explained above, this invention allows us to obtain not only data about the product that has been handled, but also attribute data of the person who has handled the product, thereby increasing the value of marketing data. [Brief explanation of the drawing]

[0009] [Figure 1] This is a conceptual diagram illustrating the object for which behavioral data is managed by a behavioral data management system according to one embodiment of this invention. [Figure 2]This is a schematic system configuration diagram showing the configuration of the behavioral data management system S. [Figure 3] This is a schematic functional block diagram illustrating the functions of the behavioral data management system S. [Figure 4] This figure shows an example of behavioral data stored in the memory unit 102. [Figure 5] An example of a sequence diagram illustrating the operation of the behavioral data management system S is shown. [Figure 6] This figure shows an example of an activity log. [Figure 7] This is a schematic diagram illustrating the process of taking product Me from shelf Sh and returning it. [Figure 8] This is a schematic diagram illustrating the case where product Me is taken from shelf Sh and the user leaves the vicinity of shelf Sh without returning it. [Figure 9] This figure shows an example of a read log when the activity log is in Case 1. [Figure 10] This figure shows an example of a read log when the activity log is in Case 2. [Figure 11] This is a schematic diagram illustrating the case where the device is determined to have passed through (no pickup operation). [Figure 12] This figure shows an example of an action log for Case 3. [Modes for carrying out the invention]

[0010] The following describes an action data management system according to one embodiment of the present invention with reference to the drawings. Figure 1 is a conceptual diagram illustrating the object for which behavioral data is managed by a behavioral data management system according to one embodiment of the present invention. The location where the behavioral data management system manages behavioral data can be any location that allows for the understanding of the relationship between product and person attributes, such as a store. The store could be any of the following: a bookstore, cosmetics store, general merchandise store, clothing store, etc.

[0011] For example, a shelf Sh is installed in the store to be managed. The shelf Sh has at least one shelf board, and at least one product Me is placed on the shelf board. One RFID (Radio Frequency Identification) tag T is attached to each product Me. The RFID tag stores individual identification information, and by using this identification information, it is possible to identify the product to which it is attached individually. A tag reader TR is provided on shelf Sh. The tag reader TR reads identification information from the RFID tag T. The range in which the tag reader TR can read is the area including shelf Sh and the product Me placed on the shelf of shelf Sh, but is limited to the area in which reading becomes impossible when product Me is removed from the shelf by person P. For example, when product Me is placed on the shelf of shelf Sh, the RFID tag T can be read, but when person P approaches the vicinity of shelf Sh, becomes interested in product Me placed on the shelf, picks up product Me from the shelf, and examines product Me up close, the RFID tag T becomes unreadable. The number of tag readers TR installed on a single shelf Sh may be one or multiple, as long as their reading range can cover the area on the shelf Sh where products can be placed.

[0012] Additionally, camera C will be mounted above shelf Sh. Camera C may be mounted on the top of shelf Sh, or on a nearby wall or ceiling above shelf Sh. Camera C photographs a person P approaching the vicinity of shelf Sh, generates image features of the person included in the captured image, and estimates the person's attributes based on these features. The attributes include at least one of the following: the person's age and gender. The shooting range Re of the camera C includes the range near the shelf Sh. For example, the shooting range Re of the camera C is set to a range that can grasp the movement line of the person P arriving near the shelf Sh and moving from near the shelf Sh to other places. Such a shooting range Re is about the same as the range that the tag reader TR can read, or is larger than the range that the tag reader TR can read. Also, the shooting range Re may be the range itself photographed by the camera C, or may be the range in which the area near the shelf Sh is extracted from the photographed range.

[0013] The display D is provided near the shelf Sh and displays various contents. The contents displayed by the display D are visible to the person P who has arrived near the shelf Sh. The person P is a person who arrives at the store and strolls inside the store.

[0014] Figure 2 is a schematic system configuration diagram showing the configuration of the behavior data management system S. The behavior data management system S includes a behavior data management device 1, a camera C, a tag reader TR, RFID tags T (for example, RFID tags T1, ···, RFID tags Tn, etc.), a content server CS, a display D, and a network NW. The behavior data management device 1 generates behavior data related to the behavior of the person P based on the imaging result of the camera C and the reading result of the tag reader TR. The behavior data management device 1 may be a physical server or a cloud server provided by a cloud computing service.

[0015] The camera C is communicably connected to the behavior data management device 1 via the network NW and transmits an analysis result based on the shooting data to the behavior data management device 1. The tag reader TR is communicably connected to the behavior data management device 1 via the network NW and transmits a reading result (for example, identification information) obtained by reading the RFID tag T to the behavior data management device 1. Each RFID tag (RFID tag T1, ..., RFID tag Tn) is attached to each product.

[0016] The content server CS stores various types of content. The content is related to products to which RFID tags T are attached. The content may include, for example, video data introducing a product, or video data introducing another product related to the product to which the RFID tag T is attached. Such content is pre-stored in the content server CS, with multiple different pieces of content associated with each product according to its attributes, and the content corresponding to the attributes can be extracted. The content server CS is connected via a network NW and receives attribute data representing the attributes of a person P obtained from the camera C or behavioral data management device 1. Display D is connected to the content server CS via the network NW and displays content transmitted from the content server CS. A network (NW) includes at least one of the following communication networks: a LAN (Local Area Network), the Internet, etc. A network (NW) also includes at least one of either wired or wireless connections.

[0017] Figure 3 is a schematic functional block diagram illustrating the functions of the behavioral data management system S. The behavioral data management system S includes a communication unit 101, a storage unit 102, a first timing generation unit 103, a second timing generation unit 104, a behavior determination unit 105, a writing unit 106, and a control unit 107. The communication unit 101 is a function that enables the camera C, tag reader TR, content server CS, etc., to communicate via the network NW.

[0018] The memory unit 102 stores behavioral data. Figure 4 shows an example of behavioral data stored in the memory unit 102. Behavioral data is data that associates attribute data of a person estimated by camera C with product information corresponding to the RFID tag attached to the product. Attribute data includes data corresponding to at least one of the attribute items, such as age and gender. Examples include data associated with "20s" and "female," data associated with "30s" and "female," data associated with "20s" and "male," and data associated with "40s" and "male." Product information may be the identification information of an RFID tag attached to the product, or it may be at least one of the following: the name of the product, manufacturer, price, specifications, etc., as identified by the identification information.

[0019] The storage unit 102 is composed of a storage medium, such as an HDD (Hard Disk Drive), flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), RAM (Random Access read / write Memory), ROM (Read Only Memory), or any combination of these storage media. This storage unit 102 can, for example, use non-volatile memory.

[0020] The first timing generation unit 103 generates the first timing if it detects an action of taking out a product during a stay period, which is indicated by the first time when a person enters the area to be managed based on the shelf and the second time when the person leaves the area, based on the camera image. The area to be managed is the area that the camera can capture. The first timing generation unit 103 generates the first timing, making it possible to use the period during which person P was being captured by camera C as a timestamp.

[0021] The second timing generation unit 104 generates a second timing period, which is the period indicated by the third time when the RFID tag attached to the product is no longer readable by the reader, and the fourth time when it becomes readable again. The second timing generation unit 104 generates the third time as the second timing if, after the third time, when the RFID tag attached to the product is no longer read by the reader, it is not read again even after the first timing has elapsed. The second timing generation unit 104 generates a second timing, which can be used as a timestamp indicating that product Me, to which the RFID tag T is attached, was picked up by person P.

[0022] The action determination unit 105 determines, based on a camera image of the area including the shelf on which the products are displayed, whether there is a correspondence between a first timing when an action of removing a product from the shelf is detected and a second timing when the RFID tag attached to the product is no longer being read by a reader set to read the vicinity of the shelf. The action determination unit 105 determines whether the first timing generated by the first timing generation unit 103 corresponds to the second timing. The action determination unit 105 determines that there is a corresponding relationship if the second timing falls within the period indicated by the first timing.

[0023] If the action determination unit 105 determines that there is a correspondence, the writing unit 106 associates the attribute data of the person estimated based on the image of the person included in the camera image with product information corresponding to the RFID tag attached to the product and writes it to the storage unit. The writing unit 106 does not write attribute data if the action determination unit 105 determines that there is no corresponding relationship.

[0024] The control unit 107 controls each function in the behavioral data management system S.

[0025] The communication unit 101 and control unit 107 are provided in the camera C, tag reader TR, content server CS, and display D respectively, enabling communication between the electronic devices and control of data exchange. The memory unit 102 may be provided in the behavioral data management device 1, or it may be provided in a separate device (for example, a database server). The first timing generation unit 103 may be provided in the camera C, or it may be provided in the behavioral data management device 1, but in this embodiment, the case in which it is provided in the camera C will be described. The second timing generation unit 104 may be provided on the tag reader TR or on the behavior data management device 1, but in this embodiment, the case where it is provided on the tag reader TR will be described. The behavior determination unit 105 and the writing unit 106 may be provided in the behavior data management device 1.

[0026] Furthermore, the communication unit 101, the first timing generation unit 103, the second timing generation unit 104, the action determination unit 105, and the writing unit 106 may be composed of a processing unit such as a CPU (Central Processing Unit) or a dedicated electronic circuit.

[0027] Figure 5 shows an example of a sequence diagram illustrating the operation of the behavioral data management system S. Here, we will explain the case where a person P arrives near shelf Sh. Camera C starts taking images within the shooting range Re (step S100), and tag reader TR starts reading RFID tags T within the reading range (step S200). When camera C starts shooting, it analyzes whether the captured image contains a person. If the captured image contains a person, it detects the time when the person was detected as the first time, monitors whether the person has left the shooting range Re, and if the person has left, it detects the time when the person's departure was detected as the second time.

[0028] Furthermore, camera C analyzes movement patterns by analyzing images of the person taken between the first and second time points (step S101). There are three types of movement patterns, for example, "passing through," "lingering (with pick-up action)," and "lingering (without pick-up action)." "Passing through" occurs when a person P who has arrived near shelf Sh leaves the vicinity of shelf Sh without lingering near shelf Sh or picking up product Me. The determination of whether or not a person is lingering may also be based on whether or not the period from the first to the second time point exceeds a predetermined time. If the period from the first to the second time point exceeds a predetermined time (for example, 5 seconds, 10 seconds, etc.), it may be determined that the person is lingering, and if it does not exceed that time, it may be determined that the person is passing through rather than lingering. Alternatively, the determination of whether or not a person is lingering may be made by analyzing images of the person captured by camera C, rather than based on whether or not a predetermined period has been exceeded. For example, camera C may analyze the image of a person to determine whether the person's legs are still. If the legs are still, it may determine that the person is stationary. If the legs are not still (for example, walking), it may determine that the person is not stationary. If it is determined that the person is stationary, it may then determine whether a pick-up action occurred based on whether or not the action of picking up the product as described above was detected.

[0029] If it is determined that there is a delay, camera C further determines whether or not a pickup action occurred, thereby determining whether the movement pattern is "delay (with pickup action)" or "delay (without pickup action)". "Lingering (with pick-up action)" occurs when a person P approaches the vicinity of shelf Sh, stops for a certain period of time or longer, and is detected to be picking up product Me. "Lingering (no pickup action)" occurs when a person P approaches the vicinity of shelf Sh, lingers near shelf Sh for a certain period of time or longer, but leaves without picking up product Me. A camera capable of analyzing such movement patterns could, for example, be equipped with an AI (artificial intelligence) engine that analyzes a person's posture and movement from captured video data.

[0030] Furthermore, camera C analyzes the images of the person taken between the first and second time points to generate features, estimates the attributes of the person based on the generated features (step S102), and generates attribute data. Here, camera C may estimate the attributes of each person P that comes into the imageable range, or it may estimate the attributes of a person P whose movement pattern analysis result is "staying (with pickup action)".

[0031] Camera C then transmits the type of movement pattern, first timing data indicating the first timing including the first and second time points, and attribute data to the behavioral data management device 1 (step S103). Camera C may transmit the type of movement pattern, the first timing, and attribute data to the behavioral data management device 1 regardless of the type of movement pattern. Camera C may also choose not to transmit this data if the analysis result of the movement pattern is "passing through" or "staying (no pickup action)," but only if the type of movement pattern is "staying (with pickup action)." The processes from step S101 to step S103 are executed each time a person is detected by camera C.

[0032] When the behavior data management device 1 receives first timing data, attribute data, and movement pattern type from camera C, it associates the first timing data, attribute data, and movement pattern type and stores them in the storage unit 102 as a behavior log (step S300). Figure 6 shows an example of an activity log. The activity log includes data such as the first timing (first time, second time), attribute data (in this case, age, gender), and movement pattern. Figure 6 illustrates a case where data was recorded indicating that between the first and second time points, a person who is in their 20s and female stayed near shelf Sh and performed a pick-up action, such as lifting an item. Such an activity log may be recorded each time person P approaches the vicinity of shelf Sh and performs a pick-up action for an item.

[0033] Meanwhile, the tag reader TR reads the RFID tags included in the reading range at regular intervals (for example, every few seconds) (step S201). In this case, if multiple products are displayed on shelf Sh, the identification information of multiple RFID tags T corresponding to the displayed products can be read. The tag reader TR compares the multiple identification pieces read with the multiple identification pieces read at the next reading timing, and determines whether there are any RFID tags T that can no longer be read based on whether there are any differences in the comparison result. If there are any identification pieces that can no longer be read, the tag reader TR detects the identification pieces that can no longer be read and extracts the time when they became unreadable as a third time point.

[0034] When the third time is extracted, the tag reader TR transmits the extracted third time as second timing data representing the second timing to the behavioral data management device 1, and also transmits the identification information of the unreadable RFID tag T to the behavioral data management device 1 (step S202).

[0035] When the behavioral data management device 1 receives the second timing data, which includes the third time, from the tag reader TR, it sends a content playback instruction to the content server CS along with the attribute data obtained when the first timing data was received (step S301).

[0036] When the content server CS receives attribute data and a content playback instruction, it sends the content corresponding to the received attribute data and the content playback instruction to the display D (step S400). When display D receives content and a playback command from content server CS, it plays the received content and displays the played content on the display screen (step S500). As a result, content corresponding to the attributes of person P who picked up product Me from shelf Sh is selected and played, and person P is displayed on display D in a visible manner. Person P who picked up the product can view content related to the product picked up from shelf Sh that corresponds to their own attributes, while confirming the product Me they are holding.

[0037] On the other hand, the tag reader TR determines whether it was able to read the unreadable identification information again within a certain time period from the third time step. Here, it sequentially refers to the reading results that were read repeatedly at certain time intervals and determines whether the unreadable identification information has become readable again. The certain time period may be a predetermined time (for example, 1 minute, 3 minutes, 5 minutes, etc.).

[0038] If the tag reader TR is able to read the identification information that had become unreadable, it extracts the time at which it was read as the fourth time point and transmits the second timing data, which includes the fourth time point, along with the identification information to the behavioral data management device 1 (step S203). Here, Figure 7 is a schematic conceptual diagram showing the case where product Me is taken from shelf Sh and returned (Case 1). When person P picks up product Me from shelf Sh, examines product Me, and then returns product Me to shelf Sh, the identification information of the RFID tag T attached to product Me becomes unreadable. Subsequently, as product Me is returned to shelf Sh, the identification information of the RFID tag T becomes readable again. In this Case 1, it is presumed that person P's actions were to show interest in product Me, pick it up and examine it, but decide not to purchase it and return product Me to shelf Sh. In this Case 1, when person P picks up product Me, the content is displayed on display D because the identification information of the RFID tag T attached to product Me can no longer be read. Therefore, person P can check both the actual product Me and the content before deciding whether or not to purchase the product.

[0039] Figure 8 is a schematic conceptual diagram illustrating the case (Case 2) where person P takes product Me from shelf Sh and leaves the vicinity of shelf Sh without returning it. If person P picks up product Me from shelf Sh, examines the product, and decides to purchase product Me, they place it in a shopping basket, carry it to the cashier, and pay. Therefore, after the identification information of the RFID tag T becomes unreadable, it remains unreadable. In this Case 2, it is presumed that person P's actions are: becoming interested in product Me, picking up product Me to examine it, deciding to purchase it, taking product Me, and heading to the cashier. In this Case 2, when person P picks up product Me, the content is displayed on display D because the identification information of the RFID tag T attached to product Me can no longer be read. Therefore, person P can check both the actual product Me and the content before deciding whether or not to purchase the product.

[0040] On the other hand, when the behavioral data management device 1 receives second timing data including the third time and identification information from the tag reader TR, it associates the second timing data and identification information and stores them in the storage unit 102 as a read log. Here, if the behavioral data management device 1 receives second timing data representing the fourth time and identification information from the tag reader TR at the fourth time, after the third time, because the RFID tag T that could not be read has become readable, it associates the second timing data including the fourth time and identification information and stores them in the storage unit 102 as a read log (step S302).

[0041] Figure 9 shows an example of a read log when the behavior log is Case 1. The read log includes second timing data (third and fourth time), identification information, and product information. The third time indicates the time when the product was removed (the time when reading became impossible), and the fourth time indicates the time when the product was returned (the time when reading became possible again). The identification information is the identification information (EPC: Electronic Product Code) of the RFID tag T attached to the removed product. The product information is information about the product to which the RFID tag T is attached. The relationship between the product information and the RFID tag identification information may be stored in advance in the storage unit 102 as a product information database, and the behavior data management device 1 may extract it based on this product information database and store it in the behavior log. The reading log shown in Figure 9 indicates that at the third time step, person P took book A, which has an RFID tag with identification information "000001" attached, from shelf Sh, checked product Me, and then at the fourth time step, returned product Me to its original display position on shelf Sh.

[0042] Figure 10 shows an example of a read log in the case of behavior log 2. In the read log shown in Figure 10, data for the 4th time point is not written. Therefore, it indicates that after person P took book A, which has an RFID tag with identification information "000001" attached, from shelf Sh at the 3rd time point, a certain amount of time (for example, the 2nd time point) has elapsed but product Me has not been returned to its original display position on shelf Sh. In this case 2, it can be inferred that after product Me was taken from shelf Sh by person P, it was taken to the cash register and purchased.

[0043] The behavioral data management device 1 determines whether a correspondence exists by determining whether the second timing falls within the period indicated by the first timing data (step S303). For example, the behavior determination unit 105 of the behavior data management device 1 determines whether the fourth time is stored before the second time (step S304), and if the fourth time is stored, it determines whether the third time corresponding to this fourth time is later than the first time. More specifically, the behavior determination unit 105 determines whether both of the following first and second conditions are met. 1st time < 3rd time... 1st condition 2nd time>4th time...2nd condition

[0044] Case 1 The behavior determination unit 105 determines, if both the first and second conditions are met, that the first timing data and the second timing data are in a corresponding relationship and that the behavior corresponds to Case 1 (step S305), and stores the attribute data and product information as behavior data in the storage unit 102 (step S306). If at least one of the first and second conditions is not met, the behavior determination unit 105 may determine that there is no correspondence and may not store the behavior data. In addition, the behavior data management device 1 may delete the first timing data, attribute data, and second timing used to determine the correspondence from the storage unit 102 if there is no correspondence.

[0045] The behavior determination unit 105 of the behavior data management device 1 determines whether the third time is later than the first time if the fourth time is not stored between the second and second time points. More specifically, the behavior determination unit 105 determines whether both of the following third and fourth conditions are met. 1st time < 3rd time... 3rd condition The fourth time zone has not been acquired... Fourth condition

[0046] Case 2 The behavior determination unit 105 determines, if the third and fourth conditions are met, that the first timing data and the second timing data are in a corresponding relationship and that the behavior falls under Case 2 (step S305), and stores the attribute data and product information as behavior data in the storage unit 102 (step S306). If at least one of the third and fourth conditions is not met, the behavior determination unit 105 may determine that there is no correspondence and may not store the behavior data. In addition, the behavior data management device 1 may delete the first timing data, attribute data, and second timing used to determine the correspondence from the storage unit 102 if there is no correspondence.

[0047] Case 3 Figure 11 is a schematic diagram illustrating the case where the movement pattern analysis result for step S101 determines that the movement was simply a pass (no pick-up action). Here, it shows the situation when person P enters the area where camera C can capture images, but leaves the area without picking up any of the products on shelf Sh. In this case, since product Me remains on shelf Sh, it is not necessary for any content to be displayed on display D, or predetermined content may be displayed.

[0048] Figure 12 shows an example of an action log. In Figure 12, a person who is in their 20s and female approaches shelf Sh between the first and second time points, but leaves the vicinity of shelf Sh without performing a pick-up action such as lifting an item. Action logs that do not involve such pick-up actions may be kept in the memory unit 102 or deleted.

[0049] According to the embodiments described above, shelves installed in stores can be used as smart shelves, and by integrating them with AI image recognition technology and combining them with information collected from RFID tags linked to products, behavioral data can be used as effective marketing data.

[0050] Furthermore, according to the embodiment described above, the tag reader TR determines, based on various timings, whether there is a correspondence between product information corresponding to the identification information of an RFID tag that has become unreadable and attribute data of a person estimated from photographic data of a person holding the product. As a result, if a correspondence exists, the product information and the person's attribute data can be associated and stored, making it possible not only to identify the product held by a person but also to understand what kind of person the person holding the product is. This increases the value of the data as marketing data compared to conventional methods. Furthermore, it is possible to display content tailored to the attributes of the person who has picked up the product, depending on when the person has picked up the product. This allows for a deeper understanding of the product and can also generate greater interest in it.

[0051] The behavioral data management device 1 in the above-described embodiment may be implemented using a computer. In that case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed. Here, "computer system" includes hardware such as an OS and peripheral devices. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into a computer system. Moreover, "computer-readable recording medium" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside a computer system that acts as a server or client in such cases. Furthermore, the above-mentioned program may be for implementing a part of the above-mentioned function, or it may be a program that can implement the above-mentioned function in combination with a program already recorded in the computer system, or it may be implemented using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0052] Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention. [Explanation of Symbols]

[0053] 1. Behavioral data management device 101 Communications Department 102 Storage section 103 First Timing Generation Unit 104 Second Timing Generation Unit 105 Behavior Judgment Department 106 Insertion part 107 Control Unit C Camera CS Content Server D Display Me products NW Network P person S Behavioral Data Management System Sh shelf T, T1, Tn RFID tags TR Tag Leader

Claims

1. Based on a camera image taken by a camera of an area including a shelf on which products are displayed, a first timing is detected in which an action is taken to remove the product from the shelf, The second timing is when the RFID tag attached to the aforementioned product is no longer read by the reader set to read the vicinity of the shelf, An action determination unit that determines whether or not there is a corresponding relationship, When the behavior determination unit determines that a correspondence exists, the writing unit writes the attribute data of the person estimated based on the image of the person included in the camera image and the product information corresponding to the RFID tag attached to the product to the storage unit. A behavioral data management system that includes [a specific feature / feature].

2. Based on the aforementioned camera image, if an action of taking out the product is detected during a period of stay indicated by a first time when a person enters the area to be managed relative to the shelf and a second time when the person leaves the area, the system has a first timing generation unit that generates the period of stay as the first timing. The aforementioned action determination unit, It is determined whether the first timing generated by the first timing generation unit corresponds to the second timing. The behavioral data management system according to claim 1.

3. The system includes a second timing generation unit that generates a second timing period, which is defined by a third time when the RFID tag attached to the product is no longer readable by the reader and a fourth time when it becomes readable again. The aforementioned action determination unit, A correspondence is determined when the second timing falls within the period indicated by the first timing. The behavioral data management system according to claim 2.

4. The second timing generation unit is: If, after the third time point in which the RFID tag attached to the product is no longer read by the reader, the first timing point has elapsed and the RFID tag is not read again, the third time point is generated as the second timing point. The aforementioned action determination unit, A correspondence is determined when the second timing falls within the period indicated by the first timing. The behavioral data management system according to claim 3.

5. The writing section is, If the aforementioned action determination unit determines that there is no corresponding relationship, the attribute data will not be written. The behavioral data management system according to any one of claims 1 to 4.

6. The area under management is the area that the camera can image. The behavioral data management system according to claim 2.

7. A method for managing behavioral data performed by a computer, Based on a camera image taken by a camera of an area including a shelf on which products are displayed, a first timing is detected in which an action is taken to remove the product from the shelf, The second timing is when the RFID tag attached to the aforementioned product is no longer read by the reader set to read the vicinity of the shelf, Determine whether or not they are in a corresponding relationship. If the aforementioned correspondence is determined to exist, the attribute data of the person estimated based on the image of the person included in the camera image and the product information corresponding to the RFID tag attached to the product are associated and written to the storage unit. A method for managing behavioral data, including the following.

Images