Avatar generation program, avatar generation method, and information processing device

Abstract

To set an avatar effective for a sales-promotion program.SOLUTION: An information processing device acquires a video obtained by photographing an area containing a commodity shelf where commodities are arranged. The information processing device specifies an action of a person and a characteristic of the person by analyzing the acquired video. The information processing device classifies the specified characteristic on the basis of a prescribed condition. The information processing device acquires a reception style to a commodity, the reception style represented by an avatar visibly indicated for each of persons. The information processing device calculates occurrence ratios of actions related to purchase of commodities for the reception style on the basis of the specified action for each of the classified characteristic. The information processing device updates the reception style for each of the classified characteristic on the basis of the occurrence ratios.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an avatar generation method and the like. [Background technology]

[0002] In recent years, the working population has been declining, and avatars are being used to serve customers. In addition, there is a conventional technology related to generative AI (Artificial Intelligence) technology that generates an avatar image for serving customers by specifying prompts that indicate the avatar's characteristics.

[0003] FIG. 24 is a diagram showing an example of a prompt. As shown in FIG. 24, prompt 5 includes keywords that indicate the characteristics of the avatar and the weight of the keywords. For example, in prompt 5, the weight of the keyword is set by a numerical value in parentheses. When the user specifies prompt 5 to the generation AI, avatar image 5a is generated. By changing the keywords and weight of prompt 5, an image different from avatar image 5a is generated.

[0004] For example, sales of products can be promoted by setting optimal avatars for various customers and having those avatars serve them, so it is necessary to set optimal avatars for various customers. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2023-94868 [Non-patent literature]

[0006] [Non-Patent Document 1] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (pp. 10684-10695). Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the above-mentioned conventional technology, the avatars set for customers with different personal characteristics such as age, gender, and personality may not be optimal, and there is room for improvement in the effectiveness of sales measures.

[0008] One possible method would be to analyze customer reactions when various types of avatars are used to serve customers and then select the optimal avatar. However, this method would result in a huge number of combinations of parameters, including keywords and weights in the prompt, making it unrealistic to identify the optimal prompt from such combinations.

[0009] In one aspect, the present invention aims to provide an avatar generation program, an avatar generation method, and an information processing device that can set an avatar that is effective for sales promotion. [Means for solving the problem]

[0010] In the first proposal, a computer performs the following process. The computer acquires video footage of an area including shelves on which products are placed. The computer identifies the behavior of a person and characteristics related to the person by analyzing the acquired video footage. The computer classifies the identified characteristics based on predetermined conditions. The computer acquires a customer service style for a product, which is represented by a visually displayed avatar for each person. The computer calculates, for each classified characteristic, the occurrence rate of behavior related to the purchase of a product based on the identified behavior for the customer service style. The computer updates the customer service style for each classified characteristic based on the occurrence rate. [Effects of the Invention]

[0011] You can set an avatar that is effective for sales strategies. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of a system according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a data structure of the customer service style DB according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a data structure of the occurrence rate DB according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating the first update process according to the first embodiment. [Figure 5] FIG. 5 is a diagram illustrating the second update process according to the first embodiment. [Figure 6] FIG. 6 is a functional block diagram illustrating a configuration of the information processing device according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the data structure of the imaging DB. [Figure 8] FIG. 8 is a diagram showing an example of a skeletal model of a human body. [Figure 9] FIG. 9 is a diagram showing an example of joint names. [Figure 10]FIG. 10 is a diagram illustrating an example of the data structure of the management DB. [Figure 11] FIG. 11 is a diagram showing an example of a prompt for the base portion. [Figure 12] FIG. 12 is a flowchart of the presentation process according to the first embodiment. [Figure 13] FIG. 13 is a flowchart of the update process according to the first embodiment. [Figure 14] FIG. 14 is a diagram illustrating an example of a system according to the second embodiment. [Figure 15] FIG. 15 is a diagram illustrating the clustering according to the second embodiment. [Figure 16] FIG. 16 is a diagram illustrating an example of a data structure of the customer service style DB according to the second embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of a data structure of the occurrence rate DB according to the second embodiment. [Figure 18] FIG. 18 is a functional block diagram illustrating a configuration of an information processing device according to the second embodiment. [Figure 19] FIG. 19 is a flowchart illustrating a procedure of the presentation process according to the second embodiment. [Figure 20] FIG. 20 is a flowchart of the update process according to the second embodiment. [Figure 21] FIG. 21 shows an example of an avatar image generated by another keyword in the prompt. [Figure 22] FIG. 22 is a diagram for explaining another first update process. [Figure 23] FIG. 23 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same functions as the information processing apparatus of the embodiment. [Figure 24] FIG. 24 is a diagram showing an example of a prompt. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the avatar generation program, avatar generation method, and information processing device disclosed in the present application will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to these embodiments. [Example]

[0014] An example of a system according to the first embodiment will be described. Fig. 1 is a diagram illustrating an example of a system according to the first embodiment. As shown in Fig. 1, this system includes cameras 10a, 10b, 10c, and 10d, display devices 15a, 15b, 15c, and 15d, and an information processing device 100. The cameras 10a to 10d, the display devices 15a to 15d, and the information processing device 100 are connected to one another via a network.

[0015] For convenience of explanation, FIG. 1 shows only cameras 10a to 10d and display devices 15a to 15d, but the system according to the first embodiment may further include other cameras and other display devices.

[0016] The cameras 10a to 10d are installed at predetermined positions in the store. A plurality of products are arranged in the store. The positions (coordinates) at which the cameras 10a to 10d are installed are different from each other. In the following description, the cameras 10a to 10d will be referred to as "camera 10" unless they are to be particularly distinguished from each other.

[0017] Camera 10 captures video of the inside of the store and transmits the captured video data to information processing device 100. The imaging range of cameras 10a to 10d includes product shelves and display devices 15a to 15d, respectively. In the following description, the video data transmitted by camera 10 to information processing device 100 will be referred to as "video data."

[0018] The video data includes a plurality of image frames in a time series. Each image frame is assigned a frame number in ascending chronological order. One image frame is a still image captured by the camera 10 at a certain timing. Time data may be assigned to each image frame. Camera identification information that identifies the camera 10 that captured the video data is set in the video data.

[0019] The display devices 15a to 15d are installed at predetermined positions within the store. They are devices that display various types of information based on information received from the information processing device 100. The display devices 15a to 15d are digital signage or the like. For example, the display devices 15a to 15d display images of avatars generated by the information processing device 100. In the following description, the display devices 15a to 15d will be referred to as "display devices 15" unless they are particularly distinguished from one another.

[0020] Information processing device 100 is a device that generates an avatar corresponding to the attributes of a customer and displays the generated avatar on display device 15. For example, information processing device 100 executes a presentation process and an update process.

[0021] First, a presentation process executed by information processing device 100 will be described. Information processing device 100 displays avatars (avatar images) of multiple customer service styles on display device 15 based on customer service style DB 141. For example, a customer service style is a prompt expressed by keywords and keyword weights. The keywords and weights included in the prompt are examples of "parameters."

[0022] FIG. 2 is a diagram showing an example of the data structure of the customer service style DB according to the first embodiment. As shown in FIG. 2, the customer service style DB 141 associates customer attributes, prompts, and the presentation ratio of customer service styles (prompts) to be presented to the customer. In the first embodiment, as an example, the customer attributes are a combination of the customer's age and gender. The ratio of customer service styles (prompts) to be presented to the customer is referred to as the "presentation ratio."

[0023] In FIG. 2, prompts 20-1, 20-2, and 20-3 are set as the initial values ​​of the prompts. For example, prompt 20-1 includes "cool:1.0, intelligent:1.0, posh:1.0." Prompt 20-2 is a "base" prompt. A base prompt is a prompt that does not depend on the customer service style. Prompt 20-3 includes "caring:1.0, sweet:1.0, casual:1.0." Each prompt in customer service style DB 141 is updated by an update process described below.

[0024] In Figure 2, the presentation rate for the avatar of prompt 20-1 to customers with the first attribute (women in their 20s) is "33%." The presentation rate for the avatar of prompt 20-2 to customers with the first attribute is "33%." The presentation rate for the avatar of prompt 20-3 to customers with the first attribute is "33%."

[0025] The presentation rate for the avatar with prompt 20-1 to customers with the second attribute (male in his 40s) is "33%." The presentation rate for the avatar with prompt 20-2 to customers with the second attribute is "33%." The presentation rate for the avatar with prompt 20-3 to customers with the second attribute is "33%."

[0026] For example, information processing device 100 identifies the customer's attribute by analyzing the video data from camera 10a. If the customer's attribute is a first attribute (female in her twenties), information processing device 100 selects a prompt based on the presentation rate corresponding to the first attribute in customer service style DB 141, and causes display device 15a to display an avatar image based on the selected prompt.

[0027] Returning to the explanation of FIG. 1, the update process executed by the information processing device 100 will be explained. By analyzing the video data from the camera 10, the information processing device 100 calculates the occurrence rate of a predetermined customer purchasing behavior for an avatar of each customer service style (prompt) for each customer attribute, and registers the calculated rate in the occurrence rate DB 142. The information processing device 100 updates the customer service style (prompt) for each customer attribute registered in the customer service style DB 141 based on the occurrence rate for each customer service style and each customer attribute registered in the occurrence rate DB 142. In the present Example 1, as an example, the predetermined customer purchasing behavior is the customer's "stopping behavior" in front of a product shelf.

[0028] 3 is a diagram illustrating an example of a data structure of the occurrence rate DB according to the first embodiment. As shown in FIG. 3, the occurrence rate DB 142 associates a plurality of attributes, a plurality of prompts, and the occurrence rate of stopping behavior. In FIG. 3, prompts 20-1 to 20-3 are set as initial values ​​of the prompts. The explanation of the prompts 20-1 to 20-3 is the same as that given in FIG. 2.

[0029] In Figure 3, the percentage (occurrence rate) of customers with the first attribute (women in their 20s) who stop when the avatar with prompt 20-1 is displayed is "15%." The percentage of customers with the first attribute who stop when the avatar with prompt 20-2 is displayed is "60%." The percentage of customers with the first attribute who stop when the avatar with prompt 20-3 is displayed is "25%."

[0030] The percentage of customers with the second attribute (men in their 40s) who stop when the avatar in prompt 20-1 is displayed is "70%." The percentage of customers with the second attribute who stop when the avatar in prompt 20-2 is displayed is "20%." The percentage of customers with the second attribute who stop when the avatar in prompt 20-3 is displayed is "10%."

[0031] Here, a description will be given in order of the first update process and the second update process executed by the information processing device 100. First, a description will be given of the first update process. Fig. 4 is a diagram for explaining the first update process according to the first embodiment.

[0032] 4 will be described using the occurrence rate of each stopping behavior for the first attribute (women in their 20s) in the occurrence rate DB 142. For convenience of explanation, in FIG. 4, the occurrence rates of prompts 20-1 to 20-3 for the first attribute will be described as 10%, 30%, and 50%, respectively.

[0033] For example, the position of prompt 20-1 is position p1 in the prompt feature space V1, the position of prompt 20-2 is position p2 in the prompt feature space V1, and the position of prompt 20-3 is position p3 in the prompt feature space V1.

[0034] Information processing device 100 scans each occurrence rate of the first attribute in occurrence rate DB 142 and determines whether there is a significant difference in the occurrence rate between each prompt. For example, information processing device 100 sets a threshold value (e.g., 20%) in advance, and if there is a prompt whose occurrence rate is higher than the threshold value compared to other occurrence rates, determines that there is a significant difference in the occurrence rate.

[0035] 4, the occurrence rate of prompt 20-2, "30%," is greater than or equal to the threshold value compared to the occurrence rate of prompt 20-1, "10%." Also, the occurrence rate of prompt 20-3, "50%," is greater than or equal to the threshold value of the occurrence rate of prompt 20-1, "10%." Therefore, the information processing device 100 determines that there is a significant difference in the occurrence rates in the example shown in FIG.

[0036] If the information processing device 100 determines in the first update process that there is a significant difference in the occurrence rates, it selects the prompt with the highest occurrence rate and the prompt with the second highest occurrence rate, and searches for the midpoint of the selected prompts.

[0037] 4, the information processing device 100 selects prompt 20-3 with the highest occurrence rate and prompt 20-2 with the second highest occurrence rate. In feature space V1, the information processing device 100 searches for position p4 that corresponds to the midpoint between position p3 of prompt 20-3 and position p2 of prompt 20-2. The prompt that corresponds to position p4 is set to "caring: 0.5, sweet: 0.5, casual: 0.5."

[0038] Based on the above calculation result, information processing device 100 updates the prompt corresponding to the first attribute registered in customer service style DB 141. Specifically, information processing device 100 updates the prompt corresponding to the first attribute to prompts 20-2, 20-4, and 20-3. Prompt 20-4 is set to a value corresponding to position p4 searched for in FIG. 4.

[0039] Next, the second update process will be described. Fig. 5 is a diagram for explaining the second update process according to the first embodiment. In Fig. 5, the explanation will be given using the occurrence rate of each stopping behavior of the first attribute (women in their twenties) in the occurrence rate DB 142. For convenience of explanation, in Fig. 5, the explanation will be given assuming that the occurrence rates of prompts 20-2, 20-4, and 20-3 for the first attribute are 30%, 60%, and 50%, respectively.

[0040] For example, the position of prompt 20-2 is position p2 in the prompt feature space V1, the position of prompt 20-4 is position p4 in the prompt feature space V1, and the position of prompt 20-3 is position p3 in the prompt feature space V1.

[0041] Information processing device 100 scans each occurrence rate of the first attribute in occurrence rate DB 142 and determines whether there is a significant difference in the occurrence rate between each prompt. For example, information processing device 100 sets a threshold value (e.g., 20%) in advance, and if there is a prompt whose occurrence rate is higher than the threshold value compared to other occurrence rates, determines that there is a significant difference in the occurrence rate.

[0042] 5, the occurrence rate of prompt 20-4, "60%," is greater than or equal to the threshold value compared to the occurrence rate of prompt 20-2, "30%." Also, the occurrence rate of prompt 20-3, "50%," is greater than or equal to the threshold value of the occurrence rate of prompt 20-1, "30%." Therefore, the information processing device 100 determines that there is a significant difference in the occurrence rates in the example shown in FIG.

[0043] If the information processing device 100 determines in the second update process that there is a significant difference in the occurrence rate, it selects the prompt with the highest occurrence rate and searches the vicinity of the selected prompt.

[0044] In the example shown in FIG. 5, the information processing device 100 selects prompt 20-4 with the highest occurrence rate. The information processing device 100 generates another prompt by changing the weight of one of the multiple keywords included in the selected prompt 20-4. For example, the information processing device 100 generates prompt 20-5 by changing "caring:0.5" in prompt 20-4 to "caring:0.4." The information processing device 100 generates prompt 20-5 by changing "caring:0.5" in prompt 20-4 to "caring:0.6." The position of prompt 20-5 in feature space V1 is position p5. The position of prompt 20-6 in feature space V1 is position p6. In FIG. 5, the information processing device 100 selected the keyword "caring" and changed its weight, but it may also select weights for other keywords "sweet" and "casual." The information processing device 100 may randomly select keywords whose weights are to be changed.

[0045] Based on the calculation result, information processing device 100 updates the prompt corresponding to the first attribute registered in customer service style DB 141. Specifically, information processing device 100 updates the prompt corresponding to the first attribute to prompts 20-5, 20-4, and 20-6.

[0046] Information processing device 100 repeatedly executes the first update process and the second update process described above until there is no significant difference in the occurrence rate between the prompts. Note that information processing device 100 may repeatedly execute the first update process and the second update process a preset number of times.

[0047] Furthermore, in the above processing, the case where the information processing device 100 updates the prompt for the first attribute has been described, but the information processing device 100 also updates prompts corresponding to other attributes in the same manner.

[0048] As described above, the information processing device 100 according to the first embodiment analyzes the video data from the camera 10 to calculate the occurrence rate of a predetermined customer purchasing behavior for an avatar of each customer service style (prompt) for each customer attribute, and registers the calculated rate in the occurrence rate DB 142. The information processing device 100 updates the customer service style (prompt) for each customer attribute registered in the customer service style DB 141 based on the occurrence rate for each customer service style and each customer attribute registered in the occurrence rate DB 142. This makes it possible to set an avatar that is effective for sales measures.

[0049] Next, a configuration example of the information processing device 100 that executes the above-mentioned processing will be described. Fig. 6 is a functional block diagram showing the configuration of the information processing device according to the first embodiment. As shown in Fig. 6, the information processing device 100 includes a communication unit 110, an input unit 120, a display unit 130, a storage unit 140, and a control unit 150.

[0050] The communication unit 110 executes data communication with the camera 10, the display device 15, an external device, etc. via a network. The communication unit 110 is a NIC (Network Interface Card) or the like. For example, the communication unit 110 receives video data from the camera 10.

[0051] The input unit 120 is an input device that inputs various types of information to the control unit 150 of the information processing device 100 .

[0052] The display unit 130 is a display device that displays information output from the control unit 150.

[0053] The storage unit 140 includes a customer service style DB 141, an occurrence rate DB 142, an image capture DB 143, a model DB 144, and a management DB 145. The storage unit 140 is a storage device such as a memory.

[0054] The service style DB 141 associates customer attributes, prompts, and the presentation ratio of service styles (prompts) to be presented to customers. Other explanations regarding the service style DB 141 are the same as those given for the service style DB 141 in FIG. 2.

[0055] The occurrence rate DB 142 associates a plurality of attributes, a plurality of prompts, and the occurrence rate of stopping behavior. Other details regarding the occurrence rate DB 142 are the same as those regarding the occurrence rate DB 142 described with reference to FIG.

[0056] The imaging DB 143 holds video data captured by the camera 10. FIG. 7 is a diagram showing an example of the data structure of the imaging DB. As shown in FIG. 7, the imaging DB 143 associates camera identification information with video data. The camera identification information is information that uniquely identifies the camera 10. For example, camera identification information C-10a, C-10b, C-10c, and C-10d correspond to the cameras 10a, 10b, 10c, and 10d, respectively. The video data is video data captured by the corresponding camera 10.

[0057] The model DB 144 has a plurality of machine learning models used by the control unit 150. For example, the model DB 144 has a first machine learning model, a second machine learning model, and a third machine learning model. The first machine learning model, the second machine learning model, and the third machine learning model are trained machine learning models such as a neural network (NN).

[0058] The first machine learning model is a model that outputs skeletal data of a person when an image of the person is input. The skeletal data is data in which two-dimensional or three-dimensional coordinates are set for multiple joints defined in a skeletal model of the human body. FIG. 8 is a diagram showing an example of a skeletal model of the human body. For example, as shown in FIG. 8, the skeletal model of the human body is defined by 21 joints ar0 to ar20.

[0059] The relationship between the joints ar0 to ar20 shown in FIG. 8 and the joint names is as shown in FIG. 9. FIG. 9 is a diagram showing an example of joint names. For example, the joint name of joint ar0 is "SPINE_BASE." The joint names of joints ar0 to ar20 are as shown in FIG. 9, and explanations thereof will be omitted.

[0060] The second machine learning model is a model that outputs a person's behavior type when time-series skeletal data is input. For example, the person's behavior types include "walking," "stopping," "looking at the shelves," "looking at the product," "reaching out for the product," etc.

[0061] The third machine learning model is a model that outputs the attributes (age, gender) of a person when an image of the person is input. Note that the third machine learning model may output other attributes as the attributes of the person. The other attributes include the person's age, clothing, etc.

[0062] Management DB 145 stores various information related to camera 10 and display device 15. Fig. 10 is a diagram showing an example of the data structure of the management DB. As shown in Fig. 10, management DB 145 associates camera identification information, camera installation positions, display device identification information, and display device installation positions.

[0063] The camera identification information is the same as described above. The camera installation position indicates the position (coordinates) where the camera 10 is installed. The display device identification information is information that uniquely identifies the display device 15. The display device identification information D-15a, D-15b, D-15c, and D-15d correspond to the display devices 15a, 15b, 15c, and 15d, respectively. The display device installation position indicates the position where the display device 15 is installed. Furthermore, the display device with the display device identification information included in the same record as the camera identification information of a certain camera means that it is included in the imaging range of the certain camera.

[0064] Next, we will move on to explaining the control unit 150. The control unit 150 has an acquisition unit 151, a skeleton detection unit 152, a motion detection unit 153, an attribute estimation unit 154, a behavior evaluation unit 155, an update unit 156, and an avatar generation unit 157. The control unit 150 is a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0065] The acquisition unit 151 acquires video data from the camera 10. The acquisition unit 151 stores the acquired video data in the imaging DB 143 in association with the camera identification information of the camera.

[0066] The skeleton detection unit 152 acquires video data from the imaging DB 143 and detects a person area (bounding box) from each of the time-series image frames included in the video data. The skeleton detection unit 152 generates time-series skeletal data by inputting the time-series images of the person area detected from the time-series image frames into a first machine learning model. The skeleton detection unit 152 outputs the time-series skeletal data to the action detection unit 153. The skeleton detection unit 152 sets, in the skeletal data, the camera identification information of the camera 10 that captured the video data from which the skeletal data is generated.

[0067] The movement detection unit 153 divides the time-series skeletal data into predetermined periods and inputs the time-series skeletal data for the predetermined periods into a second machine learning model, thereby detecting the behavior type of the person for the predetermined period. The movement detection unit 153 repeatedly executes the above process for each predetermined period, thereby outputting the behavior type of the person for each predetermined period to the behavior evaluation unit 155. The movement detection unit 153 sets the camera identification information set in the skeletal data as the behavior type of the person.

[0068] The attribute estimation unit 154 detects a person area (bounding box) from each of the time-series image frames included in the video data. The attribute estimation unit 154 estimates the attributes of the person by inputting the image of the person area into a third machine learning model. The attribute estimation unit 154 outputs the estimated person attributes to the behavior evaluation unit 155 and the avatar generation unit 157. The attribute estimation unit 154 sets a person ID, camera identification information of the camera 10 that captured the video data, and time data of the image frame as information on the person's attributes. The attribute estimation unit 154 performs tracking, etc. on the time-series image frames, and sets the same person ID for attributes of the same person.

[0069] Avatar generation unit 157 selects a prompt based on the attributes of the person acquired from attribute estimation unit 154 and customer service style DB 141. Avatar generation unit 157 generates an avatar image based on the selected prompt, and displays the generated avatar image on display device 15. The processing of avatar generation unit 157 corresponds to the presentation processing described above.

[0070] When the person's attribute is the first attribute (female in her twenties), the avatar generation unit 157 selects a prompt based on the presentation ratio set in the record for the first attribute in the customer service style DB 141 (FIG. 2). The avatar generation unit 157 also identifies a display device on which to display the avatar image based on the camera identification information set in the person's attribute acquired from the attribute estimation unit 154 and the management DB 145. When the camera identification information is "C-10a," the avatar image is displayed on the display device 15a with the display device identification information "D-15a."

[0071] When the avatar generation unit 157 displays an avatar image on the display device 15, it outputs to the behavior evaluation unit 155 display history information that associates information about the prompt from which the avatar image is generated, the time when the avatar image was displayed, and the display device identification information of the display device 15 that displayed the avatar image.

[0072] The behavior evaluation unit 155 updates the occurrence rate DB 142 based on the display history information, the behavior type of the person for each predetermined period acquired from the action detection unit 153, and the attribute of the person acquired from the attribute estimation unit 154.

[0073] For example, the behavior evaluation unit 155 associates, for each person, the person's attributes, information on prompts referenced by the person, and the person's behavior type based on the display history information, the person's behavior type for each predetermined period, and information on the person's attributes. The behavior evaluation unit 155 identifies the number of detected people and the number of people who performed stopping behavior for each person's attribute and prompt, and calculates the occurrence rate of stopping behavior. The behavior evaluation unit 155 updates the occurrence rate DB 142 based on the calculation results.

[0074] The update unit 156 updates the prompts in the customer service style DB 141 by executing a first update process and a second update process. The first update process executed by the update unit 156 is the same as the update process described in FIG. 4. The second update process executed by the update unit 156 is the same as the update process described in FIG. 5. The update unit 156 repeatedly executes the first update process and the second update process until there is no longer a significant difference in the occurrence rate between the prompts. Note that the update unit 156 may repeatedly execute the first update process and the second update process a preset number of times.

[0075] The update unit 156 may set any initial value of the prompt in the customer service style DB 141. For example, the update unit 156 obtains a basic prompt that does not depend on the customer service style from a prompt DB (not shown) and sets the basic prompt in the customer service style DB 141.

[0076] FIG. 11 is a diagram showing an example of a prompt in the base portion. Prompt 20-2 shown in FIG. 11 corresponds to prompt 20-2 shown in FIG. 2. Furthermore, the update unit 156 uses dictionary data to generate keywords "cool," "intelligent," and "well educated" corresponding to the customer service style "intelligent and calm customer service," and sets an initial weight value of "1.0" for each keyword, thereby generating prompt 21-2 in FIG. 2. The update unit 156 uses dictionary data to generate keywords "caring," "sweet," and "casual" corresponding to the customer service style "warm customer service," and sets an initial weight value of "1.0" for each keyword, thereby generating prompt 21-3 in FIG. 2.

[0077] Next, a description will be given of the processing procedures (presentation processing, update processing) of the information processing device 100 according to the embodiment 1. Fig. 12 is a flowchart showing the processing procedures of the presentation processing according to the embodiment 1. As shown in Fig. 12, the acquisition unit 151 of the information processing device 100 acquires video data from the camera 10 (step S101).

[0078] The attribute estimation unit 154 of the information processing device 100 detects a person area from image data included in the video data (step S102). The attribute estimation unit 154 estimates the attributes of the person based on the image of the person area (step S103).

[0079] Avatar generation unit 157 of information processing device 100 selects a prompt based on the person's attributes and customer service style DB 141 (step S104). Avatar generation unit 157 generates an avatar based on the selected prompt (step S105).

[0080] Avatar generation unit 157 displays the generated avatar on display device 15 (step S106). If information processing device 100 continues processing (step S107, Yes), it proceeds to step S101. On the other hand, if information processing device 100 does not continue processing (step S107, No), it ends processing.

[0081] Fig. 13 is a flowchart showing the processing procedure of the update process. As shown in Fig. 13, the acquisition unit 151 of the information processing device 100 acquires video data from the camera 10 (step S201). The skeleton detection unit 152 and the attribute estimation unit 154 of the information processing device 100 detect a person area from image data included in the video data (step S202). The attribute estimation unit 154 estimates the attributes of the person based on the image of the person area (step S203), and the process proceeds to step S206.

[0082] The skeleton detection unit 152 generates time-series skeleton data (step S204). The movement detection unit 153 of the information processing device 100 detects the type of behavior of the person based on the time-series skeleton data (step S205).

[0083] The behavior evaluation unit 155 of the information processing device 100 calculates the occurrence rate of stopping behavior for a combination of a person's attribute and a customer service style (prompt), and updates the occurrence rate DB 142 (step S206).

[0084] If there is no significant difference between the prompts set in occurrence rate DB 142 (step S207, No), information processing device 100 proceeds to step S201. On the other hand, if there is a significant difference between the prompts set in occurrence rate DB 142 (step S207, Yes), information processing device 100 proceeds to step S208.

[0085] The update unit 156 of the information processing device 100 executes the first update process and the second update process to update the prompt in the customer service style DB 141 (step S208). If the information processing device 100 continues the process (step S209, Yes), the information processing device 100 proceeds to step S201. On the other hand, if the information processing device 100 does not continue the process (step S209, No), the information processing device 100 ends the process.

[0086] Next, the effects of the information processing device 100 according to the first embodiment will be described. The information processing device 100 analyzes the video data from the camera 10 to calculate the occurrence rate of a predetermined customer purchasing behavior for an avatar of each customer service style (prompt) for each customer attribute, and registers the calculated results in the occurrence rate DB 142. The information processing device 100 updates the customer service style (prompt) for each customer attribute registered in the customer service style DB 141 based on the occurrence rate for each customer service style and each customer attribute registered in the occurrence rate DB 142. This makes it possible to set an avatar that is effective for sales measures.

[0087] The information processing device 100 selects, from among the multiple customer service styles for a certain attribute registered in the occurrence rate DB 142, multiple customer service styles with a higher occurrence rate than the other customer service styles, and updates the parameters of the customer service style for the certain attribute based on the parameters set for the selected multiple customer service styles. This makes it possible to bring the prompt parameters closer to parameters that are more effective for sales measures.

[0088] The information processing device 100 selects the service style with the highest occurrence rate from among multiple service styles for a certain attribute registered in the occurrence rate DB 142, and updates the service style for the certain attribute based on the parameters set for the selected service style. This makes it possible to search for parameters that are more effective for sales measures. [Example]

[0089] Next, an example of a system according to the second embodiment will be described. Fig. 14 is a diagram showing an example of a system according to the second embodiment. As shown in Fig. 14, this system includes cameras 10a to 10d, display devices 15a to 15d, and an information processing device 200. The cameras 10a to 10d, the display devices 15a to 15d, and the information processing device 100 are connected to each other via a network.

[0090] The description of the cameras 10a to 10d is the same as that in Example 1. When there is no particular need to distinguish between the cameras 10a to 10d, they will be referred to as "cameras 10."

[0091] The description of the display devices 15a to 15d is the same as in Example 1. When the display devices 15a to 15d are not particularly distinguished from one another, they are referred to as "display device 15."

[0092] The information processing device 200 identifies a cluster to which the customer belongs based on the customer's characteristics. The information processing device 200 also generates an avatar corresponding to the identified cluster and displays the generated avatar on the display device 15.

[0093] The process by which the information processing device 200 identifies the cluster to which a customer belongs will be described. The information processing device 200 performs clustering of customers based on the characteristics of multiple customers registered in advance, and classifies the customers into clusters. The customer characteristics include the customer's behavior type, customer's age, customer's gender, etc. The customer's behavior type, customer's age, and customer's gender may be converted into numerical values ​​(features) such as vectors using conventional technology.

[0094] FIG. 15 is a diagram illustrating clustering according to the second embodiment. For ease of explanation, FIG. 15 shows feature points of multiple customers on a feature space represented by two axes, feature amount (1) and feature amount (2). For example, the information processing device 200 classifies multiple feature points (customers corresponding to the feature points) into clusters cl1, cl2, and cl3 based on the k-means method or the like. Note that clusters cl1 to cl3 are merely examples, and multiple customers may be classified into three or more clusters.

[0095] Here, when classifying new customers into clusters, the information processing device 200 executes the following process. The information processing device 200 generates feature points based on the customer's behavior type toward the new customer, the customer's age, and the customer's gender. The information processing device 200 compares the distance between the generated feature points and each of the clusters cl1 to cl3 in the feature space, and identifies the cluster closest to the generated feature points as the cluster corresponding to the new customer. When calculating the distance to a cluster, the information processing device 200 may use the center of gravity of the cluster, or may use the feature points closest to the feature points of the new customer among the multiple feature points included in the cluster.

[0096] The information processing device 200 executes the presentation process and the update process in the same manner as in the first embodiment.

[0097] First, a description will be given of the presentation process executed by the information processing device 200. The information processing device 200 causes the display device 15 to display avatars (avatar images) of a plurality of customer service styles based on the customer service style DB 241.

[0098] Fig. 16 is a diagram illustrating an example of a data structure of the service style DB according to Example 2. As illustrated in Fig. 16, the service style DB 241 associates clusters, prompts, and presentation rates of service styles (prompts) to be presented to customers.

[0099] 16, prompts 20-1, 20-2, and 20-3 are set as the initial values ​​of the prompts. The explanation of the prompts 20-1 to 20-3 is the same as that given in FIG.

[0100] In FIG. 16, the presentation rate for presenting the avatar of prompt 20-1 to customers belonging to cluster cl1 is "33%." The presentation rate for presenting the avatar of prompt 20-2 to customers belonging to cluster cl1 is "33%." The presentation rate for presenting the avatar of prompt 20-3 to customers belonging to cluster cl1 is "33%."

[0101] The presentation rate for presenting the avatar of prompt 20-1 to customers belonging to cluster cl2 is "33%". The presentation rate for presenting the avatar of prompt 20-2 to customers belonging to cluster cl2 is "33%". The presentation rate for presenting the avatar of prompt 20-3 to customers belonging to cluster cl2 is "33%".

[0102] For example, the information processing device 200 identifies the customer's characteristics by analyzing the video data from the camera 10a, and identifies the cluster to which the customer belongs based on the feature points of the characteristics and the distances to each cluster. If the customer belongs to cluster cl1, the information processing device 200 selects a prompt based on the presentation rate corresponding to cluster cl1 in the customer service style DB 241, and displays an avatar image based on the selected prompt on the display device 15a.

[0103] Returning to the explanation of Figure 14, the update process executed by the information processing device 200 will be explained. By analyzing the video data from the camera 10, the information processing device 200 calculates the occurrence rate of a predetermined purchasing behavior of customers for avatars of each customer service style (prompt) for each cluster and registers it in the occurrence rate DB 242. The information processing device 200 updates the customer service style (prompt) for each cluster registered in the customer service style DB 241 based on the occurrence rate for each customer service style and for each cluster registered in the occurrence rate DB 242. In this Example 2, as an example, the predetermined purchasing behavior of a customer is the "behavior of reaching out for a product" of a customer in front of a product shelf.

[0104] Fig. 17 is a diagram showing an example of the data structure of the occurrence rate DB according to the second embodiment. As shown in Fig. 17, the occurrence rate DB 242 associates a plurality of attributes, a plurality of prompts, and the occurrence rate of stopping behavior. In Fig. 17, prompts 20-1 to 20-3 are set as the initial values ​​of the prompts. The explanation of the prompts 20-1 to 20-3 is the same as that given in Fig. 2 etc.

[0105] In FIG. 17, the percentage (occurrence rate) of customers belonging to cluster cl1 who reach out for a product when the avatar of prompt 20-1 is displayed is "15%." The percentage of customers belonging to cluster cl1 who reach out for a product when the avatar of prompt 20-2 is displayed is "60%." The percentage of customers belonging to cluster cl1 who reach out for a product when the avatar of prompt 20-3 is displayed is "25%."

[0106] The percentage (occurrence rate) of customers belonging to cluster cl2 who reach out for a product when the avatar in prompt 20-1 is displayed is "70%." The percentage of customers belonging to cluster cl2 who reach out for a product when the avatar in prompt 20-2 is displayed is "20%." The percentage of customers belonging to cluster cl2 who reach out for a product when the avatar in prompt 20-3 is displayed is "10%."

[0107] The information processing device 200 executes the first update process and the second update process in the same manner as the information processing device 100 in the embodiment 1. Note that the first update process and the second update process in the embodiment 2 are the same processes except that the user attributes (first attribute, second attribute, ...) are replaced with the clusters to which the user belongs (clusters cl1, cl2, ...), and therefore the explanation will be omitted.

[0108] As described above, the information processing device 200 according to the second embodiment analyzes the video data from the camera 10 to calculate the occurrence rate of a predetermined purchasing behavior of a customer for an avatar of each customer service style (prompt) for each cluster to which the customer belongs, and registers the calculated rate in the occurrence rate DB 242. The information processing device 200 updates the customer service style (prompt) for each cluster to which the customer belongs, which is registered in the customer service style DB 241, based on the occurrence rate for each customer service style and for each cluster to which the customer belongs, which is registered in the occurrence rate DB 242. This makes it possible to set an avatar that is effective for a sales strategy for each cluster to which the customer belongs.

[0109] Next, a configuration example of an information processing device 200 that executes the above-mentioned processing will be described. Fig. 18 is a functional block diagram showing the configuration of an information processing device according to Example 2. As shown in Fig. 18, the information processing device 200 includes a communication unit 210, an input unit 220, a display unit 230, a storage unit 240, and a control unit 250.

[0110] The explanations regarding the communication unit 210, the input unit 220, and the display unit 230 are the same as the explanations regarding the communication unit 110, the input unit 120, and the display unit 130 explained in FIG.

[0111] The storage unit 240 includes a customer service style DB 241, an occurrence rate DB 242, an image capture DB 243, a model DB 244, a management DB 245, and a customer characteristic DB 246. The storage unit 240 is a storage device such as a memory.

[0112] The service style DB 241 associates the cluster to which the customer belongs, the prompt, and the presentation ratio of the service style (prompt) to be presented to the customer. Other explanations regarding the service style DB 241 are the same as those given for the service style DB 241 in FIG. 16.

[0113] The occurrence rate DB 242 associates multiple clusters to which the user belongs, multiple prompts, and the occurrence rate of the behavior of reaching out for a product. Other details regarding the occurrence rate DB 242 are the same as those described in FIG. 17.

[0114] The imaging DB 243 holds video data captured by the camera 10. Other details regarding the imaging DB 243 are the same as those regarding the imaging DB 143 described in FIG.

[0115] The model DB 244 has a plurality of machine learning models used by the control unit 250. Other explanations regarding the model DB 244 are the same as the explanations regarding the model DB 144 in the first embodiment.

[0116] The management DB 245 holds various types of information relating to the camera 10 and the display device 15. Other explanations regarding the management DB 145 are the same as those regarding the management DB 145 explained in FIG.

[0117] The customer feature DB 246 stores a plurality of customer features in advance. The customer features include the customer's behavior type, customer's age, customer's gender, etc. The customer's behavior type, customer's age, and customer's gender are converted into numerical values ​​(features) such as vectors using conventional technology. The customer feature DB 246 is used when setting the clusters described in FIG. 15.

[0118] Next, we will move on to explaining the control unit 250. The control unit 250 has an acquisition unit 251, a skeleton detection unit 252, a motion detection unit 253, a cluster estimation unit 254, a behavior evaluation unit 255, an update unit 256, and an avatar generation unit 257. The control unit 250 is a CPU, a GPU, or the like.

[0119] The acquisition unit 251 acquires video data from the camera 10. The acquisition unit 251 stores the acquired video data in the imaging DB 243 in association with the camera identification information of the camera.

[0120] The skeleton detection unit 252 acquires video data from the imaging DB 243 and generates time-series skeleton data based on the video data. The skeleton detection unit 252 outputs the time-series skeleton data to the action detection unit 253. Other details regarding the skeleton detection unit 252 are the same as those of the skeleton detection unit 152 in the first embodiment.

[0121] The movement detection unit 253 divides the time-series skeletal data into predetermined periods and inputs the time-series skeletal data for the predetermined periods into a second machine learning model, thereby detecting the behavior type of the person for the predetermined period. The movement detection unit 253 repeatedly executes the above process for each predetermined period, thereby outputting the behavior type of the person for each predetermined period to the cluster estimation unit 254 and the behavior evaluation unit 255. Other descriptions regarding the movement detection unit 253 are the same as those of the movement detection unit 153 in the first embodiment.

[0122] The cluster estimation unit 254 performs the following processes to identify the characteristics of the customer and estimate the cluster to which the customer belongs. For example, the cluster estimation unit 254 performs pre-processing and estimation processing.

[0123] The following describes the pre-processing performed by the cluster estimation unit 254. The cluster estimation unit 254 performs clustering in advance on the characteristics of multiple customers registered in the customer characteristic DB 246, and classifies the multiple customers into multiple clusters. The clustering performed by the cluster estimation unit 254 is the same as the processing described in FIG. 15.

[0124] The following describes the estimation process executed by the cluster estimation unit 254. The cluster estimation unit 254 estimates customer attributes (age, gender) in the same manner as the attribute estimation unit 154 of the first embodiment. The cluster estimation unit 254 generates customer characteristics to be estimated as clusters based on the estimated customer attributes and the behavior type acquired from the action detection unit 253.

[0125] The cluster estimation unit 254 estimates the cluster to which the customer belongs based on the distance between the multiple clusters generated in pre-processing and the feature points of the customer's features that are the target of estimation. For example, as described in Fig. 15, the cluster estimation unit 254 compares the distance with each of the clusters cl1 to cl3, and identifies the cluster that is closest to the generated feature points as the cluster that corresponds to the customer that is the target of estimation.

[0126] The cluster estimation unit 254 outputs the estimated cluster to which the person belongs to the behavior evaluation unit 255 and the avatar generation unit 257. The cluster estimation unit 254 sets the person ID, camera identification information of the camera 10 that captured the video data, and time data of the image frame as information on the cluster to which the person belongs. The cluster estimation unit 254 performs tracking or the like on the time-series image frames and sets the same person ID for attributes of the same person.

[0127] Every time the cluster estimation unit 254 estimates a customer characteristic, it registers the customer characteristic in the customer characteristic DB 246. The cluster estimation unit 254 may execute the above pre-processing at predetermined intervals.

[0128] The avatar generation unit 257 selects a prompt based on the cluster to which the person belongs, acquired from the cluster estimation unit 254, and the customer service style DB 241. The avatar generation unit 257 generates an avatar image based on the selected prompt, and displays the generated avatar image on the display device 15. The processing of the avatar generation unit 257 corresponds to the presentation processing described above.

[0129] If the cluster to which the person belongs is cluster cl1, the avatar generation unit 257 selects a prompt based on the presentation ratio set in the record of cluster cl1 in the customer service style DB 241 (FIG. 16). Furthermore, the avatar generation unit 257 specifies a display device on which to display the avatar image based on the camera identification information set in the information on the cluster to which the person belongs, obtained from the cluster estimation unit 254, and the management DB 245. If the camera identification information is "C-10a", the avatar image is displayed on the display device 15a with display device identification information "D-15a".

[0130] When the avatar generation unit 257 displays an avatar image on the display device 15, it outputs to the behavior evaluation unit 255 display history information that associates information about the prompt from which the avatar image is generated, the time when the avatar image was displayed, and the display device identification information of the display device 15 that displayed the avatar image.

[0131] The behavior evaluation unit 255 updates the occurrence rate DB 242 based on the display history information, the behavior type of the person for each predetermined period acquired from the action detection unit 253, and the cluster to which the person belongs acquired from the cluster estimation unit 254.

[0132] For example, the behavior evaluation unit 255 associates, for each person, the cluster to which the person belongs, information on prompts referenced by the person, and the person's behavior type based on the above-mentioned display history information, the person's behavior type for each predetermined period, and information on the cluster to which the person belongs. The behavior evaluation unit 255 identifies, for each cluster to which the person belongs and each prompt, the number of people detected and the number of people who performed the behavior of reaching out for products, and calculates the occurrence rate of the behavior of reaching out for products. The behavior evaluation unit 255 updates the occurrence rate DB 242 based on the calculation results.

[0133] The update unit 256 updates the prompts in the customer service style DB 241 by executing a first update process and a second update process. The first update process executed by the update unit 256 is the update process described in FIG. 4, and is similar to the process in which person attributes are replaced with clusters. The second update process executed by the update unit 256 is the update process described in FIG. 5, and is similar to the process in which person attributes are replaced with clusters. The update unit 256 repeatedly executes the first update process and the second update process until there is no significant difference in the occurrence rate between the prompts. Note that the update unit 256 may repeatedly execute the first update process and the second update process a preset number of times.

[0134] Other processes related to the update unit 256 are the same as those described for the update unit 126 in the first embodiment.

[0135] Next, a description will be given of a processing procedure (presentation processing, update processing) of the information processing device 200 according to the present embodiment 2. Fig. 19 is a flowchart showing the processing procedure of the presentation processing according to the present embodiment 2. As shown in Fig. 19, the acquisition unit 251 of the information processing device 200 acquires video data from the camera 10 (step S301).

[0136] The skeleton detection unit 252 and the cluster estimation unit 254 of the information processing device 200 detect a person area from image data included in the video data (step S302). The skeleton detection unit 252 generates time-series skeleton data (step S303).

[0137] The action detection unit 253 of the information processing device 200 detects the type of behavior of the person based on the time-series skeletal data (step S304). The cluster estimation unit 254 estimates the attributes of the person based on the image of the person area (step S305). The cluster estimation unit 254 estimates the cluster to which the person belongs based on the image of the person area (step S306).

[0138] The avatar generation unit 257 of the information processing device 200 selects a prompt (step S307) based on the cluster to which the person belongs and the customer service style DB 241. The avatar generation unit 257 generates an avatar based on the selected prompt (step S308).

[0139] The avatar generation unit 257 displays the generated avatar on the display device 15 (step S309). If the information processing device 200 continues the process (step S310, Yes), the information processing device 200 proceeds to step S301. On the other hand, if the information processing device 200 does not continue the process (step S310, No), the information processing device 200 ends the process.

[0140] Fig. 20 is a flowchart showing the processing procedure of the update process according to the second embodiment. As shown in Fig. 20, the acquisition unit 251 of the information processing device 100 acquires video data from the camera 10 (step S401). The skeleton detection unit 252 and the cluster estimation unit 254 of the information processing device 200 detect a person area from image data included in the video data (step S402). The skeleton detection unit 252 generates time-series skeleton data (step S403).

[0141] The action detection unit 253 of the information processing device 200 detects the type of behavior of the person based on the time-series skeletal data (step S404). The cluster estimation unit 254 estimates the attributes of the person based on the image of the person area (step S405). The cluster estimation unit 254 estimates the cluster to which the person belongs based on the image of the person area (step S406).

[0142] The behavior evaluation unit 255 of the information processing device 200 calculates the occurrence rate of the behavior of reaching out for the product for the combination of the cluster to which the person belongs and the customer service style (prompt), and updates the occurrence rate DB 142 (step S407).

[0143] If there is no significant difference between the prompts set in the occurrence rate DB 242 (step S408, No), the information processing device 200 proceeds to step S401. On the other hand, if there is a significant difference between the prompts set in the occurrence rate DB 242 (step S408, Yes), the information processing device 200 proceeds to step S409.

[0144] The update unit 256 of the information processing device 200 executes the first update process and the second update process to update the prompt in the customer service style DB 241 (step S409). If the information processing device 200 continues the process (step S410, Yes), it proceeds to step S401. On the other hand, if the information processing device 200 does not continue the process (step S410, No), it ends the process.

[0145] Next, the effects of the information processing device 200 according to the second embodiment will be described. The information processing device 200 according to the second embodiment analyzes the video data from the camera 10 to calculate the occurrence rate of a predetermined purchasing behavior of a customer for an avatar of each customer service style (prompt) for each cluster to which the customer belongs, and registers the calculated results in the occurrence rate DB 242. The information processing device 200 updates the customer service style (prompt) for each cluster to which the customer belongs, which is registered in the customer service style DB 241, based on the occurrence rate for each customer service style and for each cluster to which the customer belongs, which is registered in the occurrence rate DB 242. This makes it possible to set an avatar that is effective for a sales strategy for each cluster to which the customer belongs.

[0146] Incidentally, in the above-described first and second embodiments, "cool," "base," and "caring" are used as prompt keywords, but other keywords may be used. For example, the information processing devices 100 and 200 may use "real," "semi-real," "anime," etc. as prompt keywords. FIG. 21 is a diagram showing an example of an avatar image generated by other prompt keywords. For example, the closer the weight of the prompt keyword "real" is to 1, the closer the avatar image is to image ab1. The closer the weight of the prompt keyword "semi-real" is to 1, the closer the avatar image is to image ab2. The closer the weight of the prompt keyword "amine" is to 1, the closer the avatar image is to image ab3.

[0147] In addition, the information processing device 100 (200) of this embodiment described above searches for the optimal value by dividing (interpolating) the feature space of the prompt, as explained in the first update process of Figure 4, but it may also search for the optimal value by expanding (extrapolating) the feature space of the prompt.

[0148] Fig. 22 is a diagram for explaining another first update process. Fig. 22 explains using the occurrence rate of each stopping behavior for the first attribute (women in their twenties) in occurrence rate DB 142. For convenience of explanation, Fig. 22 explains the occurrence rates of prompts 22-1 to 22-3 for the first attribute as 10%, 30%, and 50%, respectively.

[0149] For example, prompt 22-1 includes "conscientiousness:1.0, intelligent:1.0, posh:1.0." Prompt 22-2 is a "base" prompt. A base prompt is a prompt that is independent of customer service style. Prompt 22-3 includes "influence:1.0, sweet:1.0, casual:1.0."

[0150] The position of prompt 22-1 is position p7 in the prompt feature space V2. The position of prompt 22-2 is position p8 in the prompt feature space V2. The position of prompt 22-3 is position p9 in the prompt feature space V2.

[0151] The information processing device 100 (200) scans each occurrence rate of the first attribute in the occurrence rate DB 142 and determines whether there is a significant difference in the occurrence rate between each prompt. For example, the information processing device 100 sets a threshold value (e.g., 20%) in advance, and when there is a prompt whose occurrence rate is higher than the threshold value compared to other occurrence rates, it determines that there is a significant difference in the occurrence rate.

[0152] 22, the occurrence rate of prompt 22-2, "30%," is greater than or equal to the threshold value compared to the occurrence rate of prompt 22-1, "10%." Also, the occurrence rate of prompt 22-3, "50%," is greater than or equal to the threshold value of the occurrence rate of prompt 22-1, "10%." Therefore, the information processing device 100 determines that there is a significant difference in the occurrence rates in the example shown in FIG.

[0153] If the information processing device 100 determines in the first update process that there is a significant difference in the occurrence rates, it selects the prompt with the highest occurrence rate and the prompt with the second highest occurrence rate, and searches for the midpoint of the selected prompts.Furthermore, the information processing device 100 selects the prompt with the highest occurrence rate and the prompt with the second highest occurrence rate, and identifies an outer point on a line segment that passes through the points of the selected prompts.

[0154] 22, the information processing device 100 selects prompt 22-3 with the highest occurrence rate and prompt 22-2 with the second highest occurrence rate. In feature space V2, the information processing device 100 searches for position p10 that corresponds to the midpoint between position p9 of prompt 22-3 and position p8 of prompt 22-2. The prompt that corresponds to position p10 is set to "influence: 1.0, sweet: 1.0, casual: 1.0."

[0155] Furthermore, the information processing device 100 identifies a position p11 on a line segment l1 passing through positions p8 and p9, the position p11 being outside of position p9. For example, the distance between positions p9 and p11 may be a preset distance. The prompt corresponding to position p11 is set to "influence: 1.5, sweet: 1.5, casual: 1.5."

[0156] Based on the above calculation results, the information processing device 100 updates the prompts corresponding to the first attribute registered in the customer service style DB 141. Specifically, the information processing device 100 updates the prompts corresponding to the first attribute to prompts 22-2, 22-4, 22-3, and 22-5. Prompt 22-4 is set to a value corresponding to position p10 found in FIG. 22. Prompt 22-5 is set to a value corresponding to position p11 found in FIG. 22. Note that the information processing device 100 sets the presentation rate of each prompt in the customer service style DB 141 to "25%" so that the presentation rate of each prompt is equal.

[0157] Next, a description will be given of an example of a hardware configuration of a computer that realizes the same functions as the above-described information processing devices 100 and 200. Fig. 23 is a diagram showing an example of a hardware configuration of a computer that realizes the same functions as the information processing devices of the embodiment.

[0158] 23, computer 300 includes CPU 301 for executing various types of arithmetic processing, input device 302 for receiving data input from a user, and display 303. Computer 300 also includes communication device 304 for transmitting and receiving data to and from external devices via a wired or wireless network, and interface device 305. Computer 300 also includes RAM 306 for temporarily storing various types of information, and hard disk drive 307. Each of devices 301 to 307 is connected to bus 308.

[0159] The hard disk drive 307 stores an acquisition program 307a, a skeleton detection program 307b, a motion detection program 307c, an estimation program 307d, and a behavior evaluation program 307e. The hard disk drive 307 also stores an update program 307f and an avatar generation program 307g. The CPU 301 reads out each of the programs 307a to 307g and loads them into the RAM 306.

[0160] The acquisition program 307a functions as an acquisition process 306a. The skeleton detection program 307b functions as a skeleton detection process 306b. The movement detection program 307c functions as a movement detection process 306c. The estimation program 307d functions as an estimation process 306d. The behavior evaluation program 307e functions as a behavior evaluation process 306e. The update program 307f functions as an update process 306f. The avatar generation program 307g functions as an avatar generation process 306g.

[0161] The processing of the acquisition process 306a corresponds to the processing of the acquisition units 151 and 251. The processing of the skeleton detection process 306b corresponds to the processing of the skeleton detection units 152 and 252. The processing of the movement detection process 306c corresponds to the processing of the movement detection units 153 and 253. The processing of the estimation process 306d corresponds to the processing of the attribute estimation unit 154 or the cluster estimation unit 254. The processing of the behavior evaluation process 306e corresponds to the processing of the behavior evaluation units 155 and 255. The processing of the update process 306e corresponds to the processing of the update units 156 and 256. The processing of the avatar generation process 306e corresponds to the processing of the avatar generation units 157 and 257.

[0162] It should be noted that each of the programs 307a to 307g does not necessarily have to be stored in the hard disk drive 307 from the beginning. For example, each of the programs may be stored in a "portable physical medium" such as a flexible disk (FD), CD-ROM, DVD, magneto-optical disk, or IC card that is inserted into the computer 300. Then, the computer 300 may read and execute each of the programs 307a to 307g.

[0163] The following supplementary notes are further disclosed regarding the embodiments including the above examples.

[0164] (Appendix 1) Obtain a video of the area including the product shelves where the products are placed, Analyzing the acquired video to identify the behavior of a person and characteristics related to the person; classifying the identified features based on predetermined conditions; acquiring a customer service style for the product, the customer service style being represented by an avatar visibly displayed for each of the persons; calculating an occurrence rate of behavior related to the purchase of the product for each of the classified characteristics based on the identified behavior with respect to the customer service style; The customer service style is updated for each of the classified characteristics based on the occurrence rate. An avatar generation program that causes a computer to execute a process.

[0165] (Appendix 2) The calculation process calculates the occurrence rate for a plurality of customer service styles, each having different parameters set, for each of the classified features; The avatar generation program described in Appendix 1 is characterized in that the updating process selects, from among multiple service styles for a certain characteristic, multiple service styles whose occurrence rate is higher than other service styles, and executes a first updating process to update the service style for the certain characteristic based on parameters set for the selected multiple service styles.

[0166] (Appendix 3) The avatar generation program described in Appendix 2 is characterized in that the updating process selects, from among multiple service styles for the certain characteristic, the service style with the highest occurrence rate, and further executes a second updating process to update the service style for the certain characteristic based on parameters set for the selected service style.

[0167] (Appendix 4) The avatar generation program described in Appendix 3, characterized in that the update process repeatedly executes the first update process and the second update process until the occurrence rates for multiple customer service styles satisfy a predetermined condition.

[0168] (Appendix 5) The avatar generation program described in Appendix 1 is characterized in that the process of calculating the occurrence rate calculates the occurrence rate for a certain customer service style by dividing the number of times an action related to purchasing the product occurs for a certain customer service style included in multiple customer service styles, each of which has different parameters set, by the number of times the person is detected for the multiple customer service styles.

[0169] (Appendix 6) The avatar generation program according to Appendix 1, further comprising causing the computer to execute a process of setting multiple clusters by performing clustering on features related to multiple people, wherein the classification process classifies the features related to the people into one of the clusters based on the distance between the features related to the people and the multiple clusters.

[0170] (Appendix 7) Obtaining video footage of an area including product shelves where products are placed, Analyzing the acquired video to identify the behavior of a person and characteristics related to the person; classifying the identified features based on predetermined conditions; acquiring a customer service style for the product, the customer service style being represented by an avatar visibly displayed for each of the persons; calculating an occurrence rate of behavior related to the purchase of the product for each of the classified characteristics based on the identified behavior with respect to the customer service style; The customer service style is updated for each of the classified characteristics based on the occurrence rate. A method for generating an avatar, characterized in that the processing is executed by a computer.

[0171] (Appendix 8) The calculation process calculates the occurrence rate for a plurality of customer service styles, each having different parameters set, for each of the classified features; The avatar generation method described in Appendix 7 is characterized in that the updating process selects, from among multiple service styles for a certain characteristic, multiple service styles whose occurrence rate is higher than other service styles, and performs a first updating process to update the service style for the certain characteristic based on parameters set for the selected multiple service styles.

[0172] (Appendix 9) The avatar generation method described in Appendix 8 is characterized in that the updating process selects, from among multiple service styles for the certain characteristic, the service style with the highest occurrence rate, and further executes a second updating process to update the service style for the certain characteristic based on parameters set for the selected service style.

[0173] (Appendix 10) The avatar generation method described in Appendix 9, characterized in that the update process repeatedly executes the first update process and the second update process until the occurrence rates for multiple customer service styles satisfy a predetermined condition.

[0174] (Appendix 11) The avatar generation method described in Appendix 7 is characterized in that the process of calculating the occurrence rate calculates the occurrence rate for a certain customer service style by dividing the number of times an action related to purchasing the product occurs for a certain customer service style included in multiple customer service styles, each of which has different parameters set, by the number of times the person is detected for the multiple customer service styles.

[0175] (Appendix 12) The avatar generation method described in Appendix 7, further comprising causing the computer to perform a process of setting multiple clusters by performing clustering on features related to multiple people, and wherein the classification process classifies the features related to the people into one of the clusters based on the distance between the features related to the people and the multiple clusters.

[0176] (Appendix 13) Obtain a video of the area including the product shelves where the products are placed, Analyzing the acquired video to identify the behavior of a person and characteristics related to the person; classifying the identified features based on predetermined conditions; acquiring a customer service style for the product, the customer service style being represented by an avatar visibly displayed for each of the persons; calculating an occurrence rate of behavior related to the purchase of the product for each of the classified characteristics based on the identified behavior with respect to the customer service style; The customer service style is updated for each of the classified characteristics based on the occurrence rate. An information processing device having a control unit that executes processing.

[0177] (Appendix 14) The calculation process calculates the occurrence rate for a plurality of customer service styles, each of which has a different parameter set, for each of the classified features; The information processing device described in Appendix 13 is characterized in that the updating process selects, from among multiple customer service styles for a certain characteristic, multiple customer service styles whose occurrence rate is higher than other customer service styles, and performs a first updating process to update the customer service style for the certain characteristic based on parameters set for the selected multiple customer service styles.

[0178] (Appendix 15) The information processing device described in Appendix 14 is characterized in that the updating process selects the service style with the highest occurrence rate from among multiple service styles for the certain characteristic, and further executes a second updating process to update the service style for the certain characteristic based on parameters set for the selected service style.

[0179] (Appendix 16) The information processing device described in Appendix 15, characterized in that the updating process repeatedly executes the first updating process and the second updating process until the occurrence rate for multiple customer service styles satisfies a predetermined condition.

[0180] (Appendix 17) The information processing device described in Appendix 13 is characterized in that the process of calculating the occurrence rate calculates the occurrence rate for a certain customer service style by dividing the number of times behavior related to the purchase of the product occurs for a certain customer service style included in multiple customer service styles, each of which has different parameters set, by the number of times the person is detected for the multiple customer service styles.

[0181] (Appendix 18) The information processing device described in Appendix 13 is characterized in that the computer further executes a process of setting multiple clusters by performing clustering on features related to multiple people, and the classification process classifies the features related to the person into one of the clusters based on the distance between the features related to the person and the multiple clusters. [Explanation of symbols]

[0182] 100,200 Information processing equipment 110,210 Communications Department 120,220 Input section 130,230 Display section 140,240 storage section 141,241 Customer Service Style DB 142,242 occurrence rate DB 143,243 Imaging DB 144,244 Model DB 145,245 Management DB 150,250 Control unit 151,251 Acquisition Department 152,252 Skeleton detection unit 153,253 Motion detection unit 154 Attribute estimation part 155,255 Behavioral Assessment Department 156, 256 update section 157,257 Avatar Generation Unit 254 Cluster Estimation Unit

Claims

1. Obtaining video of an area including product shelves on which products are placed, Analyzing the captured video to identify behavior of a person and characteristics related to the person; Classifying the identified features based on predetermined conditions; acquiring a customer service style for the product, the customer service style being represented by an avatar visibly displayed for each of the persons; Calculating an occurrence rate of a behavior related to the purchase of the product for each of the classified characteristics based on the identified behavior with respect to the customer service style; The service style is updated for each of the classified features based on the occurrence ratio. An avatar generating program that causes a computer to execute a process.

2. The calculation process calculates the occurrence ratio for each of a plurality of service styles, each of which has a different parameter set, for each of the classified features; The avatar generation program described in claim 1, characterized in that the updating process selects, from among a plurality of service styles for a certain characteristic, a plurality of service styles whose occurrence rate is higher than other service styles, and executes a first updating process to update the service style for the certain characteristic based on parameters set for the selected plurality of service styles.

3. The avatar generation program described in claim 2, characterized in that the updating process further includes selecting, from among a plurality of service styles for the certain characteristic, a service style with the highest occurrence rate, and executing a second updating process to update the service style for the certain characteristic based on parameters set for the selected service style.

4. The avatar generation program according to claim 3, characterized in that the updating process repeatedly executes the first updating process and the second updating process until the occurrence rates for a plurality of customer service styles satisfy a predetermined condition.

5. The avatar generation program described in claim 1, characterized in that the process of calculating the occurrence rate includes dividing the number of times an action related to the purchase of the product occurs for a certain customer service style, which is included in a plurality of customer service styles, each of which has different parameters set, by the number of times the person is detected for the plurality of customer service styles, thereby calculating the occurrence rate for the certain customer service style.

6. The avatar generation program of claim 1, further comprising causing the computer to execute a process of setting multiple clusters by performing clustering on features related to multiple persons, and the classification process classifying the features related to the person into one of the clusters based on the distance between the features related to the person and the multiple clusters.

7. Obtaining video of an area including product shelves on which products are placed, Analyzing the captured video to identify behavior of a person and characteristics related to the person; Classifying the identified features based on predetermined conditions; acquiring a customer service style for the product, the customer service style being represented by an avatar visibly displayed for each of the persons; Calculating an occurrence rate of a behavior related to the purchase of the product for each of the classified characteristics based on the identified behavior with respect to the customer service style; The service style is updated for each of the classified features based on the occurrence ratio. A method for generating an avatar, the processing of which is executed by a computer.

8. Obtaining video of an area including product shelves on which products are placed, Analyzing the captured video to identify behavior of a person and characteristics related to the person; Classifying the identified features based on predetermined conditions; acquiring a customer service style for the product, the customer service style being represented by an avatar visibly displayed for each of the persons; Calculating an occurrence rate of a behavior related to the purchase of the product for each of the classified characteristics based on the identified behavior with respect to the customer service style; The service style is updated for each of the classified features based on the occurrence ratio. An information processing device having a control unit that executes processing.

Images