Information processing program, information processing method, and information processing device

Abstract

This information processing device acquires video including a person to be detected and acquires person information which is output using a prescribed video analysis technique and which is associated with at least one of attribute information and behavior information of the person included in the video. The information processing device inputs a prompt based on the person information to an agent that generates, in response to the inputted information, an advertisement for the person included in the video. The information processing device verifies, on the basis of the person information, whether a prescribed effect is obtained with respect to the advertisement which is output as a response to the prompt. The information processing device changes the prompt by using the relationship among the verification result, the advertisement which is output as the response, and the person information.

Description

Information Processing Program, Information Processing Method, and Information Processing Apparatus

[0001] The present invention relates to an information processing program and the like.

[0002] In retail stores, by displaying advertisements on digital signage, the staying time of people is extended, and the purchase rate and purchase amount are increased. The people are customers and the like.

[0003] For example, as a conventional technique for displaying an advertisement, there is a technique of analyzing a camera image to estimate attribute information such as the gender and age of a person, and displaying an advertisement corresponding to the attribute information based on the relationship between the pre-learned attribute information and the advertisement.

[0004] Japanese Patent Application Laid-Open No. 2022-70422

[0005] However, the above-described conventional technique has a problem that an optimal advertisement cannot be displayed for each person.

[0006] For example, in the conventional technique, since the relationship between the pre-learned attribute information and the advertisement is used, it is difficult to display an optimal advertisement according to the situation of people in a retail store.

[0007] In one aspect, an object of the present invention is to provide an information processing program, an information processing method, and an information processing apparatus that can display an optimal advertisement for each person.

[0008] In the first aspect, the computer is caused to execute the following processing. The computer acquires a video including a person to be detected, and acquires person information in which at least one of the attribute information and the action information of the person included in the video, which is output by a predetermined video analysis technique, is associated. The computer inputs a prompt based on the person information to an agent that generates an advertisement for the person included in the video according to the input information. The computer verifies whether a predetermined effect can be obtained based on the person information for the advertisement output as an answer to the prompt. The computer changes the prompt using the relationship between the verification result, the advertisement output as an answer, and the person information.

[0009] An optimal advertisement can be displayed for each person.

[0010] Figure 1 is a diagram showing the system according to this embodiment. Figure 2 is a diagram illustrating the processing of the information processing device according to this embodiment. Figure 3 is a diagram showing an example of the data structure of the layout data. Figure 4 is a diagram showing an example of the data structure of the history DB. Figure 5 is a diagram providing supplementary explanation for the LLM processing in step S12. Figure 6 is a diagram showing an example of the data structure of the person information list. Figure 7 is a diagram showing an example of the data structure of the presented advertisement list. Figure 8 is a diagram providing supplementary explanation for the LLM processing in step S17. Figure 9 is a diagram showing an example of the data structure of the history DB after updating. Figure 10 is a diagram (1) providing supplementary explanation for the video analysis unit. Figure 11 is a diagram (2) providing supplementary explanation for the video analysis unit. Figure 12 is a diagram (3) providing supplementary explanation for the video analysis unit. Figure 13 is a diagram (4) providing supplementary explanation for the video analysis unit. Figure 14 is a diagram (5) providing supplementary explanation for the video analysis unit. Figure 15 is a diagram (6) providing supplementary explanation for the video analysis unit. Figure 16 is a functional block diagram showing the configuration of the information processing device according to this embodiment. Figure 17 is a flowchart showing the processing procedure of the information processing device according to this embodiment. Figure 18 is a diagram showing an example of a computer hardware configuration that realizes the same functions as the information processing device in the embodiment.

[0011] The following describes in detail, with reference to the drawings, embodiments of the information processing program, information processing method, and information processing apparatus disclosed in this application. However, this invention is not limited by these embodiments.

[0012] An example of a system according to this embodiment will be described. Figure 1 is a diagram showing the system according to this embodiment. As shown in Figure 1, this system 5 includes a camera 10, a display device 15, and an information processing device 100. The camera 10, the display device 15, and the information processing device 100 are interconnected via a network 3.

[0013] For the sake of explanation, only the camera 10 and the display device 15 are shown in Figure 1, but the system 5 may have other cameras and other display devices.

[0014] Camera 10 captures images of the store interior and transmits the captured image data to the information processing device 100. In the following explanation, the image data transmitted by camera 10 to the information processing device 100 will be referred to as "image data".

[0015] The video data contains multiple frames in a time series. Each frame is assigned a frame number in ascending order of time. Each frame is a still image captured by camera 10 at a specific point in time. Each frame may also be assigned time data.

[0016] The display device 15 is installed in a designated location within the store. The display device 15 is a digital signage or the like. For example, the display device 15 displays advertisements generated by the information processing device 100.

[0017] The information processing device 100 acquires video data from the camera 10, generates advertising data by performing the following processing, and outputs the generated advertising data to the display device 15 for display.

[0018] Figure 2 is a diagram illustrating the processing of an information processing device according to this embodiment. For example, the information processing device 100 includes a video analysis unit 151, an LLM (Large Language Models) 152, and an advertisement display unit 153. For example, the information processing device 100 executes processing in the order of S10 to S20.

[0019] The processing of the information processing device 100 in step S10 of Figure 2 will be explained. The video analysis unit 151 of the information processing device 100 acquires pre-specified layout data 141. The layout data 141 is data relating to the layout of the store, and areas such as entrances, exits, cash registers, display devices 15, and aisles are set.

[0020] Figure 3 shows an example of the data structure of layout data. As shown in Figure 3, the layout data 141 is set as Area-1, Area-2, Area-3, Area-4, Area-5, Area-6, Area-7, Area-8, Area-9, Area-10, Area-11, Area-12, Area-13, Area-14, Area-15, Area-16, Area-17, Area-18, Area-19, Area-20, Area-21, Area-22, Area-23, Area-24, Area-25, Area-26, Area-27, Area-28, Area-29, and Area-30. Area-1 is the exit of the store. Area-2 is the entrance of the store. Area-25 is the area in front of the cash register 16. Area-18 is the area in front of the display device 15. The layout data 141 also includes location information for the product shelves 17.

[0021] In this embodiment, for the sake of explanation, a person entering Area-2 means entering the store. The moment a person enters Area-18, the advertisement displayed on the display device 15 is replaced with an advertisement intended for that person, and the person views this advertisement. A person entering Area-25 means paying at the register. A person entering Area-1 means leaving the store.

[0022] Here, the shooting range of camera 10 includes the store interior areas corresponding to Area-1 to Area-30 of the layout data 141. The information processing device 100 may also generate video data that includes the store interior areas corresponding to Area-1 to Area-30 of the layout data 141 by stitching together video data captured by multiple cameras 10. For example, the video data captured by camera 10 is the video data 30 shown in Figure 2.

[0023] The process of the information processing device 100 in step S11 of Figure 2 will be explained. The LLM 152 of the information processing device 100 reads the history DB 142.

[0024] Figure 4 shows an example of the data structure of the history database. As shown in Figure 4, the history database 142 includes person ID, attribute / behavior (1), attribute / behavior (2), attribute / behavior (3), presented advertisement, and results.

[0025] Person ID is information that identifies a person. Attribute / Behavior (1) indicates whether the person is male or female. If Attribute / Behavior (1) is "YES," it indicates that the person is male; if it is "NO," it indicates that the person is female.

[0026] Attribute / Behavior (2) indicates whether or not a person stopped moving for 5 seconds or more in any of Area-1 to Area-30. If Attribute / Behavior (2) is "YES", it indicates that "a person stopped moving for 5 seconds or more in any of Area-1 to Area-30", and if it is "NO", it indicates that "a person did not stop moving for 5 seconds or more in any of Area-1 to Area-30".

[0027] Attribute / Behavior (3) indicates whether or not a person stopped in Area-25 for 10 seconds or more. If Attribute / Behavior (3) is "YES", it indicates that "a person stopped in Area-25 for 10 seconds or more", and if it is "NO", it indicates that "a person did not stop in Area-25 for 10 seconds or more". For example, if a person stopped in Area-25 for 10 seconds or more, it means that the person stopped by register 16.

[0028] The presented advertisement indicates the pattern of the advertisement shown to the person. For example, there are three patterns of advertisements: Pattern A, Pattern B, and Pattern C. Specific examples of Patterns A, B, and C will be given later.

[0029] The result indicates whether the pre-specified result matches the analysis result output from the video analysis unit 151. For example, the specified result is the result expected from displaying the advertisement. If the result is "YES," it indicates that "the specified result and the analysis result match." If the result is "NO," it indicates that "the specified result and the analysis result do not match." If the result is "YES," it means that the desired effect was obtained by the advertisement in question. The specified result is the specified result 41 described later. The analysis result is the analysis result 44 described later.

[0030] The processing of the information processing device 100 in step S12 of Figure 2 will be explained. The LLM 152 generates list data 40 based on the read history DB 142 and outputs the list data 40 to the video analysis unit 151. For example, the list data 40 is a list that specifies the attributes and actions of the person to be analyzed.

[0031] Figure 5 is a diagram to supplement the explanation of the LLM processing in step S12. For example, the information processing device 100 generates prompt 142-1 based on the history DB 142. The contents of the history DB 142 are set in prompt 142-1. In addition, prompt 142-1 instructs the LLM 152 to add the attributes and behaviors of individuals that are deemed desirable to add in order to more appropriately judge the effectiveness of shopping, and to request analysis from the video analysis unit 151.

[0032] The information processing device 100 inputs prompt 142-1 to the LLM 152, which then generates list data 40. The list data 40 shown in Figure 5 includes attribute / action (1), attribute / action (2), attribute / action (3), and attribute / action (4). Attribute / action (4) in Figure 5 is an attribute / action newly added by the LLM 152.

[0033] For example, attribute / behavior (4) indicates whether or not a person picked up a cute product in any of the areas from Area-1 to Area-30. As described above, the LLM 152 generates list data 40 specifying the attribute types and behavior types to be analyzed based on the history DB 142 and outputs it to the video analysis unit 151. The video analysis unit 151 uses predetermined video analysis techniques based on the list data 40 to perform a process to acquire attribute information corresponding to the attribute types and behavior types corresponding to the behavior types specified by the list data 40.

[0034] The processing of the information processing device 100 in step S13 of Figure 2 will now be explained. The information processing device 100 inputs the specified result 41 to the video analysis unit 151. The specified result 41 is a result that has been specified in advance by an administrator or the like. For example, the specified result 41 is "The person who reached the exit has stopped by the cash register before that."

[0035] The processing of the information processing device 100 in step S14 of Figure 2 will be explained. The video analysis unit 151 acquires and analyzes the video data 30. For example, the video analysis unit 151 uses a predetermined video analysis technique to identify the person ID, track the person, and identify the person's attributes and actions with respect to the person in the video data.

[0036] The processing of the information processing device 100 in step 15 of Figure 2 will be explained. The video analysis unit 151 generates a person information list 42 based on the analysis results and outputs it to the LLM 152.

[0037] Figure 6 shows an example of the data structure of a person information list. As shown in Figure 6, the person information list 42 includes a person ID and attributes / behaviors (1) to (4). The explanation of the person ID and the explanation of "YES" and "NO" for attributes / behaviors (1) to (3) are the same as the explanation above. If attribute / behavior (4) is "YES", it indicates that "the person picked up a cute product in any of the areas from Area-1 to Area-30", and if it is "NO", it indicates that "the person did not pick up a cute product in any of the areas from Area-1 to Area-30".

[0038] In the example shown in Figure 6, for person ID "001214", the person information list 42 is set to "NO" for attribute / behavior (1), "YES" for attribute / behavior (2), "NO" for attribute / behavior (3), and "YES" for attribute / behavior (4).

[0039] The process of the information processing device 100 in step 16 of Figure 2 will be explained. The LLM 152 obtains the list of presented advertisements 143.

[0040] Figure 7 shows an example of the data structure of a list of suggested advertisements. As shown in Figure 7, this list of suggested advertisements 143 associates an advertisement ID with its content. The advertisement ID is a pattern that identifies the content of the advertisement. The content is the content of the advertisement.

[0041] For example, the content of ad ID "Pattern A" is "a description of children's clothing in a bright tone." The content of ad ID "Pattern B" is "a stylish description of matching outfits for adult men and children." The content of ad ID "Pattern C" is "a display of the image of a calm, mature woman."

[0042] The processing of the information processing device 100 in step 17 of Figure 2 will be explained. The LLM 152 generates a suggested advertisement 43 based on the history DB 142, the person information list 42, and the suggested advertisement list 143. The LLM 152 outputs the suggested advertisement 43 to the advertisement display unit 153. For example, one of the advertisement IDs explained in Figure 7 is set for the suggested advertisement 43. The advertisement display unit 153 displays the advertisement on the display device 15 based on the suggested advertisement 43.

[0043] Figure 8 is a diagram to supplement the explanation of the LLM processing in step S17. For example, the information processing device 100 generates prompt 142-2 based on the history DB 142, the person information list 42, and the presented advertisement list 143. The information processing device 100 inputs prompt 142-2 to the LLM 152, which then generates the presented advertisement 43.

[0044] The processing of the information processing apparatus 100 regarding step S18 in FIG. 2 will be described. The LLM 152 registers the person ID, attributes / actions, and presented advertisement 43 set in the person information list 42 in the history DB 142. For example, the LLM 152 adds a record corresponding to the person ID "001214", and sets attribute / action (1) to "NO", attribute / action (2) to "YES", attribute / action (3) to "NO", and attribute / action (4) to "YES". The result is left blank for the time being.

[0045] The processing of the information processing apparatus 100 regarding step S19 in FIG. 2 will be described. The video analysis unit 151 analyzes video data after the video data 30 acquired in step S14 to generate an analysis result 44, and outputs the analysis result 44 to the LLM 152. For example, when there is a person at the exit, the video analysis unit 151 sets the person ID of such a person and the values (YES or NO) of attributes / actions (1) to (4) in the analysis result 44.

[0046] The processing of the information processing apparatus 100 regarding step S20 in FIG. 2 will be described. The LLM 152 acquires the analysis result 44, and based on the designated result 41 and the analysis result 44, verifies whether a predetermined effect can be obtained. More specifically, the LLM 152 determines whether the contents of the designated result 41 and the analysis result 44 match. For example, when the value of attribute / action (3) in the analysis result 44 is set to "YES", the LLM 152 determines that the contents of the designated result 41 and the analysis result 44 match. When the value of attribute / action (3) in the analysis result 44 is set to "NO", the LLM 152 determines that the contents of the designated result 41 and the analysis result 44 do not match.

[0047] When the contents of the designated result 41 and the analysis result 44 match, the LLM 152 sets the result of the record registered in the history DB 142 in step S18 to "YES". On the other hand, when the contents of the designated result 41 and the analysis result 44 do not match, the LLM 152 sets the result of the record registered in the history DB 142 in step S18 to "NO".

[0048] FIG. 9 is a diagram showing an example of the data structure of the updated history DB. Comparing the history DB 142 described in FIG. 4 with the history DB 142 shown in FIG. 9, a record of person ID "001214" has been newly added. In the example shown in FIG. 9, since the result of person ID "001214" is "YES", it means that a desirable effect has been obtained by the presented advertisement "Pattern A" displayed for the person with person ID "001214".

[0049] By repeatedly executing the above processing, the information processing apparatus 100 can add information to the history DB 142 and display an optimal advertisement for each person.

[0050] Next, a supplementary explanation will be given for the video analysis unit 151 shown in FIG. 2. FIGS. 10, 11, 12, 13, 14, and 15 are diagrams for supplementary explanation of the video analysis unit.

[0051] Regarding FIG. 10, the video analysis unit 151 generates a detection pattern 50a regarding the layout based on the layout data 141. In the detection pattern 50a, "Subject", "Object", and "Relationship" are defined. For example, in the detection pattern 50a, it shows the relationship that the person (Person) corresponding to "Subject" enters the Area-18 corresponding to "Object". Area-18 is the area in front of the display device 15.

[0052] Regarding FIG. 11, the video analysis unit 151 generates detection patterns 50b, 50c, and 50d based on the list data 40 and the specified result 41.

[0053] In the detection patterns 50b and 50c, "Subject" and "Attribute" are defined. For example, in the detection pattern 50b, it is defined that the "Attribute" of the person (Person) corresponding to "Subject" is "being man". In the detection pattern 50c, it is defined that the "Attribute" of the person (Person) corresponding to "Subject" is "being stopping".

[0054] In detection pattern 50d, "Subject," "Object," and "Relationship" are defined. For example, in detection pattern 50d, the relationship is that the person corresponding to "Subject" has a "Cute item" corresponding to "Object."

[0055] Figure 12 will be explained. The video analysis unit 151 generates matching patterns 51a and 51b based on the list data 40 and the specified result 41. In matching patterns 51a and 51b, "Detection target" and "Pattern" are defined. For example, detection patterns 50a to 50d define the conditions for the detection target. Matching patterns 51a and 51b define the conditions for the matching target using the detection target definitions of detection patterns 50a to 50d.

[0056] The video analysis unit 151 generates a question 52a to obtain a person information list 42 based on the matching patterns 51a and 51b. The video analysis unit 151 also generates a question 52b to obtain an analysis result 44 based on the matching patterns 51a and 51b. The question 52a and 52b are text data.

[0057] Figure 13 will be explained. The video analysis unit 151 has an ASG generation unit 151a. The ASG generation unit 151a generates an action scene graph 60 based on the video data 30 and detection patterns 50a to 50d. The action scene graph 60 is an ASG (Action Scene Graph), Video Scene Graph, Spatio-temporal scene graph, etc. In the following explanation, detection patterns 50a to 50d will be collectively referred to as detection pattern 50.

[0058] For example, the ASG generation unit 151a performs object detection using the detection pattern 50 on time-series frames of video data 30 and tracks the detected objects. The ASG generation unit 151a generates video clips by combining the detection results and tracking results into predetermined number of frames. The ASG generation unit 151a inputs the video clips and prompts for relationship and attribute detection generated from the detection patterns into a visual detection model such as a VLM (Vision Language Model) to identify the attributes of the detected objects included in the video clips, the relationships between the detected objects, and the time when the attributes and relationships occurred.

[0059] The ASG generation unit 151a generates an action scene graph 60 based on a video clip, attribute information of the detection target identified from the video clip, interaction information between detection targets, and time. The action scene graph 60 maintains the relationship between Subject, object, and relationship, or the relationship between Subject, object, and attribute, at the event (attribute, relationship) level.

[0060] Figure 14 will now be explained. Figure 14 shows an example of the data structure of the action scene graph 60. The action scene graph 60 has time nodes n2-1, n2-2, n2-3, n2-4, n2-5, and n2-6. The action scene graph 60 has event nodes n3-1, n3-2, n3-3, n3-4, n3-5, and n3-6. The action scene graph 60 has concrete object nodes n4-1, n4-2, n4-3, n4-4, and n4-5.

[0061] The time nodes n2-1 to n2-6 are nodes that indicate time, and correspond to times T1, T2, T3, T4, T5, and T6, respectively. For example, times T1, T2, T3, T4, T5, and T6 are associated with the time (frame number) of each frame contained in the video clip.

[0062] Event nodes n3-1 to n3-6 are nodes corresponding to attributes and relationships. Specific object nodes n4-1 to n4-5 are nodes corresponding to detection targets. By using the behavior scene graph 60, it becomes possible to grasp various information about the video data 30. For example, event node n3-1, which is connected to time nodes n2-1 and n2-6, is connected to specific object node n4-2. This indicates that the detection target of the corresponding attribute exists in the video data 30 during times T1 to T6.

[0063] The ASG generation unit 151a of the video analysis unit 151 updates the action scene graph 60 by acquiring video data or by repeatedly executing the above process at predetermined intervals.

[0064] Figure 15 will be explained. The video analysis unit 151 has a graph analysis unit 151b. The video analysis unit 151 inputs a question 52a related to the video data 30, which was generated in Figure 12, into the graph analysis unit 151b. When the graph analysis unit 151b receives the input of the question 52a, it uses LLM to analyze the action scene graph 60 and generates a person information list 42.

[0065] The video analysis unit 151 inputs the question 52b related to the video data 30, which was generated in Figure 12, into the graph analysis unit 151b. Upon receiving the input of the question 52b, the graph analysis unit 151b uses LLM to analyze the behavior scene graph 60 and generates the analysis result 44.

[0066] The above provides supplementary information regarding the video analysis unit 151. The processing of the video analysis unit 151 is not limited to the above; it may also use other predetermined video analysis techniques to generate a person information list 42 and analysis results 44 from the video data 30. For example, the video analysis unit 151 may generate a person information list 42 and analysis results 44 from the video data 30 by performing in-context learning or the like. Furthermore, the video analysis unit 151 may use YOLO (You Only Look Once) or NN (Neural Network) to generate a person information list 42 and analysis results 44 from the video data 30.

[0067] Next, an example of the configuration of the information processing device 100 according to this embodiment will be described. Figure 16 is a functional block diagram showing the configuration of the information processing device according to this embodiment. As shown in Figure 16, the information processing device 100 includes a communication unit 110, an input unit 120, a display unit 130, a storage unit 140, and a control unit 150.

[0068] The communication unit 110 performs data communication with the camera 10 via the network 3. For example, the communication unit 110 receives video data from the camera 10.

[0069] The communication unit 110 performs data communication with the display device 15 via the network 3. For example, the communication unit 110 transmits advertising data generated by the control unit 150 to the display device 15.

[0070] The input unit 120 is an input device that inputs various types of information to the control unit 150 of the information processing device 100. For example, an administrator may operate the input unit 120 to input layout data 141, specified results 41, etc.

[0071] The display unit 130 is a display device that displays information output from the control unit 150.

[0072] The storage unit 140 includes layout data 141, history DB 142, and a list of displayed advertisements 143. The storage unit 140 is a memory, etc.

[0073] Layout data 141 is data relating to the store layout, and includes areas such as the entrance, exit, cash register 16, display device 15, and aisles. The explanation of layout data 141 corresponds to the explanation in Figure 3.

[0074] History DB 142 stores information about individual IDs, attributes and behaviors, displayed advertisements, and results. The explanation of History DB 142 corresponds to the explanation in Figure 4.

[0075] The ad list 143 stores ad IDs and their corresponding content. The explanation of the ad list 143 corresponds to the explanation in Figure 7.

[0076] We will now move on to the explanation of the control unit 150. The control unit 150 includes an image analysis unit 151, an LLM 152, and an advertisement display unit 153. The control unit 150 consists of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc.

[0077] The video analysis unit 151 acquires video data including the person to be detected from the camera 10 and analyzes the video data using a predetermined video analysis technique. The video analysis unit 151 exchanges data with the LLM 152 and generates a person information list 42 and analysis results 44, as explained in Figures 2 and 10 to 15.

[0078] As explained in Figure 2, the LLM 152 verifies whether the advertisement generated based on the prompt set by the person information list 42 will have a predetermined effect, and registers the information that associates the person information list 42 with the verification results in the history DB 142. As a result, the history DB 142 stores information on what kind of verification results can be obtained by presenting what kind of advertisement to what kind of person corresponding to what kind of attributes and behaviors.

[0079] As explained in Figure 5, the LLM 152 generates list data 40 based on prompt 142-1, which sets the history DB 142, and outputs it to the video analysis unit 151. For example, the LLM 152 sets in the list data 40 the attributes and behaviors of individuals that it deems desirable to add in order to more appropriately determine the effectiveness of shopping, and requests the video analysis unit 151 to analyze them. As a result, the video analysis unit 151 can obtain values ​​corresponding to the optimal attributes and behaviors for more appropriately determining the effectiveness of shopping as a person information list 42. By comparing the specified result 41 with the person information list 42, the LLM 152 can accurately verify whether the predetermined effect can be obtained.

[0080] As explained in Figure 8, the LLM 152 generates the suggested advertisement 43 using prompts 142-2 based on the history DB 142, the person information list 42, and the suggested advertisement list 143. The history DB 142 stores information on what kind of advertisements to present to people with what kind of attributes and behaviors, and what kind of verification results can be obtained, so it is possible to generate advertisements that are effective for the target person.

[0081] The LLM 152 may also communicate with an LLM server connected via network 3 to implement its functions. Alternatively, the LLM 152 may be an agent that utilizes the LLM 152. For example, the agent generates prompts 142-1 and 142-2 to be input to the LLM 152 as described in Figures 5 and 10, and uses the information output from the LLM 152 to exchange data with the video analysis unit 151 and the advertisement display unit 153.

[0082] The advertising display unit 153 displays an advertisement on the display device 15 based on the displayed advertisement 48 generated by the LLM 152. The advertising display unit 153 may also store information that associates an advertisement ID with advertisement data in advance and output the advertisement data for the advertisement ID corresponding to the displayed advertisement 48 to the display device 15.

[0083] Next, an example of the processing procedure of the information processing device 100 according to this embodiment will be described. Figure 17 is a flowchart showing the processing procedure of the information processing device according to this embodiment. As shown in Figure 17, the video analysis unit 151 of the information processing device 100 acquires layout data 141 (step S101). The LLM 152 of the information processing device 100 generates list data 40 based on the history DB 142 (step S102).

[0084] The video analysis unit 151 obtains the specified result 41 (step S103). The video analysis unit 151 obtains video data and generates a person information list 42 by analyzing the video data (step S104). The LLM 152 generates a suggested advertisement 43 based on the history DB 142, the person information list 42, and the suggested advertisement list 143 (step S105).

[0085] The advertisement display unit 153 of the information processing device 100 displays the advertisement on the display device 15 (step S106). The LLM 152 registers the person ID, attributes / behavior, and displayed advertisement 48 that were set in the person information list 42 into the history DB 142 (step S107).

[0086] The video analysis unit 151 analyzes the video data after the advertisement is displayed on the display device 15 and generates an analysis result 44 (step S108). The LLM 152 performs verification to determine whether a predetermined effect can be obtained based on the specified result 41 and the analysis result 44 (step S109).

[0087] LLM152 updates the results of history DB142 based on the verification results (step S110). If the information processing device 100 wishes to continue processing (step S111, Yes), it proceeds to step S104. On the other hand, if the information processing device 100 does not wish to continue processing (step S111, No), it terminates the process.

[0088] Next, the effects of the information processing device 100 according to this embodiment will be described. The information processing device 100 acquires a list of person information 42, which is the result of the video analysis by the video analysis unit 151, and prompts the LLM 152 to generate an advertisement based on the prompts from the list of person information 42, and verifies whether a predetermined effect can be obtained. Furthermore, the information processing device 100 changes the prompts using the relationship between the advertisement output as a response and the person information, based on the verification results. This makes it possible to display the most suitable advertisement for each person.

[0089] The information processing device 100 verifies whether a predetermined effect can be obtained for an advertisement generated based on a prompt set with a person information list 42, and registers information linking the person information list 42 and the verification results in the history database 142. This allows the history database 142 to store information on what kind of verification results can be obtained by presenting what kind of advertisement to what kind of person corresponding to what kind of attributes and behaviors.

[0090] The information processing device 100 generates list data 40 and outputs it to the video analysis unit 151 by inputting a prompt 142-1 with the history DB 142 set to the LLM 152. For example, the LLM 152 sets the attributes and behaviors of individuals that are deemed desirable to add in order to better judge the effectiveness of shopping, and requests the video analysis unit 151 to analyze them, into the list data 40. As a result, the video analysis unit 151 can obtain values ​​corresponding to the optimal attributes and behaviors for better judging the effectiveness of shopping as a person information list 42.

[0091] The information processing device 100 inputs prompt 142-2, based on the history database 142, the person information list 42, and the presented advertisement list 143, into the LLM 152 to generate the presented advertisement 43. The history database 142 stores information on what kind of advertisements should be presented to people with what kind of attributes and behaviors, and what kind of verification results can be obtained, so it is possible to generate advertisements that are effective for the target person.

[0092] Next, an example of a computer hardware configuration that realizes the same functions as the information processing device 100 shown in the above embodiment will be described in order.

[0093] Figure 18 shows an example of a computer hardware configuration that realizes similar functions to the information processing device of the embodiment. As shown in Figure 18, the computer 200 has a CPU 201 that performs various calculations, an input device 202 that receives data input from the user, and a display 203. The computer 200 also has a communication device 204 and an interface device 205 that exchange data with the camera 10, external devices, etc. via a wired or wireless network. The computer 200 also has a RAM 206 for temporarily storing various information and a hard disk drive 207. Each of the devices 201 to 207 is connected to a bus 208.

[0094] The hard disk drive 207 contains a video analysis program 207a, an LLM program 207b, and an advertising history program 207c. The CPU 201 reads each of the programs 207a to 207c and loads them into the RAM 206.

[0095] The video analysis program 207a functions as the video analysis process 206a. The LLM program 207b functions as the LLM process 206b. The advertisement presentation program 207c functions as the advertisement presentation process 206c.

[0096] The processing of the video analysis process 206a corresponds to the processing of the video analysis unit 151. The processing of the LLM process 206b corresponds to the processing of the LLM 152. The processing of the advertisement presentation process 206c corresponds to the processing of the advertisement presentation unit 153.

[0097] Furthermore, it is not necessary to store each program 207a to 207c on the hard disk drive 207 from the beginning. For example, each program may be stored on a "portable physical medium" such as a flexible disk (FD), CD-ROM, DVD, magneto-optical disk, or IC card inserted into the computer 200. Then, the computer 200 may read and execute each program 207a to 207c.

[0098] 100 Information Processing Unit 110 Communication Unit 120 Input Unit 130 Display Unit 140 Storage Unit 141 Layout Data 142 History DB 143 Advertisement List 150 Control Unit 151 Video Analysis Unit 152 LLM 153 Advertisement Display Unit

Claims

1. An information processing program characterized by causing a computer to execute the following processes:

1. Acquire video footage containing a person to be detected; acquire person information, which is associated with at least one of the attribute information and behavioral information of the person contained in the video, output by a predetermined video analysis technique; input a prompt based on the person information to an agent that generates advertisements for the person contained in the video according to the input information; verify whether a predetermined effect can be obtained based on the person information for the advertisement output as a response to the prompt; and change the prompt using the results of the verification and the relationship between the advertisement output as a response and the person information.

2. The information processing program according to claim 1, characterized in that each time the verification is performed, the computer further causes the computer to perform a process of registering the relationship between the result of the verification, the advertisement output as the answer, and the person information in a history database.

3. The information processing program according to claim 2, characterized in that it specifies attribute types and behavior types to be analyzed based on the history database, and further causes the computer to perform a process to acquire attribute information corresponding to the attribute types and behavior information corresponding to the behavior types of the person contained in the video, based on the predetermined video analysis technology.

4. The information processing program according to claim 2, wherein the agent is a large-scale language model, and the information processing program further causes a computer to execute the process of generating the advertisement by inputting the history database, the person information, the advertisement candidates, and predetermined prompts into the large-scale language model.

5. An information processing method characterized in that a computer performs the following steps: acquire video footage containing a person to be detected; acquire person information, which is associated with at least one of the attribute information and behavioral information of the person contained in the video, output by a predetermined video analysis technique; input a prompt based on the person information to an agent that generates advertisements for the person contained in the video according to the input information; verify whether a predetermined effect can be obtained based on the person information for the advertisement output as a response to the prompt; and change the prompt using the results of the verification and the relationship between the advertisement output as a response and the person information.

6. The information processing method according to claim 5, characterized in that each time the verification is performed, the computer further causes the computer to perform a process of registering the relationship between the result of the verification, the advertisement output as the answer, and the person information in a history database.

7. The information processing method according to 6, characterized in that the computer further performs a process of specifying attribute types and behavior types to be analyzed based on the history database, and acquiring attribute information corresponding to the attribute types and behavior information corresponding to the behavior types of the person contained in the video based on the predetermined video analysis technology.

8. The information processing method according to 6, wherein the agent is a large-scale language model, and the computer further performs the process of generating the advertisement by inputting the history database, the person information, and the advertisement candidates and predetermined prompts into the large-scale language model.

9. An information processing device having a control unit that performs the following processes: acquires video footage containing a person to be detected; acquires person information, which is associated with at least one of the attribute information and behavioral information of the person contained in the video, output by a predetermined video analysis technique; inputs a prompt based on the person information to an agent that generates advertisements for the person contained in the video according to the input information; verifies whether a predetermined effect can be obtained based on the person information for the advertisement output as a response to the prompt; and changes the prompt using the result of the verification and the relationship between the advertisement output as a response and the person information.

10. The information processing apparatus according to claim 9, characterized in that each time the verification is performed, the control unit further performs a process of registering the relationship between the result of the verification and the advertisement output as the answer and the person information in a history database.

11. The information processing apparatus according to claim 10, characterized in that the control unit specifies attribute types and behavior types to be analyzed based on the history database, and further performs a process to acquire attribute information corresponding to the attribute types and behavior information corresponding to the behavior types of the person included in the video based on the predetermined video analysis technology.

12. The information processing apparatus according to claim 10, wherein the agent is a large-scale language model, and the control unit further performs the process of generating the advertisement by inputting the history database, the person information, the advertisement candidate, and a predetermined prompt set to the large-scale language model.

Images