Information processing apparatus and program

Abstract

To provide an information processor and a program for improving the accuracy of information output from a generation AI.SOLUTION: In a store computer which is an information processor, a control part 100 includes a reception part, a generation part, a presentation part, a tracking part, and a recording part. The reception unit receives an input of request information indicating a request of a customer visiting a store. The generation unit generates an answer sentence indicating an answer to the request by using the generation AI on the basis of the request information. The communication control unit and the generation unit as a presentation unit present response information based on the answer sentence to the customer. The tracking unit tracks behavior of the customer after the response information is presented. The recording unit records behavior information indicating the behavior of the customer obtained by the tracking and the response information presented to the customer in association with each other.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device and a program. [Background technology]

[0002] In recent years, generative AI (Artificial Intelligence) that generates text, images, etc. has been attracting attention. The accuracy of generative AI is improving every day, and generative AI that generates text is now being used to promote products that match customer inquiries.

[0003] However, generative AI has the characteristic of outputting incorrect answers or being unable to properly answer questions that are not included in the training data. For example, in response to the input sentence "Tell me who the current Prime Minister is," it may answer the name of someone other than the current Prime Minister because it has not previously studied Prime Ministers.

[0004] Furthermore, in the case of text generation AI, in order to improve the accuracy of the generated sentences, it is important to understand whether the output text has led to changes in customer behavior. However, with conventional technology, there is almost no determination as to whether the generated text has provided useful information to users. Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the present invention is to provide an information processing device and a program that make it possible to improve the accuracy of information output from a generation AI. [Means for solving the problem]

[0006] An information processing device according to an embodiment includes a reception unit, a generation unit, a presentation unit, a tracking unit, and a recording unit. The reception unit receives input of request information representing the requests of a customer who visits a store. The generation unit generates an answer sentence representing a response to the request using a generation AI based on the request information. The presentation unit presents response information based on the answer sentence to the customer. The tracking unit tracks the customer's behavior after the response information is presented. The recording unit records behavior information representing the customer's behavior obtained by tracking in association with the response information presented to the customer. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a system diagram showing an example of the connection relationships of the devices in the system according to the embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of the SC according to the embodiment. [Figure 3] FIG. 3 is a block diagram showing an example of the functional configuration of the SC according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing an example of processing executed by the SC according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] An information processing device and a program according to an embodiment will be described in detail below with reference to Figures 1 to 4. In the embodiment described below, a store computer (SC) installed in a store such as a department store or supermarket will be described as an example of an information processing device, but the present invention is not limited to the embodiment.

[0009] 1 is a system diagram showing an example of the connection relationships of devices in an information processing system S according to an embodiment. In FIG. 1, the system includes an SC (Store Computer) 1, a POS (Point of Sales) terminal 2, multiple cameras 3, and a mobile terminal 5.

[0010] The SC1, the POS terminal 2, and the multiple cameras 3 are connected to one another via a communication line 6 such as a LAN (Local Area Network). The mobile terminal 5 is connected to the SC1 via the communication line 6 and an access point 4, which is a wireless communication repeater.

[0011] Note that the number of each device shown in Fig. 1 is an example, and the number of each device included in the system is not limited to the number shown in Fig. 1. For example, the system may include a plurality of access points 4 and mobile terminals 5.

[0012] The POS terminal 2 is a sales data processing device that executes sales registration processing for products purchased at a store. For example, the POS terminal 2 is a dedicated self-service POS terminal, a smartphone, a cart POS, etc. Note that the POS terminal 2 is not limited to a self-service device. For example, the POS terminal 2 may be a device used by a store clerk to execute sales registration processing.

[0013] The POS terminal 2 generates product sales registration information and transmits it to the SC 1 via the communication line 6 .

[0014] SC1 is a server device that collects and manages product sales registration information received from the POS terminal 2. SC1 may be configured as a single server device or as a plurality of server devices. SC1 may also be a cloud server. SC1 may also be provided on a network outside the store.

[0015] SC1 stores product information such as the prices and product names of products sold in the store.

[0016] Furthermore, SC1 accepts input of request text expressing a request from a customer via the mobile terminal 5, inputs the request text into the generative model 142 (see FIG. 2), and generates answer text expressing an answer to the request. SC1 generates response information based on the generated answer text and transmits it to the mobile terminal 5 that is the source of the request text input.

[0017] Furthermore, SC1 inputs the image (moving image) of the customer captured by the camera 3 into the behavior recognition model 143 (see FIG. 2), and generates a behavior text describing the behavior of the customer that is output from the behavior recognition model 143. For example, the behavior text is text data including information indicating the customer's travel route and the products purchased by the customer. SC1 stores the response information and the generated behavior text in association with each other.

[0018] The camera 3 captures images including customers inside the store. For example, a plurality of cameras 3 are installed at regular intervals along the aisles on the ceiling or the like near the aisles so that they can capture images of customers passing through the store.

[0019] As an example, the multiple installed cameras 3 capture images of the path that customers take from the time they enter the store until they leave. In the example of FIG. 1, n cameras 3 are installed along the aisles in the store, and these cameras 3 capture images of customers PA and PB. The images captured by the cameras 3 may include information indicating the time of capture and the capture position (for example, the installation position of the camera 3 that captured the image).

[0020] The mobile terminal 5 is carried by the customer and exchanges various information with the SC1. The mobile terminal 5 is, for example, a smartphone or a tablet terminal. For example, the mobile terminal 5 accepts input of a request text from the customer and transmits the accepted request text to the SC1. Also, for example, the mobile terminal 5 receives response information from the SC1. Also, for example, the mobile terminal 5 is equipped with a camera and transmits an image of the customer's face captured by the camera to the SC1 in a recognizable form.

[0021] Instead of the mobile terminal 5, a terminal device such as a PC (Personal Computer) capable of executing the above processes may be provided.

[0022] Next, the hardware configuration of the SC 1 will be described with reference to Fig. 2, which is a block diagram showing an example of the hardware configuration of the SC 1.

[0023] As shown in Figure 2, SC1 includes a CPU (Central Processing Unit) 11 that serves as the control body, a ROM (Read Only Memory) 12 that stores various programs, a RAM (Random Access Memory) 13 that expands various data, and a memory unit 14 that stores various programs.

[0024] The CPU 11, ROM 12, RAM 13, and memory unit 14 are connected to one another via a data bus 15. The CPU 11, ROM 12, and RAM 13 constitute a control unit 100. That is, the control unit 100 executes various processes by the CPU 11 operating in accordance with a control program 141 stored in the ROM 12 or memory unit 14 and loaded into the RAM 13. The various processes will be described later.

[0025] The RAM 13 develops various programs including the control program 141, and also temporarily stores images captured by the camera 3 until they are stored in the memory unit .

[0026] The memory unit 14 is a non-volatile memory such as an HDD (Hard Disc Drive), an SSD (Solid State Drive), or a flash memory that retains stored information even when the power is turned off, and stores programs including a control program 141. The memory unit 14 also has a generative model 142, a behavior recognition model 143, and a behavior information DB (Data Base) 144.

[0027] The generative model 142 is a generative AI that generates sentences, such as a large language model (LLM). The generative model 142 generates a response text corresponding to the customer's request in response to an input of a request text.

[0028] For example, the generative model 142 is an LLM constructed by a known deep learning technique, which generates a response text corresponding to a customer's request in response to input of text data including the request text.

[0029] The generative model 142 may be stored in a server device other than the SC1.

[0030] The behavior recognition model 143 is a trained model trained using input training data including images (video images) including customers and output training data including text data describing the behavior of customers.

[0031] As an example, the behavior recognition model 143 is a model based on a machine learning model such as a neural network whose parameters are determined by deep learning. As the model, for example, a convolutional neural network (CNN) can be used, but other networks may also be used.

[0032] In this embodiment, the behavior recognition model 143 is configured to output, through learning, characteristic text data describing the behavior of a customer in response to an input of a moving image including the customer captured by the camera 3.

[0033] The behavior recognition model 143 may be stored in a server device other than the SC1.

[0034] The behavior information DB 144 is a database that stores information about customer behavior. For example, the behavior information DB 144 stores, for each customer, a request text, response information, a face image, and a behavior text in association with each other. Note that the behavior information DB 144 may be stored in a server device other than the SC1.

[0035] An operation unit 17 and a display unit 18 are also connected to the data bus 15 via a controller 16 .

[0036] The operation unit 17 receives various inputs from an operator (user) such as a store manager, etc. For example, the operation unit 17 includes a numeric keypad for entering numbers, various function keys, and the like.

[0037] The display unit 18 displays various types of information. For example, the display unit 18 displays an analysis report, which will be described later. The display unit 18 may display images of the inside of the store acquired from the cameras 3. The display unit 18 may also display images captured by a specific camera 3. The display unit 18 may also divide the screen and display images captured by multiple cameras 3 on the same screen at the same time.

[0038] The data bus 15 is also connected to a communication I / F 19 such as a LAN I / F (Interface). The communication I / F 19 is connected to a communication line 6.

[0039] The communication I / F 19 transmits and receives various types of information. For example, the communication I / F 19 receives images captured by the camera 3 in real time.

[0040] Next, the functional configuration of SC1 will be described. Fig. 3 is a functional block diagram showing an example of the functional configuration of SC1. By following various programs including a control program 141 stored in ROM 12 and memory unit 14, control unit 100 functions as a communication control unit 101, a reception unit 102, a generation unit 103, an acquisition unit 104, a tracking unit 105, an analysis unit 106, and a feedback unit 107.

[0041] The communication control unit 101 controls communication between the POS terminal 2 and the mobile terminal 5. For example, the communication control unit 101 establishes wireless communication or wired communication with the POS terminal 2, and transmits and receives various information to and from the POS terminal 2. Furthermore, for example, the communication control unit 101 establishes wireless communication with the mobile terminal 5, and transmits and receives various information to and from the mobile terminal 5.

[0042] The reception unit 102 receives input of request text, which is text data expressing a request from a customer. For example, the reception unit 102 cooperates with the communication control unit 101 to receive from the mobile terminal 5 the request text entered by the customer. The request text sent from the mobile terminal 5 includes terminal identification information that identifies the mobile terminal 5 that sent it.

[0043] The generation unit 103 generates response information representing a response to a customer's request based on the request text. For example, the generation unit 103 generates input text to be input to the generation model 142 based on the request text accepted by the acceptance unit 102. The process of generating text to be input to the generation AI in this way is also called preprocessing.

[0044] The input text is instruction information (prompt) that instructs the AI ​​on the content of the text to be generated. Hereinafter, the content instructed by the instruction information will also be referred to as the instruction content.

[0045] For example, as a preprocessing step, the generation unit 103 instructs the generation of text including the name of a product that is recommended for purchase by the customer, a description of the product, and information representing the behavior recommended for the customer in the store, and generates input text that describes instructions that the product name is the name of a product sold in the store, that the product that is recommended for purchase by the customer is derived based on the customer's request, and that the behavior recommended for the customer is derived based on the location where the product that is recommended for purchase by the customer is displayed.

[0046] Here, templates for generating instruction information are stored in advance in the memory unit 14 or the like. Note that a plurality of templates may be stored in the memory unit 14 or the like. In this case, the generation unit 103 may use a plurality of templates depending on the situation. For example, the generation unit 103 may switch templates depending on keywords included in the request text (e.g., words that can identify the product genre (food, clothing, etc.)).

[0047] In addition to the above, the generation unit 103 may perform preprocessing such as acquiring information representing the customer's product purchase history and describing instructions to exclude information about products that have already been purchased from the generated text data.

[0048] The generation unit 103 acquires the answer text generated by the generation model 142 in response to the input of the input text. The generation unit 103 performs processing such as adding additional information to the acquired answer text or converting the output format of the answer text to generate response information. Such processing performed on the text generated by the generation AI is also referred to as post-processing.

[0049] For example, as post-processing, the generation unit 103 performs processing such as converting information contained in the response text that indicates the display location of the product that corresponds to the customer's request into information that indicates the route from the customer's current location to the display location. The customer's current location may be identified from information that indicates the position of the camera 3 recorded in an image (video) captured by the camera 3, or may be identified based on radio waves emitted by a transmitter that transmits radio waves and is installed in the store.

[0050] In addition to the above, the generating unit 103 may perform post-processing such as attaching an image of the product or converting text data into audio data that sounds natural to the human ear.

[0051] The generation unit 103 cooperates with the communication control unit 101 to transmit the answer text that has undergone the above post-processing as response information to the mobile terminal 5 that is the sender (input) of the request text. In this case, the generation unit 103 and the communication control unit 101 are an example of a presentation unit.

[0052] Here, the generation unit 103 may acquire an answer text generated using the generation model 142 built in a cloud environment, but the input text may contain personal information of the customer, etc. Therefore, from the viewpoint of preventing the leakage of personal information, it is preferable to use the locally built generation model 142, as in this embodiment, to generate the answer text.

[0053] The acquisition unit 104 acquires a face image of the customer who inputs the request text.

[0054] For example, the acquisition unit 104, at the same time as transmitting the response information, transmits a facial image transmission request to the mobile terminal 5, requesting transmission of a facial image of a customer whose behavior is to be tracked (hereinafter also referred to as a tracking target customer). The mobile terminal 5 that has received the facial image transmission request starts an application for capturing images and captures an image of the customer holding the mobile terminal 5. At this time, the mobile terminal 5 uses a known image recognition technology to capture images of the customer until an image is captured that is recognized as containing the customer's face.

[0055] The mobile terminal 5 may display a message on a display or the like, and may prompt the customer to take an image of his or her own face with a camera mounted on the mobile terminal 5.

[0056] In response to the facial image transmission request, the mobile terminal 5 transmits to SC1 the facial image of the customer associated with the response information received simultaneously with the facial image transmission request. The acquisition unit 104 of SC1 cooperates with the communication control unit 101 to acquire the facial image of the customer associated with the response information transmitted from the mobile terminal 5 via the communication I / F 19 and the communication line 6. The acquisition unit 104 also sends the acquired facial image of the customer to the tracking unit 105.

[0057] The tracking unit 105 tracks the behavior of the customer after the response information is presented. For example, the tracking unit 105 identifies a tracking target customer based on the facial image of the customer acquired by the acquisition unit 104, captures an image of the identified tracking target customer with the camera 3, and tracks the behavior of the identified tracking target customer from the video image captured by the camera 3.

[0058] Specifically, the tracking unit 105 acquires moving images captured by the camera 3 in real time, and extracts images that are recognized as including a person using a known image recognition technology as images that include the customer. Note that the tracking unit 105 may continuously acquire still images captured by the camera 3 at predetermined time intervals.

[0059] 1, the tracking unit 105 extracts an image including customer PA captured by the camera 3 and an image including customer PB from the acquired video. Here, when the acquisition unit 104 acquires a facial image of customer PA as the facial image of the tracking target customer, the tracking unit 105 identifies customer PA as the tracking target customer.

[0060] After identifying the tracking target, the tracking unit 105 continuously executes the process of extracting images of the tracking target customer from the video images acquired in real time until it is recognized by a known image recognition technology that the tracking target customer has left the store. In the above example, the tracking unit 105 continuously executes the process of extracting images of the tracking target customer from the time customer PA is identified as the tracking target customer until customer PA leaves the store.

[0061] The tracking unit 105 also inputs a moving image of the tracking target customer, which is composed of the extracted multiple images, to the behavior recognition model 143. Note that the tracking unit 105 may input multiple still images acquired at predetermined time intervals to the behavior recognition model 143 in chronological order.

[0062] The tracking unit 105 acquires the behavior text output by the behavior recognition model 143. As an example, the behavior text is text data such as "The customer went to the vegetable section, picked up a Chinese cabbage, and put it in the shopping cart."

[0063] The behavioral text may contain less information than the above, such as "The customer bought a cabbage," or it may contain more information than the above, such as "The customer went to the beverage section, walked through the aisle in the center of the store to the vegetable section, picked up a cabbage, but put it back on the shelf."

[0064] The tracking unit 105 associates the customer's face image, the response information associated with the face image, and the acquired behavior text, and stores them in the behavior information DB 144.

[0065] When storing the facial image of the customer, the response information, and the behavior text in association with each other, the tracking unit 105 may search the behavior information DB 144 using the facial image of the customer to be tracked as a key. If it is found that information corresponding to the facial image is already stored, the tracking unit 105 may store the facial image of the customer, the response information, and the behavior text in association with each other in the behavior information DB 144 by adding the information to the currently stored information.

[0066] Here, the tracking unit 105 may acquire behavior text generated using the behavior recognition model 143 built in a cloud environment, but there is a possibility that personal information of the customer may be identified from the video image input to the behavior recognition model 143. Therefore, from the viewpoint of preventing leakage of personal information, it is preferable to use the behavior recognition model 143 built locally, as in this embodiment, for generating behavior text.

[0067] The analysis unit 106 analyzes whether the response information presented to the customer led to a change in the customer's behavior based on the response information and the behavior text. For example, the analysis unit 106 compares the response information with the behavior text and calculates the percentage of the customer who followed the behavior suggested in the response information. The analysis unit 106 also calculates the percentage of the customer who achieved the goal suggested in the response information.

[0068] Let us consider a case where the response information suggests the actions of "going to the beverage section from near the entrance, going from the beverage section to the vegetable section, and going from the vegetable section to the fruit section" and the purposes of "buying beverage A, buying Chinese cabbage, and buying an apple." The acquired action text is assumed to be "going to the vegetable section from near the entrance, but not putting the Chinese cabbage in the shopping cart, going from the vegetable section to the fruit section, putting the apple in the shopping cart, going from the fruit section to the cash register, and paying."

[0069] In this case, the customer follows two of the three suggested actions in the response information, namely, heading to the vegetable section and heading to the fruit section, so the analysis unit 106 calculates the percentage of the customer who followed the actions suggested in the response information as 2 / 3 ≒ 66.6%. Also, the customer achieved one of the three goals suggested in the response information, which was to buy an apple, so the analysis unit 106 calculates the percentage of the customer who achieved their goal as 1 / 3 ≒ 33.3%.

[0070] In addition to the above, the analysis unit 106 may calculate other indicators required by the user, such as the percentage of cases where a product was picked up but not purchased.

[0071] The feedback unit 107 executes processing that leads to improvement of the response information based on the analysis result by the analysis unit 106. For example, the feedback unit 107 presents to the user an analysis report that includes the response information actually presented to the customer, the behavior text, the percentage of the customer who followed the behavior suggested in the response information, and the percentage of the customer who achieved the purpose suggested in the response information.

[0072] Methods for presenting the analysis report include displaying the analysis report on the display unit 18 of SC1, sending the analysis report to a terminal used by the user, or printing the analysis report on a printing device.

[0073] The user who is presented with the analysis report can understand trends such as which suggestions the customer follows and which suggestions they do not follow. Based on the trends that the user has understood, the user can take actions to improve the generated response information, such as adding instructions to add standard phrases to make the proposed content easier to understand as preprocessing, or adding related images to content that is difficult to understand when presented in text alone or performing reinforcement learning on the generative model 142 as postprocessing.

[0074] The feedback unit 107 may automatically improve pre-processing and post-processing, and may perform processes such as reinforcement learning of the generative model 142, based on the analysis results by the analysis unit 106. In this case, the feedback unit 107 is an example of a change processing unit. As an example, based on the accumulated analysis results, the feedback unit 107 may generate an input text by adding instructions that are unlikely to be included in the answer text for actions or purposes for which the probability that the customer will not follow the suggestions exceeds a threshold.

[0075] In addition, the feedback unit 107 may generate input text by adding instructions such that if a specific customer does not follow the suggestions even after the specific customer has been suggested the same action (or purpose) a predetermined number of times, the action (or purpose) will no longer be suggested to the specific customer.

[0076] This can prevent the customer from repeatedly making suggestions that they do not intend to follow.

[0077] Next, the processing executed by the SC1 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of the processing executed by the SC1.

[0078] First, the reception unit 102 receives a request input from a customer (step ST1). For example, the reception unit 102 cooperates with the communication control unit 101 to receive a request text input from the customer via the communication I / F 19, the communication line 6, and the mobile terminal 5. Here, the received request text includes terminal identification information that identifies the mobile terminal 5 from which the request text was input.

[0079] Next, the generation unit 103 generates response information (step ST2). For example, the generation unit 103 performs preprocessing on the request text received in step ST1 to generate an input text. The generation unit 103 inputs the generated input text to the generation model 142 and acquires a response text to the customer request generated by the generation model 142. The generation unit 103 performs postprocessing on the acquired response text to generate response information.

[0080] Next, the generation unit 103 presents the response information to the customer (step ST3). For example, the generation unit 103 cooperates with the communication control unit 101 to transmit the response information generated in step ST2 to the mobile terminal 5 that is the input source of the request text received in step ST1. Then, the mobile terminal 5 displays the response information on a display or the like, thereby presenting the response information to the user.

[0081] Next, the acquisition unit 104 acquires a face image of the customer whose behavior is to be tracked (step ST4).

[0082] For example, the acquisition unit 104 cooperates with the communication control unit 101 to transmit a facial image transmission request in association with the response information generated in step ST2 to the mobile terminal 5 that is the input source of the request text received in step ST1, simultaneously with the processing of step ST3.

[0083] The acquisition unit 104 cooperates with the communication control unit 101 to acquire the facial image of the customer transmitted from the mobile terminal 5 as a response to the facial image transmission request via the communication I / F 19 and the communication line 6. Here, the facial image is associated with the response information generated in step ST2.

[0084] Next, the tracking unit 105 identifies the customer to be tracked (step ST5). For example, the tracking unit 105 acquires video images of the inside of the store captured in real time by multiple cameras 3 installed in the store. The tracking unit 105 uses a known image recognition technology to extract images in which people appear. The tracking unit 105 identifies the customer to be tracked from the extracted images based on the facial image of the customer acquired in step ST4.

[0085] Next, the tracking unit 105 tracks the behavior of the tracking target customer (step ST6). For example, the tracking unit 105 continuously executes a process of acquiring video images captured in the store in real time and extracting images showing the tracking target customer identified in step ST5 using a known image recognition technology.

[0086] Next, the tracking unit 105 determines whether the tracking target customer has left the store (step ST7). For example, the tracking unit 105 determines that the tracking target customer has left the store when it recognizes that the tracking target customer has left the store using a known image recognition technology. If the tracking target customer has not left the store (step ST7: No), the process returns to step ST6.

[0087] On the other hand, if the tracking target customer leaves the store (step ST7: Yes), the tracking unit 105 records the tracking target customer's behavior within the store (step ST8).

[0088] For example, the tracking unit 105 inputs video data consisting of multiple images showing the tracking target customer extracted in step ST6 to the behavior recognition model 143. The tracking unit 105 acquires the behavior text output by the behavior recognition model 143. The tracking unit 105 stores the face image of the customer acquired in step ST4, the response information associated with the face image, and the acquired behavior text in the behavior information DB 144 in association with each other.

[0089] Next, the analysis unit 106 analyzes whether the response information presented to the customer has led to a change in the customer's behavior (step ST9). For example, the analysis unit 106 compares the response information with the behavior text, and calculates the percentage of customers who followed the behavior suggested in the response information and the percentage of customers who achieved the goal suggested in the response information.

[0090] Next, the feedback unit 107 outputs the analysis result (step ST10) and ends this process. For example, the feedback unit 107 causes the display unit 18 to display an analysis report including the response information actually presented to the customer, the behavior text, the percentage of the customer who followed the behavior suggested in the response information, and the percentage of the customer who achieved the purpose suggested in the response information.

[0091] Note that, although Figure 4 describes an example in which the feedback unit 107 presents an analysis report, the feedback unit 107 may also perform processing that leads to improvement of response information, such as preprocessing, postprocessing, and reinforcement learning of the generative model 142, based on the analysis results of step ST9.

[0092] As explained above, the SC (information processing device) 1 according to this embodiment accepts input of request information representing the requests of customers who visit a store, uses the generation model 142, which is a generation AI, to generate answer text based on the request information, generates response information to the customer's request based on the answer text, presents the response information to the customer, tracks the customer's behavior after the response information is presented, and records the behavior text representing the customer's behavior obtained by tracking in association with the response information.

[0093] As a result, the SC1 according to this embodiment records the response information presented to the customer and the actual customer behavior in a manner that facilitates comparison. For example, a user, such as a store manager, can compare the response information presented to the customer with the actual customer behavior to evaluate the appropriateness and legitimacy of the response text, such as whether the response is off the mark or whether it is close to the customer's request but incorrect. The user can also understand to what extent the response information presented to the customer led to a change in the customer's behavior. In this case, the user can also understand how to modify the response information to further change the customer's behavior. Understanding how to modify the response information also allows the user to understand how to improve the process of generating the response text using the generation model 142. Therefore, the SC1 according to this embodiment can improve the accuracy of the information output from the generation AI.

[0094] The above-described embodiment can be modified as needed by changing some of the configurations or functions of SC1. Therefore, below, several modifications of the above-described embodiment will be described as other embodiments. Below, differences from the above-described embodiment will be mainly described, and detailed descriptions of commonalities with the contents already described will be omitted. The modifications described below may be implemented individually or in appropriate combination.

[0095] (Variation) In the above-described embodiment, a behavioral text of a tracking target customer is generated and stored when it is recognized that the tracking target customer has left the store. In this modified example, a behavioral text of a tracking target customer is generated and stored every predetermined time interval from video images of the most recent predetermined time period.

[0096] In this modification, after identifying the tracking target customer, the tracking unit 105 inputs, at predetermined time intervals (for example, one minute), video images for the past predetermined time period (for example, video images for the past three minutes) consisting of a plurality of images showing the tracking target customer extracted from video images captured by the camera 3, into the behavior recognition model 143. This allows the tracking unit 105 to store behavior texts representing the behavior of the tracking target customer in the behavior information DB 144 in approximately real time.

[0097] Here, similar to the above-described embodiment, the tracking unit 105 may acquire behavior text generated using a behavior recognition model 143 constructed in a cloud environment. However, using a behavior recognition model 143 constructed in a cloud environment requires network traffic for communicating with a cloud server and computing costs for the cloud server. Furthermore, when behavior text is generated at short time intervals, a lag may occur due to communication between the SC1 and the cloud server. In this regard, if a locally constructed behavior recognition model 143 is used, the above problem does not occur. For this reason, it is preferable that the tracking unit 105 uses a locally constructed behavior recognition model 143.

[0098] According to this modification, it is possible to analyze the behavior of a customer to be tracked without waiting for the customer to leave the store, and to perform processing such as improving pre-processing and post-processing based on the analysis results.

[0099] The program executed by SC1 in the above-described embodiment is provided as a file in an installable or executable format recorded on a non-transitory computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

[0100] The program executed by the SC1 of the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program executed by the SC1 of the embodiment may be provided or distributed via a network such as the Internet.

[0101] Furthermore, the program executed by the SC1 of the embodiment may be provided by being pre-installed in the ROM 12 or the like.

[0102] In addition, various functional units such as the communication control unit 101, the reception unit 102, the generation unit 103, the acquisition unit 104, the tracking unit 105, the analysis unit 106, and the feedback unit 107 shown in Figure 3 may be implemented by one or more processing circuits such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0103] Although the embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents set forth in the claims. [Explanation of symbols]

[0104] 1 SC 2. POS terminals 3 Camera 4. Access Points 5. Mobile devices 6. Communication lines 11 CPU 12 ROM 13 RAM 14 Memory section 100 control section 101 Communication control unit 102 Reception 103 Generation part 104 Acquisition Department 105 Tracking Department 106 Analysis Department 107 Feedback Section [Prior art documents] [Patent documents]

[0105] [Patent Document 1] International Publication No. 2021 / 070732

Claims

1. a reception unit that receives input of request information that indicates the requests of customers who visit the store; A generation unit that generates an answer sentence representing an answer to the request using a generation AI based on the request information; a presentation unit that presents response information based on the answer sentence to the customer; a tracking unit that tracks the behavior of the customer after the response information is presented; a recording unit that records behavior information indicating the behavior of the customer obtained by tracking and the response information presented to the customer in association with each other; An information processing device comprising:

2. the tracking unit tracks the behavior of the customer after the response information is presented based on an image of the customer captured by an imaging device that captures images inside the store. The information processing device according to claim 1 .

3. a preprocessing unit that generates an input text inquiring about a recommended behavior of the customer in the store based on the request information; The generation unit inputs an input text to the generation AI to obtain a response document indicating a recommended behavior of the customer in the store. The information processing device according to claim 1 .

4. a post-processing unit that performs post-processing based on the answer text, such as adding additional information to the answer text or converting an output format of the answer text; the presentation unit presents the answer sentence that has been post-processed by the post-processing unit to the customer as the response information. The information processing device according to claim 1 .

5. an analysis unit that compares the recorded behavioral information with the response information and analyzes whether the response information has led to a behavioral change of the customer; an output unit that outputs the analysis result of the analysis unit; Further comprising: The information processing device according to claim 1 .

6. a modification processing unit that modifies a method for generating the answer sentence or a method for presenting the response information to the customer based on the analysis result of the analysis unit; The information processing device according to claim 5 , further comprising:

7. The computer of the information processing device a reception unit that receives input of request information that indicates the requests of customers who visit the store; A generation unit that generates an answer sentence representing an answer to the request using a generation AI based on the request information; a presentation unit that presents response information based on the answer sentence to the customer; a tracking unit that tracks the behavior of the customer after the response information is presented; a recording unit that records behavior information indicating the behavior of the customer obtained by tracking and the response information presented to the customer in association with each other; A program that executes the following.

Images