Information processing apparatus and storage medium

Abstract

The invention discloses an information processing apparatus and a storage medium capable of providing question content reflecting a user's thought. An information processing apparatus according to an embodiment receives image information indicating an image associated with a problem object of a user, performs image analysis processing on the image information, and acquires image analysis information indicating an image analysis result. Acquiring, from question history information indicating each of a plurality of question histories in which question content is recorded, question history information indicating a question history in which question content associated with the image analysis result indicated by the image analysis information is recorded; the candidate question information generation unit generates candidate question information indicating a candidate question in which question content for the question object is described on the basis of the question history information acquired by the question history information acquisition unit, and provides the candidate question information to the user in a selectable state. Candidate question information selected by a user is input to a generative model having a function of inputting an answer statement to question content, the answer statement output by the generative model is acquired, and the answer statement is provided to the user.

Description

[0001] This application claims priority from Japanese Application No. JP 2024-063352, filed April 10, 2024, and the contents of the aforementioned application are incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to an information processing apparatus and a storage medium. BACKGROUND

[0003] In recent years, a technology of automatically creating an article using an AI model (also referred to as a generative AI) such as a large language model (LLM: Large Language Model) has been proposed. The large language model is an AI model that is pre-trained using a large corpus in the field of natural language processing, and is given a function so as to be able to generate and output a response sentence related to a question content in a form of an instruction, for example, when a text in which the question content is described is input together with instruction information (also referred to as a hint) that instructs the output form.

[0004] Further, it is considered to use a service using the above-described AI model as an automatic response service to a user who uses a store. For example, it is considered to provide a service of automatically responding to a question from a customer by using the above-described AI model in a store such as a supermarket.

[0005] Incidentally, it is known that the accuracy of a response sentence is improved in the above-described AI model if an accurate question content is input. Therefore, if the question content is not properly input, a user can not be able to obtain a desired response sentence. For example, in an environment such as the above-described store, since a user who is not familiar with the operation of a machine is also using, there is a concern that the user can not be able to input a question content that the user wants to the AI model, and therefore a technology that can assist the operation of the user is expected. SUMMARY

[0006] In view of the above-described problems, the present application aims to provide an information processing apparatus and a storage medium that can reflect a question content of a user's idea.

[0007] To solve the above problems, an information processing apparatus according to an embodiment includes: a reception unit that receives image information including an image associated with a question object of a user; an image analysis information acquisition unit that performs image analysis processing on the image information and acquires image analysis information indicating an image analysis result; a question history record information acquisition unit that acquires, from question history record information indicating each of a plurality of question history records in which question contents are recorded, question history record information indicating a question history record in which question contents associated with the image analysis result indicated by the image analysis information are recorded; a candidate question information generation unit that generates candidate question information indicating a candidate question in which question contents for the question object are recorded, based on the question history record information acquired by the question history record information acquisition unit; a first provision unit that provides the candidate question information to the user in a selectable state; an answer sentence acquisition unit that inputs the candidate question information selected by the user to a generation model having a function of outputting an answer sentence for question contents and acquires an answer sentence output by the generation model; and a second provision unit that provides the answer sentence to the user.

[0008] According to the above information processing apparatus, it is possible to provide question contents that reflect the user's ideas.

[0009] In the above information processing apparatus, the reception unit receives image information indicating an image of a product as the image information associated with the question object, and the information processing apparatus includes: a product information acquisition unit that acquires product information related to the product indicated by the answer sentence; and an answer information generation unit that generates answer information including the answer sentence and the product information, and the second provision unit provides the answer information to the user.

[0010] According to the above information processing apparatus, it is possible to provide the user with answer information related to a product.

[0011] In the above information processing apparatus, the image information includes coordinate information indicating a specific position or region specified by the user within the image, and the image analysis information acquisition unit performs the image analysis processing on the image indicated by the coordinate information and acquires, as the image analysis information, image analysis information indicating an image analysis result for the image of the coordinate information.

[0012] According to the above information processing apparatus, it is possible to acquire analysis information of an image for a coordinate specified by the user within the image.

[0013] In the information processing apparatus described above, further comprising: an apparatus information acquisition section that acquires apparatus information related to the apparatus indicated by the answer sentence; and an answer information generation section that generates answer information including the candidate question, the answer sentence, and the apparatus information, wherein the apparatus information includes an operation manual related to an operation of the apparatus and an introduction instruction manual related to an introduction of the apparatus, and the second provision section provides the answer information to the user.

[0014] According to the information processing apparatus described above, it is possible to provide the user with an operation manual of an apparatus and an introduction instruction manual related to an introduction of the apparatus.

[0015] In the information processing apparatus described above, it includes a storage section that associates the candidate question and the answer sentence selected by the user and stores the question history record information in a storage.

[0016] According to the information processing apparatus described above, it is possible to accumulate candidate questions and answer sentences.

[0017] In the information processing apparatus described above, the reception section receives image information including an image of a product associated with a question object of the user and customer information, and the answer sentence acquisition section inputs the customer information to the generation model together with the candidate question information.

[0018] According to the information processing apparatus described above, it is possible to obtain an answer that reflects customer information.

[0019] In the information processing apparatus described above, the image analysis information acquisition section acquires a product name as image analysis information indicating an image analysis result, the question history record information acquisition section acquires a plurality of question history record information indicating a question history record in which a question content related to the acquired product name is recorded, and the candidate question information generation section generates candidate question information including a plurality of candidate questions in which a question content for the product name of the image analysis result is recorded, based on the plurality of question history record information.

[0020] According to the information processing apparatus described above, it is possible to generate a plurality of candidate questions related to a product.

[0021] In the information processing apparatus described above, the first provision section displays the plurality of candidate questions in a selectable state on a display section of an external apparatus.

[0022] According to the information processing apparatus described above, it is possible to display a plurality of candidate questions related to a product on an external apparatus.

[0023] In the information processing apparatus described above, the first providing section acquires statistical information of frequencies of occurrence of the candidate questions, and displays the candidate questions in order from a higher rank of the frequencies of occurrence on the display section of the external device.

[0024] According to the information processing apparatus described above, it is possible to display a plurality of candidate questions related to a product in order from a higher rank of frequencies of occurrence.

[0025] The storage medium according to another aspect of the present disclosure stores a program that causes a computer of an information processing apparatus to function as: a receiving section that receives image information including an image associated with a question object of a user; an image analysis information acquiring section that performs image analysis processing on the image information and acquires image analysis information indicating a result of the image analysis; a question history record information acquiring section that acquires, from question history record information indicating a plurality of question history records in which question contents are recorded, question history record information indicating a question history record in which a question content associated with the result of the image analysis indicated by the image analysis information is recorded; a candidate question information generating section that generates candidate question information indicating a candidate question in which a question content for the question object is recorded, based on the question history record information acquired by the question history record information acquiring section; a first providing section that provides the candidate question information to the user in a selectable state; an answer sentence acquiring section that inputs the candidate question information selected by the user to a generation model having a function of outputting an answer sentence for a question content, and acquires an answer sentence output by the generation model; and a second providing section that provides the answer sentence to the user.

[0026] According to the storage medium described above, it is possible to provide a function of a question content reflecting a user's idea to a computer. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a schematic diagram showing a schematic configuration of an information processing system according to the present embodiment.

[0028] Figure 2 is a block diagram showing an example of a hardware structure of an information processing apparatus according to the present embodiment.

[0029] Figure 3 is a block diagram showing an example of a functional structure of an information processing apparatus according to the present embodiment.

[0030] Figure 4 is a schematic diagram showing an example of image information according to the present embodiment.

[0031] Figure 5 is a schematic diagram showing an example of candidate question information according to the present embodiment.

[0032] Figure 6 is a schematic diagram showing an example of the answer information related to the present embodiment.

[0033] Figure 7 is a flowchart showing an example of the control processing of the information processing apparatus related to the present embodiment.

[0034] Figure 8 is a flowchart showing an example of the control processing of the information processing apparatus related to the present embodiment.

[0035] Figure 9 is a block diagram showing an example of the functional structure of the information processing apparatus related to the second modification.

[0036] Figure 10 is a schematic diagram showing an example of the image information related to the second modification.

[0037] Figure 11 is a schematic diagram showing an example of the candidate question information related to the second modification.

[0038] Figure 12 is a schematic diagram showing an example of the image information related to the second modification.

[0039] Figure 13 is a schematic diagram showing an example of the candidate question information related to the second modification.

[0040] Explanation of Reference Numerals

[0041] 1 input / output apparatus

[0042] 2 information processing apparatus

[0043] 211 reception section

[0044] 212 image analysis information acquisition section

[0045] 213 question history record information acquisition section

[0046] 214 candidate question information generation section

[0047] 215 first provision section

[0048] 216 answer sentence acquisition section

[0049] 217 commodity information acquisition section

[0050] 218 answer information generation section

[0051] 219 second provision section

[0052] 220 storage section

[0053] 221 apparatus information acquisition section DETAILED DESCRIPTION

[0054] Hereinafter, an embodiment of an information processing system will be described with reference to the drawings. In the embodiment shown below, an example of application to a store such as a supermarket will be described. In addition, the following embodiment does not limit the present application, and includes elements that can be easily thought of by those skilled in the art, substantially the same elements, and elements within the equivalent range thereof among the constituent elements in the following embodiment. Furthermore, various omissions, substitutions, changes, and combinations of the constituent elements can be made within the scope of the gist of the following embodiment.

[0055] Figure 1 is a schematic diagram showing an outline configuration of an information processing system 10 to which the embodiment is applied. The information processing system 10 is a system including an input / output device 1 (external device) and an information processing device 2.

[0056] The input / output device 1 is, for example, a portable terminal device (hereinafter also referred to as a mobile terminal) such as a smartphone. The input / output device 1 is, for example, a mobile terminal held by a user. In addition, the input / output device 1 can be a tablet-type terminal device. Furthermore, the input / output device 1 can be a mobile terminal lent to a user by a store.

[0057] For example, the input / output device 1 acquires an image input by a user and outputs the acquired image to the information processing device 2. Furthermore, for example, the input / output device 1 acquires candidate question information corresponding to an image output from the information processing device 2, and provides the acquired candidate question information to the user. Furthermore, for example, the input / output device 1 acquires an answer sentence or answer information corresponding to a candidate question output from the information processing device 2, and provides the acquired answer sentence or answer information to the user. Details of the candidate question information, the answer sentence, and the answer information will be described later.

[0058] For example, the information processing device 2 is a store server or the like, or a server device or the like of an operation company that manages and operates a restaurant. Furthermore, in the present embodiment, the information processing device 2 is, for example, a computer including a generation AI that automatically generates a text, such as a large language model (LLM: Large Language Model). The large language model generates an answer sentence corresponding to the content of a question according to, for example, the question about a product. Furthermore, the large language model generates an answer sentence corresponding to the content of a question according to, for example, a question about a process of a device used by a store clerk such as a POS terminal.

[0059] The information processing apparatus 2 is constituted by one or a plurality of computers. The information processing apparatus 2 performs promotion assistance of a product sold in a store on the basis of a question sentence acquired via the input / output apparatus 1. The information processing apparatus 2 is connected to the input / output apparatus 1 via a communication line such as a mobile network.

[0060] Next, the hardware structure of the information processing apparatus 2 will be described using Figure 2 Figure 2 is a block diagram showing an example of the hardware structure of the information processing apparatus 2 according to the embodiment. As shown in Figure 2 , the information processing apparatus 2 includes a CPU (Central Processing Unit) 21, a RAM (Random Access Memory) 22, a ROM (Read Only Memory) 23, an auxiliary storage device 24, and a communication I / F (Interface) 25.

[0061] The CPU 21 is a control main body. The RAM 22 loads programs or various data. The ROM 23 stores various programs. The auxiliary storage device 24 stores various programs or data. The communication I / F 25 is an interface for performing data communication with the input / output apparatus 1.

[0062] The CPU 21, the RAM 22, the ROM 23, the auxiliary storage device 24, and the communication I / F 25 are connected to each other via a bus 26. The CPU 21, the RAM 22, and the ROM 23 constitute a control section. That is, the control section performs the control processing of the information processing apparatus 2 described later by causing the CPU 21 to act in accordance with a control program stored in the ROM 23 or the auxiliary storage device 24 and loaded in the RAM 22.

[0063] The auxiliary storage device 24 is constituted by a nonvolatile memory such as an HDD (Hard Disc Drive) or a flash memory that retains stored information even when a power supply is turned off. The auxiliary storage device 24 is an example of a storage section. The auxiliary storage device 24 stores image information or question information received by the information processing apparatus 2, image analysis information or question history record information acquired by the information processing apparatus 2, an answer sentence, product information, and candidate question information or answer information generated by the information processing apparatus 2. In addition, detailed contents of the image information, the question information, the image analysis information, the question history record information, the answer text, the product information, the candidate question information, and the answer information will be described later.

[0064] ​The information processing apparatus 2 is disposed in a secure environment. The secure environment refers to a configuration that is configured and used in a local environment. By being the configuration, the information processing apparatus 2 described later includes customer information or the like that can be included in input text information for performing processing of a large language model, and thus by configuring and using the information processing apparatus 2 in the local environment, it is possible to contribute to the protection of personal information.

[0065] Figure 3 is a block diagram illustrating an example of a functional configuration of the information processing apparatus 2 according to the embodiment. As shown in Figure 3 , the information processing apparatus 2 includes a reception unit 211, an image analysis information acquisition unit 212, a question history record information acquisition unit 213, a candidate question information generation unit 214, a first provision unit 215, an answer sentence acquisition unit 216, a product information acquisition unit 217, an answer information generation unit 218, a second provision unit 219, and a storage unit 220. In addition, the functional configuration included in the information processing apparatus 2 is not limited thereto.

[0066] The reception unit 211 receives image information including an image associated with an object of a question of a user. Specifically, the reception unit 211 receives image information including an image associated with a question of a user, which is output by the input-output apparatus 1. In addition, the reception unit 211 receives image information representing an image of a product as image information associated with an object of a question of a user. Specifically, the reception unit 211 receives image information representing an image of a product as image information associated with an object of a question of a user, which is output by the input-output apparatus 1.

[0067] Here, the image information will be described using Figure 4 . Figure 4 is a schematic view illustrating an example of image information according to the embodiment. Figure 4 The image information 41 shown in

[0068] The image analysis information acquisition unit 212 performs image analysis processing on image information and acquires image analysis information representing an image analysis result. Specifically, the image analysis information acquisition unit 212 performs image analysis processing on image information received by the reception unit 211 and acquires image analysis information representing an image analysis result. For example, when performing image analysis processing on the image information 41 shown in Figure 4When the illustrated image information 41 performs the image analysis processing, the image analysis information acquisition section 212 acquires image analysis information including the result of the image analysis that is the product name "AA". The image analysis information acquisition section 212 saves the acquired image analysis information in the auxiliary storage device 24.

[0069] In addition, the method of the image analysis is not limited, for example, the product reflected in the image is specified by performing a known object recognition or character recognition technique. The object recognition is also called generic object recognition.

[0070] Returning to Figure 3 , the problem history information acquisition section 213 acquires, from the problem history information of each of the plurality of problem history records in which the problem content is recorded, the problem history information of the problem history record indicating the problem content associated with the image analysis result indicated in the image analysis information. Specifically, the problem history information acquisition section 213 acquires, from the problem history information of each of the plurality of problem history records in which the problem content is recorded stored in the auxiliary storage device 24, the problem history record indicating the problem content associated with the image analysis result indicated in the image analysis information acquired by the image analysis information acquisition section 212. For example, if the result of the image analysis is the product name "AA", the problem history information acquisition section 213 extracts the problem history information associated with the product name "AA" from the auxiliary storage device 24. That is, the image analysis information acquisition section 212 acquires the product name as the image analysis information indicating the image analysis result. The problem history information acquisition section 213 acquires the plurality of problem history information indicating the problem history record in which the problem content associated with the product name of the image analysis result is recorded.

[0071] The candidate problem information generation section 214 generates candidate problem information indicating a candidate problem in which the problem content for the problem object is recorded, based on the problem history information. Specifically, the candidate problem information generation section 214 generates the candidate problem information indicating the candidate problem in which the problem content for the problem object is recorded, based on the problem history information acquired by the problem history information acquisition section 213. Here, the Figure 5 candidate problem information will be described. Figure 5 is a schematic view indicating an example of the candidate problem information related to the embodiment. In Figure 5 the candidate problem information 51 illustrated, a plurality of candidate problems 52 are included. The providing section 215 to be described later displays the plurality of candidate problems 52 in a selectable state on the display section of the input output device 1 (external device). In addition, in Figure 5In the display example of FIG. 5, the pull-down menu 53 is shown so that the user can select one of the candidate questions 52 from among the plurality of candidate questions.

[0072] For example, in a case where the question history record information associated with the product name "AA" includes a question to confirm whether or not the product with the product name "AA" is sold in the store, the candidate question information generating section 214 generates a question such as "Is AA sold in Store B?". Further, in a case where the other question history record information associated with the product name "AA" includes a question to confirm the sales price of the product with the product name "AA" in the store, the candidate question information generating section 214 generates a question such as "Please tell me the price of AA in Store B". The question history record information obtaining section 213 obtains a plurality of question history record information. Then, the candidate question information generating section 214 generates the candidate question information 51 with each of the generated questions as the candidate question 52. That is, the candidate question information generating section 214 generates the candidate question information 51 indicating the candidate questions in which the question content for the product name of the result of the image analysis is described, based on the plurality of question history record information. The plurality of candidate questions 52 are included in the candidate question information 51.

[0073] Further, with respect to "B Store" indicating the store name, either the store name can be obtained from the question history record information or the store name included in the customer information and frequently used by the user can be extracted and added to the question content. Further, with respect to the display order of the candidate questions, the statistical information of the frequency of appearance of the candidate questions or the like can be obtained from the auxiliary storage device 24 and the candidate questions can be displayed in order from the upper rank of the higher frequency of appearance.

[0074] Returning to Figure 3 , the first providing section 215 provides the candidate question information to the user in a selectable state. Specifically, the first providing section 215 provides the candidate question information generated by the candidate question information generating section 214 to the user in a selectable state. For example, the first providing section 215 outputs the candidate question information 51 in a selectable state and provides it to the user. Further, the first providing section 214 obtains the statistical information of the frequency of appearance of the candidate questions 52 from the auxiliary storage device 24. Then, the candidate questions 52 are displayed in order from the upper rank of the higher frequency of appearance on the input / output device 1.

[0075] The receiving section 211 receives the question information including the candidate question selected by the user. Specifically, the receiving section 211 receives the question information indicating the candidate question selected by the user, which is output from the input / output device 1. The question information can include the personal information of the user who proposes the question, such as the name, the telephone number, and the e-mail address, or the like. Further, the receiving section 211 saves the received question information in the auxiliary storage device 24.

[0076] The answer sentence acquisition unit 216 inputs the candidate question information selected by the user into a generation model having a function of outputting an answer sentence for the content of a question, and acquires an answer sentence output by the generation model. Specifically, the answer sentence acquisition unit 216 inputs the candidate question information received by the reception unit 211 into a generation model having a function of outputting an answer sentence for the content of a question, and acquires an answer sentence output by the generation model.

[0077] For example, the answer sentence acquisition unit 216 inputs the candidate question information, instruction information (also referred to as a hint) indicating an answer to the candidate question, to a large language model using a publicly known method, thereby causing the large language model to generate an answer sentence to the answer question sentence, and acquires an answer sentence output by the large language model. The answer sentence includes at least product information related to the candidate question and an article related to a promotion of the product. The product information is, for example, a type of the product, a name of the product, and the like. In addition, the question sentence can be included in the hint.

[0078] For example, the answer sentence acquisition unit 216 generates a hint indicating (recording) that product information related to the candidate question is added, an article related to a promotion of the product is added, and the like, in order to answer the question from the user. In addition, the hint can use a hint prepared in advance in the auxiliary storage device 24, or can be dynamically generated by the answer sentence acquisition unit 216. Furthermore, the hint can include a record indicating an information source (resource) in which information on the product as a candidate is stored. Also, if the image information includes customer information, the customer information can be added to the hint. Thereby, an answer sentence of an answer reflecting the customer information can be acquired. The answer sentence acquisition unit 216 stores the acquired answer sentence in the auxiliary storage device 24.

[0079] The product information acquisition unit 217 acquires product information related to a product indicated in the answer sentence. Specifically, the product information acquisition unit 217 acquires product information related to a product indicated in the answer sentence from the auxiliary storage device 24. The product information includes product information related to a product as a target of a promotion, such as a type of the product, a product code, a product name, a price, and the like. In addition, the product as a target of a promotion can be set to a product as a target of a discount, a product with a large number of sales, a product with a large inventory among products owned by a store, a product as a target of an event, and the like.

[0080] The answer information generation unit 218 generates answer information including an answer sentence and product information. Specifically, the answer information generation unit 218 generates answer information including an answer sentence acquired by the answer sentence acquisition unit 216 and product information acquired by the product information acquisition unit 217. Here, the answer information is explained using Figure 6 Figure 6 is a schematic view showing an example of the answer information according to the embodiment. In Figure 6 ​The answer information 61 shown in the drawing shows the candidate question 62 and the answer sentence 63 selected by the user.

[0081] For example, if the candidate question 62 is "Does AA sell at B store?", the answer sentence 63 is "AA sells at B store." In addition, although information on the product information is not shown in the answer information 61, the content of the answer sentence 63 can include, for example, the price related to the product name "AA." The content of the answer text 63 can include a message (product information) of "on sale" as information indicating that it is a sales promotion object.

[0082] Returning to Figure 3 , the second providing section 219 provides the answer sentence to the user. Specifically, the second providing section 219 provides the answer sentence acquired by the answer sentence acquiring section 216 to the user. For example, the second providing section 219 outputs the answer sentence to the input / output device 1 and provides it to the user.

[0083] In addition, the second providing section 219 provides the answer information 61 to the user. Specifically, the second providing section 219 provides the answer information 61 generated by the answer information generating section 218 to the user. For example, the second providing section 219 outputs the answer information 61 to the input / output device 1 and provides it to the user.

[0084] The saving section 220 saves the candidate question selected by the user as the question history record information in the storage section. Specifically, the saving section 220 stores the candidate question selected by the user received by the receiving section 211 in the auxiliary storage device 24 as the question history record information.

[0085] In addition, the saving section 220 saves the answer information as the question history record information in the storage section. Specifically, the saving section 220 saves the answer information generated by the answer information generating section 218 in the auxiliary storage device 24 as the question history record information.

[0086] Next, the control processing of the information processing apparatus 2 will be described using Figure 7 and Figure 8 . Figure 7 and Figure 8 are flowcharts showing an example of the control processing of the information processing apparatus 2 according to the embodiment. Figure 7 The control processing of providing the candidate question to the user will be described.

[0087] The reception section 211 receives image information indicating an image associated with the user's question, which is output from the input / output device 1 (step S71). Next, the image analysis information acquisition section 212 performs image analysis processing on the image information received by the reception section 211, and acquires image analysis information indicating the image analysis result (step S72). Next, the question history record information acquisition section 213 acquires question history record information indicating a question history record in which the content of a question associated with the image analysis result indicated by the image analysis information acquired by the image analysis information acquisition section 212 is recorded, from the question history record information of each question history record indicating a plurality of question history records in which the content of a question is recorded, which is stored in the auxiliary storage device 22 (step S73).

[0088] Next, the candidate question information generation section 214 generates candidate question information indicating a candidate question in which the content of a question for a question object is recorded, based on the question history record information acquired by the question history record information acquisition section 213 (step S74). Next, the first provision section 215 provides the candidate question information generated by the candidate question information generation section 214 to the user in a selectable state (step S75). When this processing ends, the processing of the information processing device 2 ends.

[0089] Accordingly, the user can confirm a candidate question associated with the image. Further, the user can select a candidate question reflecting the user's idea from the confirmed candidate questions. Therefore, the information processing device 2 can provide information reflecting the user's idea in accordance with the image input by the user.

[0090] Figure 8 A control process of providing an answer to a candidate question to the user is explained. The reception section 211 receives question information indicating a candidate question selected by the user, which is output from the input / output device 1 (step S81). Next, the answer sentence acquisition section 216 inputs the candidate question information received by the reception section 211 to a generation model having a function of outputting an answer sentence for the content of a question, and acquires an answer sentence output from the generation model (step S82). Next, the product information acquisition section 217 acquires product information on a product shown in the answer sentence, from the auxiliary storage device 24 (step S83).

[0091] Next, the answer information generation section 218 generates answer information including the answer sentence acquired by the answer sentence acquisition section 216 and the product information acquired by the product information acquisition section 217 (step S84). Next, the second provision section 219 provides the answer sentence acquired by the answer sentence acquisition section 216 to the user. Further, the second provision section 219 provides the answer information 61 generated by the answer information generation section 218 to the user (step S85).

[0092] Next, the saving section 220 saves the candidate question selected by the user, which is received by the receiving section 211, as the question history record information in the auxiliary storage device 24. Further, the saving section 220 stores the answer information generated by the answer information generating section 218 in the auxiliary storage device 24 as the question history record information (step S86). When this processing ends, the processing of the information processing apparatus 2 ends. Thereby, the user can confirm the answer sentence corresponding to the candidate question.

[0093] As described above, the information processing apparatus 2 of the present embodiment receives image information indicating an image related to a question of a user, executes image analysis processing on the image information, and acquires image analysis information indicating image analysis performed on the image information, acquires question history record information of a question history record related to the image analysis shown in the image analysis information, generates candidate question information indicating a candidate question of the question based on the question history record information, and provides the candidate question information to the user. Then, the information processing apparatus 2 inputs the candidate question selected by the user to a large language model having a function of outputting an answer sentence with respect to the candidate question selected by the user, acquires the answer sentence output by the large language model, and provides the candidate question and the answer sentence to the user.

[0094] Thereby, it is possible to provide the user with candidate question information reflecting the idea of the user in the candidate question information provided by the information processing apparatus 2. Then, it is possible to provide the user with answer information corresponding to the candidate question selected by the user in the answer information provided by the information processing apparatus 2. Therefore, the information processing apparatus 2 can provide question content reflecting the idea of the user according to the image.

[0095] In addition, the above-described embodiments can be implemented by appropriately modifying a part of the structure or function possessed by each of the above-described apparatuses. Therefore, several modified examples related to the above-described embodiments will be described as other embodiments below. In addition, the points different from the above-described embodiments will be mainly described below, and the detailed description of the points common to the already described content will be omitted. Further, the modified examples described below can be implemented alone or in appropriate combination.

[0096] (First Modified Example)

[0097] For example, the image information can include coordinate information indicating a specific position or area within the image that the user designates. When a plurality of goods are shown in the same image, the image information includes coordinate information. The coordinate information is two-dimensional coordinates on the image, and is coordinates of a position indicated by the user by a touch operation or the like on the image. For example, the coordinate information is set by the user performing a touch operation on a specific good among the plurality of goods. At this time, the image analysis information acquisition section 212 of the information processing apparatus 2 performs image analysis processing on the image indicated by the coordinate information, and acquires image analysis information indicating an image analysis result for the image of the coordinate information as the image analysis information.

[0098] Then, the question history information acquisition section 213 acquires question history information indicating a question history in which a question content associated with the image analysis result is recorded, the candidate question information generation section 214 generates candidate question information based on the question history information, and the first provision section 215 provides the candidate question information to the user in a selectable state. By this means, the information processing apparatus 2 can provide the user with a question related to the good designated by the user.

[0099] (Second Modified Example)

[0100] In the above-described embodiment, with regard to the image information, an image related to a good sold at a store or the like has been described. In the second modified example, with regard to the image information, an image related to an apparatus used in a store or the like will be described.

[0101] Figure 9 is a block diagram showing an example of a functional structure of the information processing apparatus 2 related to the second modified example. As shown in Figure 9 , the information processing apparatus 2 includes a reception section 211, an image analysis information acquisition section 212, a question history information acquisition section 213, a candidate question information generation section 214, a first provision section 215, an answer sentence acquisition section 216, a good information acquisition section 217, an answer information generation section 218, a second provision section 219, a storage section 220, and an apparatus information acquisition section 221.

[0102] The reception section 211 receives image information indicating an image associated with a question of a user, which is output from the input / output apparatus 1. In the second modified example, the input / output apparatus 1 is an apparatus used in a store, such as a POS terminal. Here, the image information will be described. Figure 10 Figure 10 is a schematic view showing an example of the image information related to the second modified example.

[0103] For example, Figure 10 ​The illustrated image information 101 is an image related to a display screen of a POS terminal used in a store, and shows product registration information 102 indicating a list of products registered in the POS terminal, product individual information 103 about each product related to the number and unit price of a product "KKK" displayed in the product registration information 102, and amount information 104 of a total value of an amount obtained by multiplying the number of each product displayed in the product registration information 102 by the unit price. The image information 101 can include information such as an identifier of the POS terminal that transmits the image information 101 and an identifier of a user who uses the POS terminal.

[0104] Returning to Figure 9 , the image analysis information acquisition section 212 performs image analysis processing on the image information 101 received by the reception section 211, and acquires image analysis information indicating the image analysis result for the image information. Further, the problem history information acquisition section 213 acquires, from the auxiliary storage device 24, problem history information indicating a problem history related to the image analysis result indicated in the image analysis information acquired by the image analysis information acquisition section 212. Also, the candidate problem information generation section 214 generates candidate problem information indicating a candidate problem of a problem based on the problem history information acquired by the problem history information acquisition section 213.

[0105] For example, if the image analysis result is a screen of a POS terminal, the problem history information acquisition section 213 acquires, from the auxiliary storage device 24, a problem history associated with the POS terminal. Then, the candidate problem information generation section 214 generates the candidate problem information in a manner that displays the candidate problems in order from a higher level with a higher frequency of occurrence, according to the problem history associated with the POS terminal.

[0106] Here, the Figure 11 candidate problem information will be described. Figure 11 is a schematic view indicating an example of the candidate problem information involved in the second modification example. Figure 11 The candidate problem information 111 illustrated indicates a plurality of candidate problems 112. Further, in the Figure 11 , a drop-down menu 113 from which a user can select one candidate problem is indicated with respect to the plurality of candidate problems 112.

[0107] In the Figure 11 candidate problems 112, a plurality of candidate problems such as "Please tell me the operation method of the POS", "How to operate in the case of canceling a product?", "How to deal with a product with a discount label?", and the like are indicated with respect to the operation of the POS terminal.

[0108] Returning to Figure 9The first providing section 215 outputs the candidate question information 111 to the input / output device 1 and provides it to the user. At this time, with respect to the candidate question included in the candidate question information 111 that the user outputs to the input / output device 1, it is possible that the idea of the user is not reflected. In this case, the user again specifies the image associated with the question of the user, and transmits the image information including the two-dimensional coordinates in the specified image specified by the user from the input / output device 1 to the information processing device 2.

[0109] Here, the two-dimensional coordinates in the image specified by the user will be explained using Figure 12 Figure 12 is a schematic view that shows an example of the image information involved in the second modification example. Figure 12 is a schematic view that shows the image information 101 shown in Figure 10 to which a schematic view showing the specified image 105 specified by the user is added. Figure 12 The specified image 105 shown in

[0110] Returning to Figure 9 , the receiving section 211 again receives the image information 101 including the coordinate information corresponding to the image, and the image analysis information acquisition section 212 performs the image analysis processing on the image represented by the coordinate information with respect to the image information 101 that the receiving section 211 has received, and acquires the image analysis information representing the image analysis of the image for the coordinate information as the image analysis information. Further, the question history record information acquisition section 213 acquires the question history record information representing the question history record associated with the image analysis represented in the image analysis information acquired by the image analysis information acquisition section 212 from the auxiliary storage device 24. Also, the candidate question information generation section 214 generates the candidate question information representing the candidate question of the question based on the question history record information acquired by the question history record information acquisition section 213.

[0111] Here, the candidate question information regenerated will be explained using Figure 13 Figure 13 is a schematic view that shows an example of the candidate question information involved in the second modification example. Figure 13 The plurality of candidate questions 132 are shown in the candidate question information 131 shown in Figure 13 The drop-down menu 133 is shown so that the user can select one candidate question from the plurality of candidate questions 132.

[0112] In the candidate questions 132 shown in Figure 13 , with respect to the indicator of the POS terminal, a plurality of candidate questions such as "When does the indicator display?", "Please tell the list of icons displayed on the indicator", "Please tell the response when the icon is displayed on the indicator" are shown. ​​

[0113] Returning to Figure 9 , the first providing section 215 outputs the candidate question information 131 to the input / output device 1 and provides it to the user. Further, the receiving section 211 receives question information indicating a candidate question selected by the user, which is output from the input / output device 1, and the answer sentence acquisition section 216 inputs the candidate question received by the receiving section 211 to a large language model having a function of outputting an answer sentence for the candidate question, and acquires an answer sentence output from the large language model.

[0114] The device information acquisition section 221 acquires device information related to a device shown in the answer sentence. Specifically, the device information acquisition section 221 acquires device information related to a device shown in the answer sentence from the auxiliary storage device 24. The device information includes, for example, an operation manual related to the operation of the device and an introduction manual related to the introduction of the device.

[0115] The answer information generation section 218 generates answer information including the candidate question, the answer sentence, and the device information. Specifically, the answer information generation section 218 generates answer information including the candidate question received by the receiving section 211, the answer sentence acquired by the answer sentence acquisition section 216, and the device information acquired by the device information acquisition section 221. Further, the second providing section 219 provides the answer information generated by the answer information generation section 218, which includes the candidate question, the answer sentence, and the device information, to the user.

[0116] Then, the saving section 220 associates the candidate question selected by the user received by the receiving section 211, the answer sentence acquired by the answer sentence acquisition section 216, and the device information acquired by the device information acquisition section 221, and saves them as question history record information in the auxiliary storage device 24. Accordingly, even if a candidate question not reflecting the user's idea is provided from the information processing device 2, a candidate question reflecting the user's idea can be provided to the user by again designating an image reflecting the user's idea. In addition, the user can designate an image reflecting the user's idea at any time, and transmit image information including two-dimensional coordinates in the designated image from the input / output device 1 to the information processing device 2.

[0117] (Third Modification)

[0118] In the above-described embodiment, the information processing device 2 including the large language model is configured to be arranged in a local environment and used, but is not limited thereto, and the information processing device 2 including the large language model can be configured to be arranged in a high-security cloud environment and used. The high-security cloud environment is a cloud environment that is not disclosed in the Internet but is connected via an internal server environment and a highly confidential communication line.

[0119] In addition, the program executed by the information processing apparatus 2 of the embodiments can be stored in a computer connected to a network such as the Internet, and can be provided by downloading via the network. Furthermore, the program executed by the information processing apparatus 2 of the embodiments and the modified examples can be provided or distributed via a network such as the Internet.

[0120] In addition, the program executed by each of the devices of the above-described embodiments is provided in a state of being installed in advance in a ROM, a storage section, or the like. The program executed by each of the devices of the above-described embodiments is stored in a computer-readable recording medium such as a CD-ROM, a floppy disk (FD), a CD-R, or a DVD (Digital Versatile Disk) in a file in an installable form or an executable form and is provided.

[0121] In addition, the program executed by each of the devices of the above-described embodiments can be stored in a computer connected to a network such as the Internet, and can be provided by downloading via the network. Furthermore, the program executed by each of the devices of the above-described embodiments can be provided or distributed via a network such as the Internet.

[0122] Thus far, the embodiments of the present application have been described, but the present embodiments are presented as examples and are not intended to limit the scope of the application. These novel embodiments and the modified examples thereof can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations can be made within the scope of the gist of the present application. These embodiments and the modified examples thereof are included in the scope or gist of the present application, and are included in the scope of the present application and equivalents thereof recited in the claims.

Claims

1. An information processing device, characterized in that include: a receiving unit that receives image information including an image associated with a user's question object; an image analysis information acquisition unit that performs image analysis processing on the image information and acquires image analysis information indicating an image analysis result; a question history information acquisition unit that acquires question history information indicating a question history recording question contents associated with the image analysis result indicated by the image analysis information from question history information indicating each of a plurality of question history recording question contents; a candidate question information generating unit that generates candidate question information indicating a candidate question having a question content for the question object, based on the question history information acquired by the question history information acquiring unit; a first providing unit, providing the candidate question information to the user in a selectable state; an answer sentence acquisition unit that inputs the candidate question information selected by the user into a generation model having a function of outputting an answer sentence in response to the question content, and acquires the answer sentence output by the generation model; as well as The second providing unit provides the answer sentence to the user.

2. The information processing device according to claim 1, wherein The receiving unit receives image information representing an image of a product as image information associated with the question object, wherein: The information processing device includes: a product information acquisition unit that acquires product information related to the product indicated by the answer sentence; and The answer information generating unit generates answer information including the answer sentence and the product information, wherein: The second providing unit provides the answer information to the user.

3. The information processing device according to claim 1, wherein The image information includes coordinate information indicating a specific position or area specified by the user within the image. The image analysis information acquisition unit executes the image analysis process on the image indicated by the coordinate information, and acquires image analysis information indicating an image analysis result of the image indicated by the coordinate information as the image analysis information.

4. The information processing device according to claim 3, wherein: Also includes: a device information acquisition unit that acquires device information related to the device indicated by the answer sentence; as well as The answer information generating unit generates answer information including the candidate question, the answer sentence and the device information, wherein: The device information includes an operation manual related to the operation of the device and an introduction manual related to the introduction of the device. The second providing unit provides the answer information to the user.

5. The information processing device according to any one of claims 1 to 4, comprising: The storage unit associates the candidate question selected by the user with the answer sentence and stores the associated answer in a storage unit as the question history information. The information processing apparatus according to claim 1 , wherein: The receiving unit receives image information including an image of a product related to a question object of the user and customer information, The answer sentence acquisition unit inputs the customer information and candidate question information into the generation model.

7. The information processing apparatus according to claim 2, wherein: The image analysis information acquisition unit acquires a product name as image analysis information indicating an image analysis result. The question history information acquisition unit acquires a plurality of question history information indicating question history records containing question contents related to the acquired product name. The candidate question information generating unit generates candidate question information including a plurality of candidate questions each including a question content regarding a product name of an image analysis result, based on a plurality of question history information.

8. The information processing apparatus according to claim 7, wherein: The first providing unit displays a plurality of candidate questions on a display unit of an external device in a selectable state.

9. The information processing apparatus according to claim 8, wherein: The first providing unit acquires statistical information on the frequency of appearance of candidate questions and displays the candidate questions in order from the highest frequency of appearance on the display unit of the external device.

10. A storage medium storing a program that causes a computer of an information processing device to function as: a receiving unit that receives image information including an image associated with a user's question object; an image analysis information acquisition unit that performs image analysis processing on the image information and acquires image analysis information indicating an image analysis result; a question history information acquisition unit that acquires question history information indicating a question history recording question contents associated with the image analysis result indicated by the image analysis information from question history information indicating each of a plurality of question history recording question contents; a candidate question information generating unit that generates candidate question information indicating a candidate question having a question content for the question object, based on the question history information acquired by the question history information acquiring unit; a first providing unit, providing the candidate question information to the user in a selectable state; an answer sentence acquisition unit that inputs the candidate question information selected by the user into a generation model having a function of outputting an answer sentence in response to the question content, and acquires the answer sentence output by the generation model; as well as The second providing unit provides the answer sentence to the user.

Images