Data processing apparatus, data processing method, and program

Abstract

To provide a data processing apparatus, a data processing method, and a program capable of appropriately responding to a user's speech.SOLUTION: A data processing apparatus includes: an input unit for receiving a user input; an acquisition unit for acquiring a unique prompt associated with an object around a user; a processing unit which performs specific processing using a text generation model which generates a text according to an input text; and an output unit which outputs a result of the specific processing. The processing unit includes inputting, to the text generation model, information obtained from the user input and the unique prompt acquired by the acquisition unit, as the input text.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a data processing device, a data processing method, and a program. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] In an AI model such as a chatbot, in order to output an appropriate answer, it is necessary to have the information necessary for the appropriate answer. However, with conventional technology, the information necessary for the answer may be insufficient. In addition, there are limitations in terms of the effort and cost involved in reconstructing the AI ​​model to obtain the necessary information. Therefore, there is room for improvement in the current AI model technology in terms of responding appropriately to user utterances. [Means for solving the problem]

[0005] A first aspect of the data processing device according to the technology disclosed herein includes an input unit that accepts user input, an acquisition unit that acquires a unique prompt associated with an object around the user, a processing unit that performs a specific processing using a sentence generation model that generates a sentence according to the input sentence, and an output unit that outputs a result of the specific processing, wherein the processing unit inputs information obtained from the user input and the unique prompt acquired by the acquisition unit into the sentence generation model as the input sentence.

[0006] A second aspect of the data processing device according to the technology disclosed herein relates to the data processing device of the first aspect described above, wherein the acquisition unit further acquires a user prompt associated with a user who performs the user input, and the processing unit inputs the input sentence further including the user prompt into the sentence generation model.

[0007] A data processing device of a third aspect according to the technique of the present disclosure relates to the data processing device of the first or second aspect described above, wherein the acquisition unit further acquires position information of the object.

[0008] A fourth aspect of the data processing device according to the technology disclosed herein is related to any one of the first to third aspects described above, and further includes an emotion determination unit that determines an emotion of a user, and the processing unit inputs the input sentence, which further includes the emotion of the user, into the sentence generation model.

[0009] A data processing method of a fifth aspect related to the technology disclosed herein includes the steps of accepting user input, acquiring a unique prompt associated with an object around the user, generating an input sentence to be input to a sentence generation model that generates a sentence corresponding to the input sentence from information obtained from the user input and the acquired unique prompt, inputting the generated input sentence to the sentence generation model to execute a specific process, and outputting a result of the specific process.

[0010] A sixth aspect of the program relating to the technology of the present disclosure causes at least one processor of a computer constituting a data processing device to perform the following processes: accepting user input; acquiring a unique prompt associated with an object around the user; generating an input sentence to be input into a sentence generation model that generates a sentence corresponding to the input sentence from information obtained from the user input and the acquired unique prompt; and inputting the generated input sentence into the sentence generation model to execute a specific process. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a conceptual diagram illustrating an example of a configuration of a data processing system. [Diagram 2] FIG. 2 is a conceptual diagram showing an example of main functions of a data processing device and a smart device. [Figure 3A] 13 shows an overview of an example of a specific process. [Figure 3B] 13 shows an outline of another example of the specific processing. [Figure 4] 2 shows a schematic functional configuration of a specific processing unit of the data processing device. [Diagram 5] 13 illustrates an example of an operational flow of a specific process performed by a data processing device. [Figure 6A] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 6B] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an example of an embodiment of a data processing device, a data processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0013] First, the terms used in the following description will be explained.

[0014] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0015] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0016] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0017] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0019] FIG. 1 shows an example of the configuration of a data processing system 10 according to the embodiment.

[0020] As shown in FIG. 1, a data processing system 10 may include a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server. An example of the smart device 14 is a smartphone. In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of the present disclosure. Note that various components of the data processing device according to the technology of the present disclosure may be mounted on a device other than the data processing device 12 described below, such as the smart device 14.

[0021] The data processing device 12 may include a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 may include a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 may be connected to each other via a bus 34 so as to be able to communicate with each other. The database 24 and the communication I / F 26 may also be connected to each other via the bus 34 so as to be able to communicate with each other. The database 24 may store various information necessary for executing processes including specific processes described later. The database 24 may be realized in the form of a cloud or the like. The communication I / F 26 may be connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0022] The smart device 14 may include a computer 36, a reception device 38, an output device 40, a camera 42, a GPS (Global Positioning System) 43, and a communication I / F 44. The computer 36 may include a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 may be connected to a bus 52. The reception device 38, the output device 40, the camera 42, and the GPS 43 may also be connected to the bus 52.

[0023] The reception device 38 may include a touch panel 38A and a microphone 38B, and can receive user input. The touch panel 38A can receive user input by touching an indicator (e.g., a pen or a finger) by detecting the contact of the indicator. The microphone 38B can receive user input by voice by detecting the user's voice. The processor 46 transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0024] The output device 40 may include a display 40A and a speaker 40B, and presents data to the person 20 by outputting the data in a form of expression (e.g., voice and / or text) that can be perceived by the person 20. The display 40A displays visual information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46.

[0025] The camera 42 may be, for example, a built-in type camera of a smartphone, and may be a small digital camera equipped with an optical system such as a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. In addition to capturing an image of an object, the camera 42 may have a reading function capable of reading a code such as a two-dimensional code when the object is a code. The GPS 43 may obtain location information of the smart device 14.

[0026] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 handle the transmission and reception of various information between the processor 46 and the processor 28 via the network 54. In addition, the communication I / F 44 is also compatible with short-range wireless communication such as near field communication (NFC) represented by RFID (Radio frequency identification) and infrared communication.

[0027] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Fig. 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 may be stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 can read the specific processing program 56 from the storage 32 and execute the read specific processing program 56 on the RAM 30. The specific processing can be realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 may store a data generation model 58 as an example of a sentence generation model and an emotion identification model 60. The data generation model 58 and the emotion identification model 60 may be used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output process. The storage 50 may store a reception output program 62. The reception output program 62 may be used together with the specific processing program 56 by the data processing system 10. The processor 46 may read out the reception output program 62 from the storage 50 and execute the read reception output program 62 on the RAM 48. The reception output process may be realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 62 executed on the RAM 48.

[0031] Next, the processing of the specific processing unit 290 will be described when the data processing device 12 performs a series of specific processing steps, in which the data processing device 12 obtains a prompt associated with an object near the user, combines it with the user input, inputs it into a sentence generation model, and obtains an answer.

[0032] A sentence generation model capable of outputting specific answer data from an input sentence (input data) performs output based on parameter information collected in advance. Therefore, a general sentence generation model does not reflect in the output result new information discovered after the sentence generation model is generated, or detailed information that could not be collected when the sentence generation model was generated. In consideration of this point, the present disclosure aims to increase the possibility of obtaining the answer data desired by the user by adding information necessary for the answer to the input sentence as additional information when the user inputs the input.

[0033] In consideration of the above, in the present disclosure, information specific to various objects (specifically, people, things, living things, places including facilities, etc.) existing around the user who has the smart device 14 is prepared in advance as specific prompts. Then, when a specific process is executed in the data processing device 12, the specific prompts are collected and reflected in the input sentences of the sentence generation model, thereby improving the accuracy of answers.

[0034] A unique prompt, which is information unique to an object and is used in a specific process, can be set in advance by various methods. Below, a method for setting a unique prompt is illustrated for each object. Note that the combination of the setting method and object described below is merely an example, and different combinations may be adopted. Also, methods other than the setting methods illustrated below may be adopted as long as the unique prompt of an object can be obtained at any time.

[0035] For example, when the object is an object, such as a product sold in a store, a manufacturer that produces or sells the product may store information about the product as a unique prompt in the database 24, and a code (e.g., a two-dimensional code) or a tag (e.g., an IC tag) linked to the stored unique prompt may be printed or attached to the product. In this case, the user may use the smart device 14 to read the code attached to the product, or may bring the smart device 14 close to the product, whereby the specific processing unit 290 may obtain information about the unique prompt associated with the product from the database 24.

[0036] Also, for example, if the object is a living thing, for example, a living thing kept in an aquarium, a zoo, or a pet shop, there is a method in which a staff member of the aquarium or the like installs a beacon (specifically, an infrared beacon, a BLE beacon, a wireless router, or the like) for each tank in which living things, for example, fish, are kept, and a unique prompt associated with the fish in the tank is transmitted from the beacon to the smart device 14. In this case, the specific processing unit 290 can acquire the unique prompt by transmitting the unique prompt acquired by the user carrying the smart device 14 approaching the tank (more specifically, a beacon attached to the tank) to the data processing device 12.

[0037] Furthermore, for example, when the object is a place, such as a property managed by a real estate company or a facility such as a tourist spot, an employee of the real estate company or the like may store information about the property as a unique prompt in the database 24 in association with the location information of the property. In this case, the specific processing unit 290 can obtain the unique prompt of the property from the database 24 by detecting that the user has approached the property while holding the smart device 14 equipped with the GPS 43, for example.

[0038] Furthermore, for example, if the object is a person, the person may directly register information about himself / herself in the smart device 14 that he / she owns. In this case, the specific processing unit 290 can acquire the unique prompt by transmitting the unique prompt acquired by executing data communication between the smart devices 14 owned by the people to the data processing device 12.

[0039] Here, the person as the object described above may include a user who uses the data processing device 12 of the present disclosure. Note that a prompt registered by a user himself / herself is used in a different manner in the data processing device 12 than a unique prompt associated with another object. Therefore, in the following explanation, a prompt registered by a user himself / herself will be called a "user prompt" and will be distinguished from a unique prompt for the sake of explanation.

[0040] The contents of the user prompt may be set directly by the user, may be automatically generated from information input by the user, or may be automatically generated based on the user's tracking information, information on a social networking service (SNS), etc. The details of the information on which the user prompt is based are not particularly limited, but may include, for example, personal information such as the user's gender and residential area, and information such as hobbies, preferences, and interests. The generated user prompt may be stored in the storage 50 in the smart device 14 or in the database 24 of the data processing device 12.

[0041] The specific process of this embodiment is executed in a state where the above-mentioned specific prompt or user prompt has been set. An overview of the specific process will be described below using an example in which the specific process is executed in an aquarium.

[0042] 3A is an explanatory diagram showing an example of a situation in which a specific process is executed. Here, a beacon 75 is installed in each tank 70 in the aquarium, and the beacon 75 is set in advance by the aquarium staff or the like to transmit a unique prompt that constitutes information about the creatures, for example, fish, kept in the tank 70.

[0043] When a user carrying the smart device 14 visits the aquarium and moves around the aquarium, the smart device 14 is ready to receive a unique prompt transmitted from a beacon 75 installed in a nearby aquarium 70. If the user then performs a user input to the smart device 14 to execute a specific process, the specific process can be executed using the unique prompt. The specific processing unit 290 can receive the desired unique prompt by having the smart device 14 transmit the unique prompt transmitted from the beacon 75 to the data processing device 12.

[0044] As shown in FIG. 4, the specific processing unit 290 includes an input unit 292, an acquisition unit 294, a processing unit 296, and an output unit 298.

[0045] The input unit 292 acquires a user input received by the smart device 14. Specifically, the input unit 292 can acquire, as the user input, a voice input of the user received by the microphone 38B of the smart device 14 or text information input via the touch panel 38A.

[0046] The acquisition unit 294 acquires a unique prompt associated with an object around the user. As described above, there are several examples of a method for acquiring the unique prompt, related to the method for setting the unique prompt. Specifically, there is a method in which the unique prompt acquired by the smart device 14 is transmitted to the data processing device 12, and a method in which a specific unique prompt is identified from among the unique prompts stored in advance in the database 24 based on the position information of the smart device 14 or information acquired by the smart device 14 via a code or tag.

[0047] In addition, the acquisition unit 294 can acquire a user prompt associated with a user who performs user input, specifically, a user who owns the smart device 14. The user prompt can be acquired in the same manner as the unique prompt. In addition, it is preferable that the content of the user prompt can be updated on the smart device 14 side.

[0048] However, when performing a specific process, there are cases where multiple objects are present around the user. For example, in an aquarium, multiple tanks for different creatures are often arranged adjacent to each other. In addition, it often happens that users act in groups of three or more people, or are in a place (for example, a residential area) where properties with unique prompts are densely packed together. In such a case, the acquisition unit 294 may acquire multiple unique prompts. Here, if a large amount of information that is less relevant to the user input is included in the input sentence, this may adversely affect the accuracy of the answer of the sentence generation model. Therefore, in the acquisition unit 294 of this embodiment, as a method of estimating the importance or priority of the unique prompt, position information of the object to which the unique prompt is set is further acquired.

[0049] More specifically, for example, when the reception device 38 receives a user input from a user who is viewing a specific tank, for example, tank 70A, in an aquarium, the user input is likely to be related to the creatures in tank 70A. However, in an actual aquarium, as shown in FIG. 3B, there may be several tanks, for example, two tanks 70B and 70C, surrounding the user, each of which has a different creature than that in tank 70A. In such a case, depending on the communication range of beacons 75A, 75B, and 75C installed in each tank 70A, 70B, and 70C, the smart device 14 may receive all the unique prompts transmitted from each beacon 75A, 75B, and 75C.

[0050] The three unique prompts received by the smart device 14 from the beacons 75A, 75B, and 75C and transmitted to the acquisition unit 294 are used in the input sentence to the sentence generation model by the processing unit 296. At this time, if the acquired multiple unique prompts are simply used in the input sentence, the importance or priority of each unique prompt is unknown, so there is a risk that the answer of the sentence generation model will be different from what the user wants. Therefore, the acquisition unit 294 not only acquires the unique prompts, but also acquires position information of the object to which the unique prompt is set (in the case of FIG. 3B, the creatures in the aquarium or the aquarium).

[0051] The location information of the object acquired by the acquisition unit 294 together with the unique prompt is preferably information specifying the distance between the object and the user (specifically, the smart device 14 held by the user). For example, the distance between the object and the user may be estimated from the radio wave intensity of a beacon installed on the object or a data communication device held by the object. Alternatively, the distance may be calculated from the coordinate information of the object registered in advance in relation to the object and the positioning result of the GPS 43 of the smart device 14.

[0052] Returning to the explanation of Fig. 4, the processing unit 296 performs a specification process using the data generation model 58 as an example of a sentence generation model. Specifically, the processing unit 296 first determines whether a predetermined trigger condition is satisfied. More specifically, the trigger condition is the receipt of a question from the user by voice or text (e.g., "Can I keep this fish at home?").

[0053] Then, when the trigger condition is satisfied, the processing unit 296 inputs an input sentence, more specifically, text data, which indicates an instruction for obtaining data for a specific process, to the data generation model 58, and acquires a processing result based on the output of the data generation model 58. More specifically, the processing unit 296 generates text data to be input to the data generation model 58 from information obtained from the user input accepted by the input unit 292 and the unique prompt acquired by the acquisition unit 294, or the unique prompt and the user prompt. Then, the processing unit 296 inputs the generated text data to the data generation model 58, and acquires the processing result.

[0054] The processing unit 296 may estimate the user's emotion using the emotion identification model 60, and perform identification processing using the user's emotion and the data generation model 58. In this case, the text data generated by the processing unit 296 is generated by also using a prompt corresponding to the user's emotion estimated by the emotion identification model 60. An example of a method of identifying the user's emotion using the emotion identification model 60 will be described later.

[0055] The output unit 298 outputs the result of the specific processing. A specific output method includes, for example, transmitting the result of the specific processing to the smart device 14, and notifying the user of the result of the specific processing in the smart device 14 via the output device 40 or the like.

[0056] When the result of the specific processing is notified to the user by the above-mentioned method, the user may perform a user input in response to the result. For example, when the microphone 38B acquires a voice indicating a user input in response to the result of the specific processing, the control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. Then, the data processing device 12 executes the specific processing again by the specific processing unit 290.

[0057] The data generation model 58 may be a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image may be input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0058] Next, the operation of the data processing system 10 will be described with reference to Fig. 5. Specifically, an example of the flow of a specific process will be described with reference to Fig. 5. The flow of the specific process shown in Fig. 5 is an example of a "data processing method" according to the technology of the present disclosure. This flow of the specific process can be performed by the processor 28 of the data processing device 12, and the processing by the processor can be realized by the processor executing a specific processing program 56.

[0059] When the data processing system 10 operates and the specific process is started, as shown in Fig. 5, the specific processing unit 290 first monitors whether the reception device 38 of the smart device 14 has received a user input (step S101). When the input unit 292 receives the user input received by the reception device 38 (Yes in step S101), the acquisition unit 294 of the specific processing unit 290 acquires a user prompt (step S102) and a unique prompt (step S103). The order of acquiring the above-mentioned two prompts is not particularly limited. Furthermore, if a user prompt is not set, step S102 may be omitted.

[0060] Once the acquisition unit 294 has acquired the user prompt and the specific prompt, the processing unit 296 then generates an input sentence to be input to the data generation model 58 from the information obtained from the user input and the acquired specific prompt, or the specific prompt and the user prompt (step S104). The generated input sentence is input to the data generation model 58, whereby a sentence corresponding to the input sentence is generated (step S105).

[0061] The sentence generated by the data generation model 58 is sent to the smart device 14 as a result of the specific processing, and is output by the output device 40 (step S106).

[0062] In the above-mentioned series of specific processes, some examples of input sentences input to the data generation model 58 and the corresponding output results are given below.

[0063] For example, a case will be described in which a user makes a voice input to the smart device 14 as a user input, "Can I keep this fish at home?" near a specific tank 70 in an aquarium as shown in FIG. 3A. When the voice input is acquired, the data processing device 12 operates the acquisition unit 294 using the voice input as a trigger to acquire a unique prompt associated with an object around the user and a user prompt. It is assumed that the acquired unique prompt is only the unique prompt transmitted from the beacon 75 installed in the tank 70, and that this unique prompt is "The fish kept in the tank are guppies. Their lifespan is 1 to 2 years. The appropriate environment for keeping them is 24 to 27 degrees." It is also assumed that the user prompt is "I want to keep a fish. The average indoor temperature is 25 degrees."

[0064] In the above case, the processing unit 296 generates a prompt as an input sentence, for example, "The user wants to keep a fish, and the average indoor temperature of the room where the fish will be kept is 25 degrees. Guppies are kept in an aquarium near the user, and their lifespan is 1 to 2 years, and the appropriate breeding environment is 24 to 27 degrees. The user asks, 'Can I keep this fish at home?' How should I respond?" and inputs it to the data generation model 58. When the data generation model 58 receives the above prompt, it can output text data, for example, "Yes. You can buy guppies at your home."

[0065] On the other hand, for example, a case will be described in which a user makes a voice input to the smart device 14 saying, "Can I keep this fish at home?" near a specific aquarium 70A in an aquarium in which multiple aquariums 70A to 70C are installed adjacent to each other as shown in FIG. 3B. When the voice input is acquired, the data processing device 12 uses the voice input as a trigger to operate the acquisition unit 294 and acquires a unique prompt and a user prompt associated with an object around the user. The acquired unique prompts are assumed to be three unique prompts transmitted from the beacons 75A to 75C installed in the three aquariums 70A to 70C. In addition, the content of each unique prompt is different. Specifically, for example, the unique prompt transmitted from beacon 75A is "The fish kept in the aquarium are guppies. Their lifespan is 1-2 years. The appropriate breeding environment is 24-27 degrees," the unique prompt transmitted from beacon 75B is "The fish kept in the aquarium are goldfish. Their lifespan is 7-8 years. The appropriate breeding environment is 20-22 degrees," and the unique prompt transmitted from beacon 75C is "The fish kept in the aquarium are clownfish. Their lifespan is 5-6 years. The appropriate breeding environment is 28-30 degrees." In addition, the user prompt is "I want to keep a fish. The average indoor temperature is 25 degrees."

[0066] Here, the acquisition unit 294 can further acquire position information of objects around the user. The acquired position information may be the distance between the user and the object. The position information acquired by the acquisition unit 294 can be associated with the three unique prompts transmitted from each of the beacons 75A to 75C, for example, in the form of a prompt added to the unique prompt. In the present embodiment, the acquired position information is used as a prompt format together with the unique prompt in the input sentence, but it is also possible to select a unique prompt to be used in the input sentence from among the multiple unique prompts acquired by the acquisition unit 294 based on the position information, for example. In this case, by using only the unique prompt selected based on the position information in the input sentence, it is possible to prevent information less related to the user input from being included in the input sentence.

[0067] In the above cases, the processing unit 296 generates a prompt as an input sentence, for example, "The user wants to keep a fish, and the average indoor temperature of the room where the fish will be kept is 25 degrees. The aquarium 1 meter away from the user keeps guppies, which have a lifespan of 1 to 2 years, and the appropriate breeding environment is 24 to 27 degrees. The aquarium 2.5 meters away from the user keeps goldfish, which have a lifespan of 7 to 8 years, and the appropriate breeding environment is 20 to 22 degrees. The aquarium 3 meters away from the user keeps clownfish, which have a lifespan of 5 to 6 years, and the appropriate breeding environment is 28 to 30 degrees. The user asks, 'Can I keep this fish at home?' How should I answer?" and inputs the generated prompt into the data generation model 58.

[0068] At this time, it is important to note that the prompt of the input sentence contains information on the distance to the aquarium in association with the specific prompt. In this way, by adding location information in association with each specific prompt, it is expected that the accuracy of the answer of the data generation model 58 will be improved. In the above example, it is possible to accurately estimate which aquarium the "this fish" included in the user input belongs to. When the above-mentioned prompt is input, the data generation model 58 can output text data such as "Yes, you can buy guppies at your home."

[0069] In the above two examples, specific examples of specific processing were explained using the states shown in Figures 3A and 3B as examples, but the specific processing can also be performed in other situations and various additional effects can be obtained.

[0070] For example, when a user inputs "What is attractive about this healthy ramen?" as a question related to a specific item, for example, cup noodles sold in a store, the identification processing unit 290 operates as follows. That is, when the user input is acquired, the data processing device 12 first operates the acquisition unit 294 using the user input as a trigger to acquire a unique prompt associated with an object around the user and a user prompt. The unique prompt acquired here can be identified by, for example, the user reading a two-dimensional code attached to the cup noodles with the smart device 14. It is assumed that the unique prompt identified in this way is "Product name: healthy ramen, Features: 50% calories of normal cup noodles, Calories: 200 kcal, Ingredients: lettuce, egg, green onion". It is also assumed that the user prompt is "on a diet".

[0071] In the above case, the processing unit 296 generates a prompt as an input sentence, for example, "The user is on a diet. There is a cup noodle called Healthy Ramen near the user, and the calorie information of this cup noodle is 200 kcal, which is 50% of the calorie information of normal cup noodles, and the ingredients include lettuce, egg, and green onion. The user asks, 'What is the appeal of this healthy ramen?' How should I answer that?" and inputs it to the data generation model 58. When the above-mentioned prompt is input, the data generation model 58 can output text data such as, for example, "The calorie information of this healthy ramen is 50% of that of normal ramen, and the ingredients include lettuce."

[0072] As described above, when an object related to a user input is a product such as instant noodles, it may not have been registered as a parameter in advance in the data generation model 58 depending on the time of release. Therefore, in the conventional method, there is a risk that the answer of the data generation model 58 is generated based on inaccurate information. In addition, re-learning to register the instant noodle information in the data generation model 58 is costly, and re-learning every time a new product is released is also time-consuming. However, if the information of the object is added to the input sentence as a unique prompt as in the present embodiment, the accurate information of the object can be reflected in the answer of the data generation model 58 without re-learning the data generation model 58. Therefore, it is expected that an answer based on accurate information of the object will be generated.

[0073] Also, for example, when a user is viewing a rental property such as an apartment or condominium in a specific location, and the user inputs a voice input of "Does this apartment meet my requirements?", the identification processing unit 290 operates as follows. That is, when the user input is acquired, the data processing device 12 first operates the acquisition unit 294 using the user input as a trigger to acquire a unique prompt associated with an object around the user and a user prompt. The unique prompt acquired here can identify the corresponding unique prompt in the database 24 by identifying a rental property as an object linked to a position adjacent to a position measured by the GPS 43 of the smart device 14, for example. The unique prompt identified in this way is assumed to be "rent: 112,000 yen, floor plan: 1LDK". Also, the user prompt is assumed to be "looking for a rental property, hoping for a rent of about 110,000 yen, hoping for a 2LDK floor plan".

[0074] In the above cases, the processing unit 296 generates a prompt as an input sentence, for example, "The user is looking for a rental property with a rent of about 110,000 yen and a layout of 2LDK. There is a rental property with a rent of 112,000 yen and a layout of 1LDK near the user. The user asks, 'Does this apartment meet my requirements?' How should I respond?" and inputs this into the data generation model 58. When the data generation model 58 receives the above-mentioned prompt, it can output text data such as, "The monetary cost is not a problem. However, the layout of the room seems to be a little smaller than desired."

[0075] As described above, when the object related to the user input is a property, it is usually necessary for a real estate company employee or the like to accompany the viewing and provide an explanation. However, by adding object information to the input sentence as a unique prompt as in the present embodiment, the user can obtain an answer based on accurate information, and can obtain the necessary information without having a real estate company employee or the like provide an explanation. In addition, the real estate company does not need to prepare personnel to explain the property, which can reduce personnel.

[0076] As described above, in the data processing device 12, data processing method, and program according to the present embodiment, the input sentence input to the data generation model 58 includes not only the user input but also at least the unique prompt associated with the object around the user. Therefore, it is possible to provide the data generation model 58 with information related to the user input, and it is expected that the accuracy of the specific processing will be improved. Furthermore, it is possible to more appropriately respond to the user's question. In addition, it is expected that the accuracy of the specific processing will be further improved by adding location information to the unique prompt or using a user prompt to generate the input sentence.

[0077] In addition, in this embodiment, by using unique prompts in the input sentences, re-learning to incorporate the latest information into the data generation model 58 is not necessary, and answer accuracy can be improved at low cost.

[0078] The emotion identification model 60 may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 60 may determine the emotion of the user according to an emotion map (see FIG. 6A), which is a specific mapping.

[0079] FIG. 6A is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0080] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0081] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0082] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0083] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0084] The emotion identification model 60 inputs user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. In addition, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 6B. FIG. 6B shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0085] The emotion value identified by the emotion identification model 60 can be used in the input sentence. When used in the input sentence, the emotion value may be changed to a prompt and made a part of the input sentence. Therefore, the emotion identification model 60 is an example of an "emotion determination unit" according to the technology of the present disclosure. In this way, by using the emotion value in the input sentence, the result of the identification process can reflect the emotion of the user.

[0086] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0087] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0088] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0089] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0090] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0091] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0092] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0093] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0094] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0095] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0096] 10 Data Processing System 12 Data Processing Device 14 Smart Devices 58 Data generation model (an example of a text generation model) 60 Emotion identification model (an example of the emotion determination part) 290 Specific Processing Department 292 Input section 294 Acquisition Department 296 Processing section 298 Output section

Claims

1. an input unit for accepting user input; An acquisition unit for acquiring unique prompts associated with objects around a user; A processing unit that performs a specific process using a sentence generation model that generates a sentence according to an input sentence; an output unit that outputs a result of the specific processing, The processing unit inputs information obtained from the user input and the unique prompt acquired by the acquisition unit into the sentence generation model as the input sentence. Data processing device.

2. The acquiring unit further acquires a user prompt associated with a user who performs the user input; The processing unit inputs the input sentence, the input sentence further including the user prompt, to the sentence generation model.

2. A data processing apparatus according to claim 1.

3. The acquisition unit further acquires position information of the object.

2. A data processing apparatus according to claim 1.

4. Further comprising an emotion determining unit for determining an emotion of a user, The processing unit inputs the input sentence further including the emotion of the user into the sentence generation model.

2. A data processing apparatus according to claim 1.

5. accepting user input; obtaining a unique prompt associated with an object in the user's surroundings; generating an input sentence from information obtained from a user input and the acquired unique prompt, to be input to a sentence generation model that generates a sentence corresponding to the input sentence; inputting the generated input sentence into the sentence generation model and executing a specific process; and outputting a result of the specific processing. Data processing methods.

6. At least one processor of a computer constituting a data processing device A process for accepting user input; obtaining a specific prompt associated with an object in the user's surroundings; generating an input sentence from information obtained from a user input and the acquired unique prompt, to be input to a sentence generation model that generates a sentence corresponding to the input sentence; inputting the generated input sentence into the sentence generation model and executing a specific process; program.

Images