Information processing device, information processing method, and program

Abstract

An information processing device according to the present invention comprises a text generation unit and an input processing unit. The text generation unit generates at least one of a first text related to a state of a moving body, a second text related to a driver of the moving body, a third text related to a passenger of the moving body, and a fourth text related to an operation state of a device mounted on the moving body. The input processing unit generates instruction information for causing a large-scale language model to generate response information related to the load on a driver of the moving body, the instruction information including at least one of the first text, the second text, the third text, and the fourth text, and executes processing for inputting the instruction information into the large-scale language model.

Description

Information Processing Apparatus, Information Processing Method, and Program

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

[0002] In recent years, technologies have been developed to support a driver riding in a moving object using information processing technologies. For example, Patent Document 1 discloses a device that calculates a driving load. This device acquires vehicle exterior information indicating the situation outside the vehicle and also acquires state information indicating the state of the driver of this vehicle. Then, based on the acquired vehicle exterior information and state information, this device calculates the driving load of the driver. Patent Document 1 discloses using artificial intelligence such as a neural network for the calculation of the driving load.

[0003] Japanese Unexamined Patent Application Publication No. 2019 - 53436

[0004] For estimating the load applied to the driver, it is required to improve the accuracy while suppressing an increase in the amount of calculation. An example of the object of the present invention is to improve the accuracy while suppressing an increase in the amount of calculation in estimating the load applied to the driver.

[0005] The invention according to claim 1 includes: a text generation unit that generates at least one of a first text regarding the situation of a moving object, a second text regarding the driver of the moving object, a third text regarding a passenger of the moving object, and a fourth text regarding the operation status of equipment mounted on the moving object; and an input processing unit that generates instruction information for causing a large - language model to generate response information regarding the load of the driver of the moving object, the instruction information including the at least one of the first text, the second text, the third text, and the fourth text, and executes a process for inputting the instruction information into the large - language model.

[0006] The invention described in claim 14 is an information processing method in which a computer generates at least one of a first text relating to the status of a mobile body, a second text relating to the driver of the mobile body, a third text relating to a passenger of the mobile body, and a fourth text relating to the operating status of equipment mounted on the mobile body, and generates instruction information for a large-scale language model to generate response information relating to the load of the driver of the mobile body, the instruction information comprising at least one of the first text, the second text, the third text, and the fourth text, and performs processing to input the instruction information into the large-scale language model.

[0007] The invention described in claim 15 is a program that provides a computer with: a text generation unit that generates at least one of a first text relating to the status of a mobile body, a second text relating to the driver of the mobile body, a third text relating to a passenger of the mobile body, and a fourth text relating to the operating status of equipment mounted on the mobile body; and an input processing unit that generates instruction information for a large-scale language model to generate response information relating to the load of the driver of the mobile body, the instruction information including at least one of the first text, the second text, the third text, and the fourth text, and performs processing to input the instruction information into the large-scale language model.

[0008] This diagram illustrates the operating environment and functional configuration of the information processing apparatus according to the embodiment. It shows items that may be included in the first text, the second text, the third text, and the fourth text. It also shows an example of instruction information, an example of the hardware configuration of the information processing apparatus, and a flowchart illustrating an example of processing performed by the information processing apparatus.

[0009] Embodiments of the present invention will be described below with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted as appropriate.

[0010] (First Embodiment) Figure 1 is a diagram illustrating the usage environment and functional configuration of the information processing device 10 according to this embodiment. The information processing device 10 performs processing to estimate the load of the driver riding in the mobile body 20. The information processing device 10 also performs processing to support the driver using the results of this processing. One example of this processing is controlling a target device 220 mounted on the mobile body 20. The mobile body 20 is, for example, a car or a bus, but is not limited to these.

[0011] The information processing device 10 may be mounted on the mobile device 20 or located outside the mobile device 20. For example, the information processing device 10 can be incorporated as a function of a car navigation system or an in-vehicle device. The information processing device 10 may also communicate with an external device, such as a server, as needed. One example of such an external device is a device that performs route suggestion and route search and stores map information. This map information also contains information about facilities such as stores. Furthermore, the information processing device 10 may be realized by a combination of a device mounted on the mobile device 20 and a device outside the mobile device 20.

[0012] The information processing device 10 is used together with the sensor 210, the target device 220, and the model device 30. The sensor 210 and the target device 220 move together with the mobile body 20. The sensor 210 and the target device 220 are, for example, incorporated into the mobile body 20, but may also be incorporated into a portable communication device held by the driver. Alternatively, one of the sensor 210 and the target device 220 may be incorporated into the mobile body 20, and the other into the communication device. In addition, the mobile body 20 may have multiple sensors 210 and multiple target devices 220 incorporated into it. In this case, some of the multiple sensors 210 may be incorporated into the mobile body 20, and the remaining sensors 210 may be incorporated into the communication device. The same applies to the multiple target devices 220.

[0013] Sensor 210 repeatedly generates data used when generating information about the driver's load and transmits the generated data to the information processing device 10. Hereinafter, the information generated by sensor 210 will be referred to as sensor information. Sensor 210 is, for example, at least one of the following. At least some of these examples may also be handled by the control unit that controls various devices of the mobile body 20. - Speedometer of the mobile body 20 - Acceleration sensor of the mobile body 20 - Sensor that detects the open / closed state of the doors and windows of the mobile body 20 - Sensor that detects the illuminated state of the lights (including fog lamps) for the front lighting of the mobile body 20 - Sensor that detects operations performed on the steering wheel of the mobile body 20 - Sensor that detects the current position of the mobile body 20 (e.g., GPS) - Sensor that acquires or detects information indicating at least one of the type of road the mobile body 20 is currently traveling on and traffic conditions (e.g., congestion status such as whether there is a traffic jam or whether there is construction) (e.g., navigation device) - Imaging device that photographs at least one of the surroundings of the mobile body 20, such as the front, rear, and side. - Imaging device that photographs the passenger space of the mobile body 20. - A sensor that detects the driver's biometric information. The biometric information includes, for example, heart rate, blood pressure, respiratory rate, and sweating status. This sensor may be attached to the driver. - A clock - A thermometer that detects the ambient temperature around the mobile body 20 - A thermometer that detects the ambient temperature in the passenger compartment of the mobile body 20 - A microphone that detects the sound in the passenger compartment of the mobile body 20 - A microphone that detects the ambient sound around the mobile body 20

[0014] The target device 220 is, for example, at least one of the following: • A device that controls the opening and closing state of the windows of the mobile body 20; • Air conditioning equipment with temperature control function; • Navigation device; • Audio device; • Interactive device. It may also have the function of controlling at least one of the navigation device and the audio device. • Lights (including fog lamps) for the forward illumination of the mobile body 20.

[0015] When the information processing device 10 performs processing to estimate the load on the driver, it may also use data indicating the operating status of the target device 220. Hereinafter, this information will be referred to as operating status data. The operating status data is generated, for example, by the target device 220. The target device 220 repeatedly generates the operating status data and transmits it to the information processing device 10.

[0016] The model device 30 performs processing using a model such as a machine learning model. The model used here is, for example, at least one of a Large Language Model (LLM) and a Small Language Model (SLM), but may also be other foundation models, or hybrid models that combine these with other classifiers (e.g., logistic regression and gradient boosting). The information processing device 10 generates instruction information, such as a prompt, to be input to the model device 30, and transmits this instruction information to the model device 30. The model device 30 inputs this instruction information into the Large Language Model, obtains the response information generated by the Large Language Model, and transmits it to the information processing device 10.

[0017] The information processing device 10 then performs processing using the response information. One example of this processing is to cause the target device 220 to perform a predetermined process. A specific example of this predetermined process will be described later.

[0018] The information processing device 10 may also serve as the model device 30.

[0019] The information processing device 10 includes a text generation unit 110, an input processing unit 120, and a response processing unit 130, and can utilize a storage unit 140. The storage unit 140 stores various types of information used by the information processing device 10. The storage unit 140 may be part of the information processing device 10 or may be located outside the information processing device 10.

[0020] The text generation unit 110 generates at least one of the following: a first text relating to the status of the mobile body, a second text relating to the driver of the mobile body, a third text relating to the passengers of the mobile body, and a fourth text relating to the operating status of the equipment mounted on the mobile body. For example, the text generation unit 110 acquires sensor information and processes this sensor information to generate at least one of the first text, second text, third text, and fourth text. Here, an example of equipment mounted on the mobile body is the target device 220.

[0021] As a first example, the text generation unit 110 generates at least one of a first text, a second text, a third text, and a fourth text by processing sensor information using a machine learning model. This machine learning model may be owned by the information processing device 10 or by an external device.

[0022] As a second example, if the sensor information includes a string (for example, a number), the text generation unit 110 makes at least a part of this string at least a part of the first text. The second example can also be used when generating at least one of the second text, the third text, and the fourth text.

[0023] As a third example, if the sensor information indicates the operation of a device mounted on the mobile body 20, the text generation unit 110 generates a fourth text by processing the sensor information according to predetermined rules.

[0024] As a fourth example, if the sensor information includes an image (e.g., a video), the text generation unit 110 processes this image to detect at least one of the vehicle's occupants, such as the driver and a passenger, and processes this detection result according to predetermined rules to generate at least one of the first text, second text, third text, and fourth text.

[0025] As a fifth example, if the sensor information includes voices based on the occupants of the vehicle, for example, voices based on the speech of at least one of the driver and passengers, the text generation unit 110 generates at least one of the first text, second text, third text, and fourth text by converting this voice into text, or by processing this text according to, for example, a machine learning model or predetermined rules.

[0026] The input processing unit 120 generates instruction information to be input to the large-scale language model used by the model device 30. This instruction information is for causing the large-scale language model to generate response information regarding the load of the mobile device's driver, and includes text generated by the text generation unit 110 from among the first text, second text, third text, and fourth text, but may also include at least one of structured data (e.g., JSON), embedding vectors, and numerical sequences.

[0027] The instruction information may further include indicating the load in multiple stages. In this case, the instruction information preferably includes the number of stages. This number is, for example, 2 to 5, but is not limited thereto. This number is, for example, pre-stored in the storage unit 140. The instruction information also preferably includes information indicating at least one criterion for the multiple stages. This information preferably indicates, for example, the criteria for classifying into each stage in text. This information is, for example, pre-stored in the storage unit 140.

[0028] The levels of difficulty are indicated by the following four stages, for example, 1 to 4 (the higher the number, the greater the difficulty): 1: You have enough capacity to have a difficult conversation. 2: You can have a normal conversation, but not a difficult one. 3: You can give simple answers, but not have a normal conversation. 4: You cannot have a conversation.

[0029] Furthermore, item "4" above may be further divided into two stages.

[0030] If the instruction information includes "indicating the load in multiple stages," the large-scale language model used by the model device 30 includes information in the response information indicating which stage the driver load is at. Furthermore, if the instruction information includes information indicating at least one criterion for multiple stages, the large-scale language model used by the model device 30 determines which stage the driver load is at according to this information.

[0031] The input processing unit 120 then performs processing to input the instruction information into the large-scale language model. One example of this processing is to send the instruction information to the model device 30. However, if the information processing device 10 also functions as the model device 30, this processing is to input the instruction information into the large-scale language model.

[0032] The text generation unit 110 preferably generates two or more, preferably all, of the first text, second text, third text, and fourth text. Doing so increases the accuracy of the operational load included in the response information generated by the large-scale language model.

[0033] The response processing unit 130 performs processing using the response information. One example of this processing is to cause the target device 220 to perform a predetermined process. The predetermined process is, for example, at least one of the following:

[0034] - Control of the amount of information provided according to the driver load and optimization of interactive devices. When the driver load is high, the response processing unit 130 reduces the amount of information that the target device 220, such as an interactive device, provides to the driver compared to when the driver load is low. By adjusting the amount of information that the target device 220 provides to the driver, the stability of operation by the driver is improved.

[0035] Specific examples of "information quantity control" are as follows: • For audio and text information, change to shorter instructions or summaries when the load is high. • For visual information, display only the minimum necessary information on the screen when the load is high. • Adjust the timing of information delivery. For example, do not provide less urgent information when the load is high, and provide it when the load is low.

[0036] If the target device 220 is an interactive information device, further specific examples of "controlling the amount of information" include: - Adjusting the format of questions. For example, when the load is high, change to closed questions that can be answered with "yes / no". - Adjusting the rhythm of the dialogue. For example, when the load is high, slow down the pace of the dialogue compared to when the load is low. - Providing information according to its importance. For example, navigation information is always provided, but music information is only provided when the load is low.

[0037] The response processing unit 130 may use the load levels included in the response information to cause the target device 220, which moves together with the mobile body 20, to perform a predetermined process. This is intended to adjust the load applied to the driver and improve the stability of the driver's operation. In particular, when the instruction information includes criteria for multiple levels, the load determination criteria by the large-scale language model become stable. Therefore, the response processing unit 130 can cause the target device 220 to perform a more appropriate process compared to when the instruction information does not include criteria for multiple levels.

[0038] The prescribed processing here includes, for example, at least one of the following: • Controlling at least one of the quantity and quality of information output according to the load level. This control may include not providing information when the load is high. • Adjusting the format of the questions according to the load level. • Adjusting the quantity of questions according to the load level. • Providing information appropriate to the load level.

[0039] The following will provide a detailed explanation of the case where the target device 220 provides route guidance and the load level is set to one of five levels, Lv1 to Lv5 (the higher the number, the greater the load).

[0040] First, when the load level is "Lv1 (very low)," the output information may consist of detailed information or multiple choices (open-ended questions, free input). A concrete example of the output information is, "We suggest three routes to your destination. Which one would you like to take?"

[0041] Also, when the load level is "Lv2 (low)", the information output is information presentation with key points condensed (simple open questions). A specific example of the information output is "Which is better, the shortest route to the destination or the route to avoid traffic jams?"

[0042] Also, when the load level is "Lv3 (medium)", the information output is about two options narrowed down (closed questions). A specific example of the information output is "Do you want to reroute to avoid traffic jams? Yes / No"

[0043] Also, when the load level is "Lv4 (high)", the information output is minimal information presentation (for example, presentation centered on confirmation). A specific example of the information output is "Rerouting to avoid traffic jams."

[0044] Also, when the load level is "Lv5 (very high)", the response processing unit 130 temporarily stops information presentation. Note that the response processing unit 130 may display text indicating that, for example, "Safety confirmation in progress. Guidance will resume later" on the display.

[0045] Also, when the target device 220 is a content output device that outputs music, the response processing unit 130 may cause the target device 220 to output music only when the load level is below the reference value.

[0046] FIG. 2 shows items that may be included in the first text, items that may be included in the second text, items that may be included in the third text, and items that may be included in the fourth text. Each of the first text to the third text also shows the specific content of these items.

[0047] The information included in the first text, that is, information regarding the situation of the moving object, may include at least one of the time zone, the current position of the moving object, the weather at the current position, the traffic information at the current position, information regarding the road at the current position, and information regarding the guidance information from the current position to the destination. More specifically, this information may include at least one of the following items. At least one of these pieces of information may be obtained from an external server, for example, a server storing weather information or a server storing road information. Also, the quality of the road can be determined, for example, from the magnitude of road noise. - Travel speed - Traffic information at the current position (e.g., traffic volume, degree of congestion (e.g., congestion, crowding, or smooth), presence or absence of regulations (e.g., speed limit, lane regulation, or road closure), presence or absence of accidents...) - Situation of other types of moving objects (e.g., many pedestrians, many bicycles...) - Time zone (e.g., commuting time zone, late at night...) - Road condition at the current position (e.g., under construction, frozen...) - Quality of the road at the current position (e.g., paved, unpaved...) - Type of the road at the current position (e.g., toll road, highway, general road...) - Surrounding situation of the current position (e.g., urban area, suburbs...) - Weather at the current position and its change - Temperature at the current position - Humidity at the current position - Lighting condition at the current position (e.g., daytime, nighttime...) - Guidance information from the current position to the destination (e.g., distance, type of road...) - Whether the current position is within the living area (e.g., familiar road, unfamiliar road...) - Information regarding failure (e.g., prone to failure, no problem...)

[0048] The information included in the first text may include at least one of the following. - Information regarding a second moving object existing around the moving object. For example, at least one of the behavior of the second moving object (e.g., vehicle) (e.g., presence or absence of aggressive driving or sudden cut-in), warning sounds emitted by the second moving object (e.g., horn and siren), and the jumping out of a person or animal as the second moving object. And an example of the first text in this example is "An accident has occurred ahead, and an urgent lane change is required."

[0049] This information is generated by processing the detection results of various sensors (cameras, LiDAR, microphones, etc.) mounted on the vehicle. The information processing device 10 may perform this processing, or a device other than the information processing device 10 may perform this processing.

[0050] The information included in the first text may include at least one of the following: • The purpose of the movement (e.g., commuting, work, picking up / dropping off someone, travel, etc.) • The time constraints of the movement (e.g., the amount of leeway before the time of arrival at the destination)

[0051] This information is obtained using, for example, destination information set in the route guidance system, schedule information of the people on board the vehicle, and voice input from the people on board the vehicle. The schedule information is managed, for example, by a schedule management application program installed on a portable communication device carried by the driver. The text generation unit 110 then obtains the schedule information from this communication device. An example of the first text in this example is, "This drive is for business purposes, and we need to arrive at the destination within 15 minutes."

[0052] The information included in the second text, namely the information about the driver of the vehicle, may include, for example, at least one of the following: the driver's biometric information, the driver's emotions, the time since the vehicle was started to be driven, information about breaks taken since the vehicle was started to be driven, and information about the driver's driving experience. More specifically, this information may include at least one of the following items: • Biometric information (e.g., fatigue, vital information such as heart rate, etc.) • Speech content (e.g., it's getting dark, etc.) • Emotions (e.g., happy, irritated, etc.) • Time since the start of driving to the present, whether or not breaks have been taken, and the time since the last break to the present

[0053] The information included in the second text may include the following: information about at least one of the driver's driving experience and tendencies (e.g., length of driving experience (e.g., 10 years), past stress levels while driving (e.g., increased stress in traffic jams), past accident history or near-miss experiences, whether the road is familiar or unfamiliar (familiarity / unfamiliarity)...).

[0054] This information is generated using, for example, driving history data, driver profiles, and map display history from the route guidance system. Of these, the driving history data and map display history are stored, for example, in the route guidance system. The text generation unit 110 then acquires this information from the route guidance system. The text generation unit 110 may also acquire the driver profile from, for example, a portable communication device carried by the driver. An example of the second text in this example is, "This is a road I've never traveled before, and I have a history of increased stress in similar situations in the past."

[0055] The information included in the second text may include: • Information regarding tasks to be performed on the moving object (e.g., telephone calls, deliveries, reports, etc.)

[0056] The text generation unit 110 acquires this information from at least one of the schedule information of the person on board the mobile vehicle and voice input from the person on board the mobile vehicle.

[0057] The information contained in the third text, i.e., information about passengers in the vehicle, may include at least one of the following items. This information is identified, for example, using images (sometimes video) taken inside the vehicle, and audio information including conversations that took place inside the vehicle. • Number of people • Relationship to the driver (e.g., family, friend, boss, customer, etc.) • Attributes (e.g., age, gender, etc.) • Behavior and actions inside the vehicle (e.g., talking to the driver, etc.)

[0058] The number of passengers can affect the driver's workload because, for example, having multiple passengers can increase noise and conversation within the vehicle, potentially raising the driver's workload. Similarly, the behavior of passengers can affect the driver's workload because, for example, frequent talking or disruptive behavior can increase the driver's workload. Furthermore, the attributes of passengers can affect the driver's workload because, for example, having infants or elderly passengers can increase the driver's attention span and mental burden.

[0059] The number of passengers, their attributes, behavior, actions, and their relationship to the driver are identified, for example, by processing at least one of the following: images generated by a camera that photographs the inside of the mobile device 20, or audio data generated by a microphone installed inside the mobile device 20. The information processing device 10 may perform this identification process, or an external device may perform this identification process. A machine learning model may also be used in this identification process. The relationship to the driver may be input to the information processing device 10, for example, by a person inside the mobile device 20 operating an input device located inside the mobile device 20.

[0060] And an example of the third type of text is at least one of the following: • "There are three passengers." • "The passengers are frequently talking to the driver." • "The passengers are making noise in the back seat." • "The passengers are one elderly woman and one infant." • "The passenger is the driver's supervisor." • "The passenger is the driver's child." • "The passenger appears angry and is speaking harshly." • "The passenger is tense and speaks little."

[0061] The fourth text, as described above, is information regarding the operating status of equipment mounted on the mobile device. The text generation unit 110 obtains this information, for example, from the equipment. The equipment here is, for example, at least one of the following: • Route guidance device (e.g., a device with a car navigation function) • Content output device (e.g., a device that controls at least one of a display and a speaker, and outputs at least one of video and music) • Air conditioning equipment (e.g., a device with a temperature control function; it may also have a humidity control function) • Portable communication device (e.g., a so-called smartphone, tablet terminal, etc.)

[0062] If the above device is a portable communication device, the fourth text may indicate at least one of the incoming calls, messages, and alerts received by the communication device. The text generation unit 110 may obtain this information from the communication device as described above, or it may generate it by voice recognition of the output from a microphone installed in the mobile body 20. An example of the fourth text in this example is, "Three notifications have arrived on your smartphone while driving, one of which is an urgent business contact."

[0063] Furthermore, the information included in the fourth text may include the following items: • The type of equipment in operation and its operating status (e.g., the car navigation system is providing route guidance, the audio system is functioning as a radio, etc.)

[0064] An example of the instruction information generated by the input processing unit 120 is as follows: The following is information regarding the current driving situation. Based on this, please estimate the driver's workload level on a 5-point scale (1: very low to 5: very high). 'Status of the vehicle (Text 1)' ・Current location: Shinjuku-ku, Tokyo ・Time: 6:30 PM (evening rush hour) ・Weather: Rain ・Traffic information: Congested, many traffic lights and frequent stops ・Road conditions: Paved road, poor visibility 'Driver's information (Text 2)' ・Biometric information: Slightly elevated heart rate, feeling fatigued ・Emotion: Feeling anxious ・Time since starting to drive: 1 hour 15 minutes ・Break: None ・Driving experience: 10+ years 'Information of passengers (Text 3)' ・Number of people: 2 (wife and infant) ・Attributes: Wife is a woman in her 30s, infant is 3 years old ・Relationship: Family ・Behavior: Infant is making noise in the back seat. Wife is quietly using her smartphone. ・Emotional state: Infant is excited, wife is calm 'Operation status of onboard equipment (Text 4)' Based on the following information: • Navigation: Providing route guidance to the destination. • Audio: Playing music for infants. • Air conditioning: Operating in automatic adjustment mode. Please indicate the driver's load level numerically and briefly explain the reason.

[0065] As shown in the example of instruction information above, the text generation unit 110 generates at least two texts that are included in any of the first text, second text, third text, and fourth text, and the input processing unit 120 may include these at least two texts in instruction information, such as a prompt. In this case, the order of these at least two texts may change the response information output by the large-scale language model. Therefore, it is preferable for the input processing unit 120 to arrange these at least two texts in a predetermined order and include them in the instruction information.

[0066] Here, at least two texts may belong to different classifications among the first text, second text, third text, and fourth text, or any of the texts may belong to the same classification as the other texts.

[0067] The input processing unit 120 may set a predetermined order using a predetermined priority order. The predetermined priority order may be set in units of the above-mentioned classification, namely the first text, second text, third text, and fourth text. For example, the storage unit 140 stores information indicating the priority order among the first text, second text, third text, and fourth text. The input processing unit 120 then uses this information to determine the predetermined order.

[0068] The input processing unit 120 may also set a predetermined order using information contained in the text. This information includes, for example, biometric information, time of day, and the driver's emotions, as listed as specific examples for the first text, second text, third text, and fourth text. For example, the memory unit 140 stores information indicating the priority among multiple pieces of information. The information stored here is set such that, for example, the more impact it has on the driving load, the higher its priority. The input processing unit 120 then uses this information to determine the predetermined order described above. For example, the memory unit 140 stores data in a table format showing an impact score for each type of text (e.g., a score of 1 for sunny weather, 5 for rain, 10 for thunderstorms, etc.). The input processing unit 120 then determines the predetermined order described above such that the higher the score, the higher the priority.

[0069] The input processing unit 120 may also suppress variations in the response information output by the large-scale language model in the following manner.

[0070] - Ensemble learning / majority voting is used. For example, the input processing unit 120 inputs instruction information to multiple large-scale language models and obtains response information from each of these multiple large-scale language models. The input processing unit 120 may obtain multiple response information by inputting the same instruction information to the same large-scale language model multiple times. The input processing unit 120 then generates response information to be used by the response processing unit 130 by statistically processing (e.g., majority voting) these multiple response information.

[0071] - Few-Shot Learning / Example Prompt Combination For example, the input processing unit 120 prepares one or more combinations of input examples and expected response examples, and trains the large-scale language model to respond. As an example, the input processing unit 120 trains the large-scale language model to respond based on level 4 if the location is in Tokyo and the weather is rainy, and raises to level 5 depending on other factors.

[0072] - Standardization of output format: For example, the input processing unit 120 includes information in the instruction information that specifies at least a part of the output format. The output format specified here may be, for example, a table or a bulleted list.

[0073] Figure 3 shows a first example of instruction information generated by the input processing unit 120. This instruction information includes a first text, i.e., text indicating the "status of the moving object," a second text, i.e., text relating to the "driver," a third text, i.e., text relating to the "passengers," and a fourth text, i.e., text relating to the "status of the equipment." The instruction information shown in Figure 3 further includes information indicating the criteria for each stage of the load information.

[0074] Figure 4 shows an example of the hardware configuration of the information processing device 10. The information processing device 10 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0075] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060 to send and receive data to and from each other. However, the method of connecting the processor 1020 and the other components to each other is not limited to bus connection.

[0076] Processor 1020 is a processor implemented using components such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).

[0077] Memory 1030 is a main memory device implemented as RAM (Random Access Memory), etc.

[0078] The storage device 1040 is an auxiliary storage device implemented as a removable media such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or memory card, or as ROM (Read Only Memory). The storage device 1040 stores program modules that implement each function of the information processing device 10 (for example, the text generation unit 110, the input processing unit 120, and the response processing unit 130). The processor 1020 reads these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 may also function as a storage unit 140.

[0079] The input / output interface 1050 is an interface for connecting the information processing device 10 with various input / output devices. For example, the information processing device 10 may communicate with at least one of the sensor 210 and the target device 220 via the input / output interface 1050.

[0080] The network interface 1060 is an interface for connecting the information processing device 10 to a network. This network may be, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method by which the network interface 1060 connects to the network may be wireless or wired. The information processing device 10 may communicate with at least one of the sensor 210 and the target device 220 via the network interface 1060.

[0081] Figure 5 is a flowchart illustrating an example of the processing performed by the information processing device 10. The information processing device 10 repeatedly performs the processing shown in this figure. First, the information processing device 10 acquires sensor information from the mobile body 20 (step S10). Then, the text generation unit 110 uses this sensor information to generate at least one of the first text, second text, third text, and fourth text (step S20). Then, the input processing unit 120 generates instruction information and transmits this instruction information to the model device 30 (step S30).

[0082] The model device 30 inputs instruction information into a large-scale language model and obtains response information from this large-scale language model. The model device 30 then transmits this response information to the information processing device 10. The response processing unit 130 of the information processing device 10 obtains this response information (step S40). The response processing unit 130 then uses this response information to perform predetermined processing (step S50).

[0083] In this way, the information processing device 10 generates at least one of the following: a first text concerning the status of the mobile body, a second text concerning the driver of the mobile body, a third text concerning the passengers of the mobile body, and a fourth text concerning the operating status of the equipment mounted on the mobile body, and generates instruction information including the generated text. The information processing device 10 then inputs this instruction information into a large-scale language model to obtain response information regarding the driver's load. Therefore, by using the information processing device 10, the driver's load can be estimated with high accuracy, and the amount of computation required for this estimation can be reduced.

[0084] The embodiments of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted.

[0085] Furthermore, while the flowcharts used in the above description show multiple steps (processes) in sequence, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content. Also, the above embodiments can be combined to the extent that their contents do not conflict.

[0086] This application claims priority based on Japanese Patent Application No. 2024-172085, filed on 1 October 2024, and priority based on International Application PCT / JP2025 / 008763, filed on 10 March 2025, and incorporates all disclosures thereof herein.

[0087] 10 Information processing device 20 Mobile device 30 Model device 110 Text generation unit 120 Input processing unit 130 Response processing unit 140 Storage unit 210 Sensor 220 Target device

Claims

1. An information processing device comprising: a text generation unit that generates at least one of a first text relating to the status of a mobile body, a second text relating to the driver of the mobile body, a third text relating to a passenger of the mobile body, and a fourth text relating to the operating status of equipment mounted on the mobile body; and an input processing unit that generates instruction information for a large-scale language model to generate response information relating to the load of the driver of the mobile body, the instruction information comprising at least one of the first text, the second text, the third text, and the fourth text, and performs processing for inputting the instruction information into the large-scale language model.

2. An information processing apparatus according to claim 1, wherein the instruction information further includes indicating the load in multiple stages in the response information.

3. An information processing apparatus according to claim 2, wherein the instruction information includes information indicating at least one criterion of the plurality of stages.

4. An information processing device according to any one of claims 1 to 3, wherein the first text includes at least one of the following: time of day, current location of the moving object, weather at the current location, traffic information at the current location, information about roads at the current location, information about second moving objects present around the moving object, purpose of movement of the moving object, time constraints of the moving object, and information about guidance information from the current location to the destination.

5. An information processing device according to any one of claims 1 to 4, wherein the second text includes at least one of the driver's biometric information, the driver's emotions, the time since the start of driving the mobile vehicle, information regarding breaks since the start of driving the mobile vehicle, information regarding work to be performed on the mobile vehicle, and information regarding the driver's driving experience.

6. An information processing apparatus according to any one of claims 1 to 5, wherein the text generation unit acquires sensor information generated by a sensor mounted on the moving body, and generates at least one of the first text, second text, third text, and fourth text using the sensor information.

7. An information processing apparatus according to any one of claims 1, 2, and 4 to 6, comprising a response processing unit that acquires the response information and causes a target device moving together with the moving body to perform a predetermined process using the response information.

8. An information processing apparatus according to claim 3, comprising a response processing unit that acquires the response information and causes a target device moving together with the moving body to perform a predetermined process using the stage of the load contained in the response information.

9. An information processing apparatus according to any one of claims 1 to 8, wherein the third text includes at least one of the number of passengers, the actions of the passengers, the state of the passengers, the attributes of the passengers, and the relationship between the driver and the passengers.

10. An information processing device according to any one of claims 1 to 9, wherein the device relating to the fourth text is at least one of a route guidance device, a content output device, an air conditioning device, and a portable communication device.

11. An information processing apparatus according to any one of claims 1 to 10, wherein the text generation unit generates at least two texts that are included in any of the first text, the second text, the third text, and the fourth text, and the input processing unit arranges the at least two texts in a predetermined order and includes them in a single instruction information.

12. An information processing device according to claim 11, wherein the input processing unit sets the predetermined order using a predetermined priority order.

13. An information processing device according to claim 11 or 12, wherein the input processing unit sets the predetermined order using information contained in each of the at least two texts.

14. An information processing method comprising: a computer generating at least one of a first text relating to the status of a mobile body, a second text relating to the driver of the mobile body, a third text relating to the passengers of the mobile body, and a fourth text relating to the operating status of equipment mounted on the mobile body; generating instruction information for a large-scale language model to generate response information relating to the load of the driver of the mobile body, the instruction information comprising at least one of the first text, the second text, the third text, and the fourth text; and performing a process to input the instruction information into the large-scale language model.

15. A program that provides a computer with: a text generation unit that generates at least one of a first text relating to the status of a mobile body, a second text relating to the driver of the mobile body, a third text relating to the passengers of the mobile body, and a fourth text relating to the operating status of equipment mounted on the mobile body; and an input processing unit that generates instruction information for a large-scale language model to generate response information relating to the load of the driver of the mobile body, the instruction information including at least one of the first text, the second text, the third text, and the fourth text, and performs processing to input the instruction information into the large-scale language model.

Images