Service provision system, edge device, server, service provision method, and program

Abstract

An information management unit acquires passenger information that relates to one or more passengers in a vehicle and is retained by a plurality of related devices, and status information that indicates the status of the related devices or the passengers, where the related devices include mobile devices of the passengers in the vehicle. An event detection unit (in S110 to S120) detects the occurrence of an event related to the vehicle or the passengers. When the occurrence of an event is detected by the event detection unit, a control instruction unit (in S130 to S160) generates a control instruction for providing a service suitable for the passenger information and the status information at the time of occurrence of the event by using a large-scale language model, and transmits the control instruction to at least one of the related devices.

Description

Service providing system, edge device, server, service providing method, and program Cross-reference to related applications

[0001] This international application claims the benefit of Japanese Patent Application No. 2024-162614, filed with the Japan Patent Office on September 19, 2024, and Japanese Patent Application No. 2025-105101, filed with the Japan Patent Office on June 20, 2025, the entire disclosures of which are incorporated herein by reference.

[0002] The present disclosure relates to a technology for providing services to vehicle users using a large language model.

[0003] Patent Document 1 below describes a technology related to a dialogue system using a large language model (hereinafter, LLM) that obtains an output using data stored in an automobile.

[0004] Japanese Unexamined Patent Application Publication No. 2024-43564

[0005] However, as a result of the inventors' detailed examination, the following problems have been found in the prior art.

[0006] That is, in the prior art, although it may be possible to make a proposal suitable for everyone according to the tendency extracted from the past data collected and stored for a large number of users, it is difficult to correctly understand the characteristics of individual users and the scenes in which the users are placed, and to make a proposal based on that situation. Also, if the information obtained from the vehicle is limited to grasping the scene in which the user is placed, the understanding of the scene will be limited.

[0007] In one aspect of the present disclosure, a technology is provided that realizes the provision of services based on the characteristics of users and the situations in which the users are placed.

[0008] One aspect of this disclosure is a service provision system comprising an information management unit, an event detection unit, and a control instruction unit. The information management unit is configured to acquire passenger information relating to passengers in a vehicle and status information indicating the status of the associated devices or passengers, with each of the vehicle and one or more passengers' mobile terminals being associated devices. The event detection unit is configured to detect the occurrence of events relating to the vehicle or passengers. When the event detection unit detects the occurrence of an event, the control instruction unit is configured to use a large-scale language model to generate control instructions for realizing a service that matches the passenger information and status information at the time of the event, and to transmit these instructions to at least one of the associated devices.

[0009] This configuration makes it possible to provide services that take into account the characteristics and circumstances of the vehicle and its passengers.

[0010] One aspect of this disclosure is an edge device mounted on a vehicle, comprising an information management unit, an event detection unit, and a control instruction unit.

[0011] One aspect of this disclosure is a server that communicates with a vehicle, comprising an information management unit, an event detection unit, and a control instruction unit.

[0012] One aspect of this disclosure is a service delivery method implemented by a vehicle or a server communicating with a vehicle. The service delivery method includes acquiring passenger information about the passengers of the vehicle and status information indicating the status of the associated devices or passengers, with the vehicle and one or more passengers' mobile terminals each being associated devices. The service delivery method includes detecting the occurrence of an event related to the vehicle or passengers. When the occurrence of an event is detected, the service delivery method includes using a large-scale language model to generate control instructions to realize a service that matches the passenger information and status information at the time of the event, and transmitting these instructions to at least one of the associated devices.

[0013] One aspect of this disclosure is a program that causes a computer to function as an information management unit, an event detection unit, and a control instruction unit.

[0014] This is a block diagram showing the device configuration of the service provision system. This is a block diagram showing the functional configuration of the service provision system. This is an explanatory diagram showing the information stored in the integrated memory unit. This is an explanatory diagram illustrating the contents of an instruction file. This is an explanatory diagram illustrating the contents of control information. This is a flowchart of the main process executed by the controller. This is a sequence diagram showing the system's operation when a baby has been left behind is detected. This is a sequence diagram showing the system's operation when a driver's lack of sleep is detected. This is an explanatory diagram showing the information stored in the local memory unit. This is a block diagram showing the device configuration of the service provision system of the second embodiment. This is a block diagram showing the functional configuration of a passenger's mobile terminal. This is a sequence diagram showing the operation when providing services in cooperation with a passenger's mobile terminal while the driver is in the vehicle. This is a sequence diagram showing the operation when providing services in cooperation with a passenger's mobile terminal after the driver has disembarked.

[0015] Embodiments of this disclosure will be described below with reference to the drawings.

[0016] [First Embodiment] [1-1. Overall Configuration] The service provision system 1 of the first embodiment shown in Figure 1 comprises a vehicle 2, a mobile terminal 30, and a cloud server 50. The vehicle 2, the mobile terminal 30, and the cloud server 50 may be connected to each other so as to be able to communicate with each other. The vehicle 2, the mobile terminal 30, and the cloud server 50 all correspond to the devices of this disclosure.

[0017] Vehicle 2 is equipped with an edge device 10.

[0018] The edge device 10 is connected to the vehicle's in-vehicle network and collects various vehicle data and performs various vehicle controls via the in-vehicle network. The edge device 10 can have any application program (hereinafter referred to as "app") installed on it. The edge device 10 may perform various functions by running the installed app.

[0019] The edge device 10 comprises a control unit 11, a communication unit 12, a storage unit 13, and a vehicle I / F unit 14.

[0020] The control unit 11 is an electronic control device centered around a computer equipped with a CPU 111, ROM 112, RAM 113, and the like.

[0021] The communication unit 12 performs wireless communication with the mobile terminal 30 and the cloud server 50. For communication with the mobile terminal 30, for example, Bluetooth may be used. Bluetooth is a registered trademark. For communication with the cloud server 50, for example, a wide-area communication network may be used.

[0022] The memory unit 13 stores at least information necessary to realize the in-vehicle large-scale language model (hereinafter referred to as the in-vehicle LLM), which will be described later. The LLM is a language model constructed using a very large dataset and deep learning technology, enabling fluent conversation similar to that of a human, and performing various processes using natural language with high accuracy.

[0023] The vehicle interface unit 14 is connected to various in-vehicle devices via the vehicle's in-vehicle network, etc., and acquires various information from these devices. The in-vehicle network may include CAN and Ethernet. CAN stands for Controller Area Network. Ethernet is a registered trademark. The in-vehicle devices connected to the vehicle interface unit 14 may include devices that are originally installed in the vehicle 2, as well as external devices that are added later. The in-vehicle devices may include sensors, cameras, sound devices, and display devices.

[0024] The mobile terminal 30 is, for example, a smartphone or tablet carried by the occupants of the vehicle 2. The mobile terminal 30 communicates with the edge device 10 and the cloud server 50. The mobile terminal 30 may run various application programs that are optionally installed.

[0025] The mobile terminal 30 comprises a control unit 31, a communication unit 32, and a storage unit 33.

[0026] The control unit 31 is an electronic control device centered around a computer equipped with a CPU 311, ROM 312, RAM 313, and the like.

[0027] The communication unit 32 performs wireless communication with the edge device 10 and the cloud server 50. For communication with the edge device 10, for example, Bluetooth may be used. For communication with the cloud server 50, for example, a wide-area communication network may be used.

[0028] The memory unit 33 stores at least the information necessary to realize the mobile LLM described later.

[0029] The cloud server 50 is connected to a wide-area communication network accessible by, for example, the edge device 10 and the mobile terminal 30.

[0030] The cloud server 50 comprises a control unit 51, a communication unit 52, and a storage unit 53.

[0031] The control unit 51 is an electronic control unit that is primarily composed of a computer equipped with a CPU 511, ROM 512, RAM 513, and the like.

[0032] The communication unit 52 performs wireless communication with the edge device 10 and the mobile terminal 30. For communication with the edge device 10 and the mobile terminal 30, a wide-area communication network may be used, for example.

[0033] The memory unit 53 stores at least the controller LLM, which will be described later.

[0034] The various computer functions of each of the control units 11, 31, and 51 described above are realized by the CPUs 111, 311, and 511 executing programs stored in non-transitional physical recording media. In this example, ROMs 112, 312, and 512 correspond to non-transitional physical recording media that store the programs. Furthermore, the execution of these programs executes methods corresponding to the programs. Note that some or all of the functions executed by the CPUs 111, 311, and 511 may be configured in hardware using one or more ICs. Also, the number of computers constituting each of the control units 11, 31, and 51 may be one or more.

[0035] [1-2. Functional Configuration] The various functions provided by the service provision system 1 are described below.

[0036] [1-2-1. Cloud Server] As shown in Figure 2, the cloud server 50 includes a controller 60.

[0037] The controller 60 comprises an information management unit 61, an integrated storage unit 62, and a controller AI unit 63. In this embodiment, a configuration in which the controller 60 is located on a cloud server 50 is described as an example, but the controller 60 may be located on, for example, a vehicle 2.

[0038] The information management unit 61 has the function of acquiring user information and status information from the vehicle 2 and the mobile terminal 30 and storing them in the integrated storage unit 62. User information is information about the occupants of the vehicle 2 (hereinafter referred to as vehicle users) and terminal users who are users of the mobile terminal 30, and is static information whose values ​​are unlikely to change and which will remain the same for a long period of time. In this embodiment, the vehicle user and the terminal user are the same, and will be simply referred to as users below. Status information is information about the status of the vehicle 2 and the mobile terminal 30, and is dynamic information whose values ​​change in the short term.

[0039] The information management unit 61 may have the function of acquiring general information from an external information source 70, and further, the function of storing the acquired information in the integrated storage unit 62. The information acquired from the external information source 70 may include information related to the determination of event occurrence and information related to control instructions for realizing services provided to the user. The information management unit 61 may also have the function of requesting user information and status information from the vehicle 2 and the mobile terminal 30 as needed.

[0040] As shown in Figure 3, the integrated storage unit 62 stores LLM parameters 621, profiles 622, and contexts 623. The integrated storage unit 62 may also store history information 624, instruction files 625, control information 626, and general information 627, etc.

[0041] The LLM parameters 621 are a set of parameters that constitute a large-scale language model (i.e., LLM). The LLM realized by the LLM parameters stored in the integrated memory unit 62 is called the controller LLM.

[0042] The profile 622 is user information collected by the information management unit 61 from the vehicle 2 and the mobile terminal 30. The context 623 is situation information collected by the information management unit 61 from the vehicle 2 and the mobile terminal 30. The general information 627 is general information obtained from an external information source 70. The external information source 70 is, for example, another server connected to the cloud server 50 via a communication network. The general information 627 may include, for example, weather information, traffic congestion information, review site information, news, and music rankings, etc.

[0043] The history information 624 may include the interaction / conversation history between the controller LLM and other LLMs. Here, other LLMs refer to the in-vehicle LLMs and mobile LLMs installed in the vehicle 2 and the mobile terminal 30 that make up the service providing system 1. The in-vehicle LLMs and mobile LLMs will be described later. The history information 624 may include the history information 254, 454 provided from the vehicle 2 and the mobile terminal 30. The history information 624 may be used for generating prompts and control instructions, and for learning and tuning the controller LLM, etc.

[0044] The instruction file 625 is a file that shows the prerequisite information for controlling the conversation between LLMs by imposing certain constraints on the controller LLM, the in-vehicle LLMs, and the mobile LLMs. That is, the instruction file 625 is used to learn and tune each LLM so that an appropriate answer can be obtained from each LLM.

[0045] For example, the role of the controller LLM may be described in the instruction file 625. For example, the role of the in-vehicle LLM may be described in the instruction file 255 to be described later, and for example, the role of the mobile LLM may be described in the instruction file 455 to be described later.

[0046] In the instruction file 625, as shown in FIG. 4, the following may be described as the role of the LLM. For example, it may be described that the controller LLM is required to act as an assistant to the occupant, and is required to collect information necessary for user support from the vehicle 2 or the mobile terminal 30 as needed. It may also be described that the in-vehicle LLM and the mobile LLM are required to generate queries for accessing a database or the like where user information and situation information are managed for a given question, and that only the query is to be provided without adding explanations or additional information when generating the query.

[0047] The instruction file 625 may further include information specifying a format (e.g., JSON format) used when the LLM converses with a device or a user. The format may include information indicating the conversation content, information specifying the related device that is the conversation partner, and information specifying the output format of the conversation. The related devices include the vehicle 2 and the mobile terminal 30, and may further include the in-vehicle LLM and in-vehicle equipment 23 of the vehicle 2, and the mobile LLM and mobile application 43 of the mobile terminal 30.

[0048] The control information 626 describes information for correctly handling related devices such as the in-vehicle equipment 23 and the mobile application 43 to be controlled in the service providing system 1. That is, the control information 626 is loaded into the LLM to train and tune the LLM so that the LLM can generate accurate instructions for the related devices. The control information 626 may include the name of the control target, the description of the control target, the state of the control target, and the like. The control information 626 is described, for example, as shown in FIG. 5.

[0049] The controller AI unit 63 has a function to detect the occurrence of an event indicating a situation requiring assistance to the user, based on predetermined event occurrence conditions, using information stored in the integrated memory unit 62 or information acquired by the information management unit 61 for temporary use. The event occurrence conditions may include, for example, the elapsed time since the user boarded or alighted. The event occurrence conditions may also include the notification to the controller AI unit 63 of an event detected by a related device (i.e., the vehicle 2 or mobile terminal 30). Event detection may be performed according to a predetermined logic, or it may be performed by a controller LLM that has been learned or tuned to detect the occurrence of events by taking user information and situation information as input.

[0050] The controller AI unit 63 has the function of acquiring the status of related devices and the user by communicating with related devices based on the content of the events that have occurred, and issuing control instructions to the related devices as necessary. Control instructions may include instructions to activate equipment on the related devices, and instructions to provide information on the related devices.

[0051] The controller AI unit 63 has a function to generate prompts used for conversations with the controller LLM and related devices (i.e., the vehicle 2 and the mobile terminal 30). A prompt is text that gives instructions or questions to the AI, etc. In other words, it is an input sentence that tells the AI ​​what to do. The controller AI unit 63 may generate the prompt in the controller LLM and then use the generated prompt to query the related devices. In this case, the controller AI unit 63 can obtain a response from the queried related device that takes into account the profile and context. The prompt may also specify the format to be used for describing the response.

[0052] The controller AI unit 63 may use the controller LLM to determine whether or not to acquire further information about user information and status information, and to generate control instructions for devices to realize services according to the user information and status information.

[0053] For example, in the case of sending control instructions regarding the air conditioner to a vehicle, the conditions for sending the control instructions may differ depending on the user information. Specifically, normally, a control instruction is sent when an event is detected indicating that the interior temperature of the vehicle has reached a standard temperature (e.g., 25 degrees Celsius or higher). However, if there is user information indicating that the user is sensitive to heat, it is conceivable that a control instruction would be sent when an event is detected indicating that the temperature has reached a lower standard temperature than usual (e.g., 23 degrees Celsius or higher).

[0054] [1-2-2. Vehicle] Vehicle 2 comprises an in-vehicle information source 21, an information collection unit 22, in-vehicle equipment 23, a control device 24, a local storage unit 25, and an in-vehicle AI unit 26. The information collection unit 22, the local storage unit 25, and the in-vehicle AI unit 26 are mounted on the edge device 10.

[0055] The in-vehicle information source 21 includes in-vehicle sensors such as cameras and radar, as well as an in-vehicle ECU mounted in the vehicle. ECU stands for Electronic Control Unit.

[0056] The information gathering unit 22 has the function of collecting user information and status information from the in-vehicle information source 21 and storing it in the local storage unit 25. The user information collected by the information gathering unit 22 includes personal information such as the user's age, gender, and driving characteristics, and is information that indicates the static characteristics and circumstances of the user related to the vehicle. The status information collected by the information gathering unit 22 includes in-vehicle image information and vehicle status information, and is information that indicates the dynamic characteristics and circumstances of the user related to the vehicle. The information gathering unit 22 periodically updates the user information and status information stored in the local storage unit 25. User information that does not change much may be updated less frequently, while status information that is constantly changing may be updated more frequently (for example, in real time). The in-vehicle LLM may be retrained and tuned according to the updated content each time the information in the local storage unit 25 is updated.

[0057] The information collection unit 22 has the function of synchronizing the user information and status information stored in the local storage unit 25 with the controller 60 by transmitting the user information and status information stored in the local storage unit 25 to the controller 60 according to predetermined criteria. For example, the information collection unit 22 may perform the synchronization of user information and status information periodically. User information that does not change often may be synchronized less frequently (for example, once a day), while status information that is constantly changing may be synchronized more frequently (for example, synchronized in real time each time the status information in the local storage unit 25 is updated). Furthermore, the synchronization of user information and status information may be performed according to instructions from the controller 60.

[0058] The in-vehicle equipment 23 consists of various pieces of equipment necessary to provide services that support the user. The in-vehicle equipment 23 may include, for example, an audio device, a display device, a communication device, and various actuators (for example, actuators that operate the air conditioner or power windows).

[0059] The control device 24 is an electronic control device that controls the in-vehicle equipment 23 and has the function of controlling the in-vehicle equipment 23 according to control instructions from the controller 60. The in-vehicle information source 21, the in-vehicle equipment 23, and the control device 24 are in-vehicle devices connected via the vehicle I / F unit 14.

[0060] As shown in Figure 9, the local storage unit 25 stores LLM parameters 251, profiles 252, context 253, history information 254, instruction files 255, control information 256, and the like.

[0061] The LLM parameter 251 is a set of parameters that constitute a large-scale language model. The LLM implemented by the LLM parameter 251 installed in vehicle 2 is the in-vehicle LLM.

[0062] Profile 252 is user information collected by the information collection unit 22. Context 253 is status information collected by the information collection unit 22. Profile 252 and context 253 may include user information and status information generated by the in-vehicle LLM based on information obtained from the in-vehicle information source 21. Profile 252 and context 253 may also include user information and status information notified by the controller 60.

[0063] The history information 254 may include the history of interactions and conversations between the in-vehicle LLM and the user, and the history of interactions and conversations between the in-vehicle LLM and other LLMs. Here, "other LLMs" refers to the controller LLM installed in the controller 60 that constitutes the service provision system 1, and the mobile LLM installed in the mobile terminal 30.

[0064] The in-vehicle AI unit 26 has the function of generating a response to an inquiry (for example, status information) using the in-vehicle LLM when it receives an inquiry from the controller 60, and transmitting it to the controller 60. The in-vehicle AI unit 26 has the function of searching for information stored in the local storage unit 25 in order to generate a response to an inquiry, and the function of exchanging information with the control device 24. The response to the inquiry may be generated based on the history information 254.

[0065] The in-vehicle AI unit 26 has the function of generating responses in conversations with the controller AI unit 63.

[0066] The in-vehicle AI unit 26 may have a function to store in the local storage unit 25 the results of recognizing the user's characteristics and the in-vehicle situation using the in-vehicle LLM, based on information such as information inside and around the vehicle 2 acquired from the in-vehicle information source 21, as user information and situation information.

[0067] The in-vehicle AI unit 26 may have a function to operate the in-vehicle equipment 23 in the control device 24 according to control instructions from the controller 60. The control instructions from the controller 60 may be instructions in a format that the control device 24 can execute (hereinafter referred to as JOB), or they may be conversational instructions that indicate the items necessary for generating a JOB. If the control instructions from the controller 60 are JOBs, the in-vehicle AI unit 26 may transfer the JOBs directly to the control device 24. If the control instructions from the controller 60 are conversational instructions, the in-vehicle AI unit 26 may convert the conversational instructions into JOBs using the in-vehicle LLM and transfer them to the control device 24.

[0068] [1-2-3. Mobile Devices] Mobile devices 30 are, for example, smartphones or tablet devices that can be carried by the user.

[0069] The mobile terminal 30 comprises a mobile information source 41, an information collection unit 42, a mobile application 43, a control device 44, a local storage unit 45, and a mobile AI unit 46.

[0070] The mobile information source 41 includes sensors installed in the mobile terminal 30, an application program (hereinafter referred to as a mobile app) 43 executed by the mobile terminal 30, and an OS that manages the execution of the mobile app. The mobile app 43 may include an app that collects healthcare information regarding the user's health status, and an app that manages the user's schedule, etc.

[0071] The information obtained from the mobile information source 41 may include information specific to the terminal user and information indicating the status of the mobile terminal 30. Information specific to the terminal user may include, for example, usage information of the mobile application 43 installed on the mobile terminal 30, healthcare information of the terminal user obtained by running the mobile application 43, and the terminal user's planned activities and activity history. Information indicating the status of the mobile terminal 30 may include, for example, location information.

[0072] The information gathering unit 42 has the function of saving user information and status information from the mobile information source 41 to the local storage unit 45. Other functions of the information gathering unit 42 are the same as those of the information gathering unit 22 of the vehicle 2.

[0073] The control device 44 is an electronic control device that controls the equipment of the mobile terminal 30 and the mobile application 43 installed on the mobile terminal 30, and has the function of controlling the mobile application 43, etc., in accordance with control instructions from the controller 60.

[0074] As shown in Figure 9, the local storage unit 45 stores LLM parameters 451, profiles 452, context 453, history information 454, instruction files 455, control information 456, and the like.

[0075] The LLM parameter 451 is a set of parameters that constitute a large-scale language model. The LLM implemented by the LLM parameter 451 installed on the mobile terminal 30 is called mobile LLM.

[0076] Profile 452 is user information collected by the information collection unit 42. Context 453 is status information collected by the information collection unit 42. Profile 452 and context 453 may include user information and status information generated by the mobile LLM based on information obtained from the mobile information source 41. Profile 452 and context 453 may also include user information and status information notified by the controller 60.

[0077] The history information 454 may include the history of interactions and conversations between the mobile LLM and the terminal user, and the history of interactions and conversations between the mobile LLM and other LLMs. Here, "other LLMs" refers to the controller LLM installed in the controller 60 that constitutes the service provision system 1, and the in-vehicle LLM installed in the vehicle 2.

[0078] The mobile AI unit 46 has the same functions as the in-vehicle AI unit 26. However, in the description of the mobile AI unit 46, the in-vehicle LLM in the description of the in-vehicle AI unit 26 should be read as mobile LLM, and the vehicle 2 as mobile terminal 30. Furthermore, the in-vehicle information source 21, in-vehicle equipment 23, control device 24, and local storage unit 25 should be read as mobile information source 41, mobile application 43, control device 44, and local storage unit 45, respectively.

[0079] [1-3. Processing] The main processing performed by the controller 60 in order to function as the controller AI unit 63 will be explained using the flowchart in Figure 6. In addition to the main processing, the controller 60 also performs tuning processing to cause the controller LLM to read the instruction file 625 and control information 626 when the controller 60 is started up and when the instruction file 625 and control information 626 are updated.

[0080] The main process is repeatedly executed by the controller 60.

[0081] In S110, the controller 60 monitors the user information and status information stored in the integrated storage unit 62 and performs event detection processing to detect the occurrence of an event by determining whether the user information and status information meet the pre-set event occurrence conditions.

[0082] In S120, the controller 60, if an event has been detected in the event detection process, proceeds to S130; otherwise, it returns to S110.

[0083] In S130, the controller 60 determines whether additional information is needed to determine the content of a service (i.e., support for the user) corresponding to the event that occurred. The need for additional information may be determined by referring to a table prepared in advance that associates events with the necessary information and the source of that information. Alternatively, the need for additional information may be determined by generating a prompt and querying the controller LLM. In this case, the controller LLM outputs a response that includes the necessary information and the source of that information.

[0084] If the controller 60 determines that additional information is needed, it proceeds to process S140; if it determines that additional information is not needed, it proceeds to process S150.

[0085] In S140, the controller 60 retrieves the necessary information from the related device that is the source of the information, according to the information obtained from the table or controller LLM in S130, and returns the process to S130. Information may be retrieved from multiple related devices, or additional information may be retrieved based on the information obtained. The related device may access its own local storage units 25 and 45 in response to the information retrieval request from the controller 60 to obtain the necessary information. Alternatively, the related device may obtain the necessary information by querying its own LLM.

[0086] In S150, the controller 60 generates a control instruction using the controller LLM. The controller 60 causes the controller LLM to generate a control instruction based on the event that occurred, the user information of the user involved in the event, and the status information of the vehicle and the user. The user information and status information include information stored in the integrated storage unit 62 and information acquired in S140. The control instruction is written, for example, in JSON format and indicates the related device to which the control instruction is to be sent and the content of the control instruction. The control instruction may be generated for multiple related devices.

[0087] In S160, the controller 60 sends the control instruction generated in S150 to the destination related device and terminates the process.

[0088] The related device that receives the control instruction executes the processing according to the control instruction. For example, it may activate an actuator, output audio, or display information on a screen. Furthermore, it may send notifications to necessary locations. The related device may also notify the controller 60 of the control result of the control instruction.

[0089] [1-4. Use Cases] Use cases for Service Delivery System 1 are described below.

[0090] [1-4-1. Baby Abandonment Detection] This use case assumes a situation where a baby is left alone in a car on a hot day.

[0091] As shown in Figure 7, in S11, when the edge device 10 of the vehicle 2 detects that the user has disembarked based on information from the in-vehicle information source 21, it notifies the controller 60 of the cloud server 50 of the detection of the user's disembarkation.

[0092] Upon receiving notification of passenger disembarkation, the controller 60 detects the occurrence of an event in S12, as the event conditions are met. Since this event requires information gathering, the controller 60 inquires about the in-vehicle conditions from the edge device 10 of the vehicle 2 that sent the disembarkation notification.

[0093] Upon receiving the inquiry, the edge device 10 acquires status information indicating the in-vehicle situation from an in-vehicle information source 21 such as an in-vehicle camera or from a local storage unit 25 in S13. Furthermore, if the edge device 10 detects from the acquired status information that there is only one occupant in the vehicle, such as a baby, it notifies the controller 60 of the detected information.

[0094] Upon receiving notification of additional information from the edge device 10, the controller 60 determines in S14 that additional information is needed to determine the support to be provided to the user, and queries the mobile terminal 30 held by the user for location information, time spent at the location, etc.

[0095] Upon receiving the inquiry, the mobile terminal 30 obtains the location information of the mobile terminal 30 (i.e., the user possessing the mobile terminal 30) and the user's estimated average stay time at the facility from the mobile information source 41 in S15. The mobile terminal 30 also notifies the controller 60 of the obtained location information and average stay time as additional information indicating the user's status. For the average stay time, for example, information pre-stored in the local storage unit 45 of the mobile terminal 30 linked to the facility's location information is used. In this use case, it is assumed that an entertainment facility that the user regularly visits is notified as location information, and the user's average stay time at that entertainment facility is 60 minutes. Note that information on the average stay time at the entertainment facility may also be pre-stored as user information in the integrated storage unit 62 of the controller 60.

[0096] Upon receiving notification of additional information from the mobile terminal 30, the controller 60, in S16, uses the controller LLM to make a situation assessment and derive a solution based on the acquired additional information and the information stored in the integrated storage unit 62. In this use case, to implement the derived solution, for example, the controller 60 may send a control instruction to the edge device 10 to activate the air conditioner and a control instruction to the mobile terminal 30 to output a voice prompt urging the user to return to their vehicle.

[0097] [1-4-2. Driver Sleep Deprivation Detection] This use case assumes a situation where a user who is sleep-deprived and stressed is commuting to work by car.

[0098] The integrated storage unit 62 of the controller 60 may have user information pre-stored in it, such as the user's vehicle driving time (for example, 7:00 AM when they go to work) and driving characteristics, which are obtained from a mobile terminal.

[0099] As shown in Figure 8, in S21 and S22, when the mobile terminal 30 detects from a mobile information source 41, such as a healthcare app, that the user is sleep-deprived and that the user is under stress, it notifies the controller 60 of the detected information as status information indicating the user's condition.

[0100] In step S23, when the mobile terminal 30 detects from a mobile information source 41, such as a schedule app, that a meeting is scheduled for early morning, it notifies the controller 60 of the detected information as status information indicating the user's status.

[0101] The controller 60 sequentially updates the context 623 of the integrated storage unit 62 based on the status information notified from the mobile terminal 30.

[0102] The controller 60 acquires information related to vehicle operation from the external information source 70 as needed and sequentially updates the context 623 of the integrated storage unit 62. In this case, the information acquired from the external information source 70 includes traffic congestion information.

[0103] In S24, the controller 60 determines, based on traffic congestion information and schedule information notified from the mobile terminal 30, that the vehicle's route to its destination is congested, and detects that the conditions for an event to occur have been met, thereby detecting the occurrence of an event.

[0104] In S25, the controller 60 uses the controller LLM to make a situation assessment and derive a solution based on the user information and situation information that is known at the time the event occurs. At this time, the controller LLM is input with situation information indicating the user's situation, including not only that there is traffic congestion, but also that a meeting is scheduled early in the morning, that the user is sleep-deprived, and that the user is under stress, and derives a solution according to the user's situation. In this use case, in order to implement the derived solution, the controller 60 may send a voice output control instruction to the mobile terminal 30 suggesting a change to the usual departure time. Furthermore, the controller 60 may send a control instruction to the edge device 10 suggesting setting an alternative route to avoid the congested route, or, taking into account the user's anxiety about being late for a meeting, a control instruction to guide them to a place where they can safely park and make a phone call.

[0105] Upon receiving a control instruction for voice output, the mobile terminal 30, in S26, uses its built-in sound device to output a voice suggestion for changing the departure time. Upon receiving a control instruction for setting a detour, the edge device 10, in S27, processes the navigation system or the like installed in the vehicle 2 to set a detour. Furthermore, upon receiving a control instruction for guidance to a safe location, the edge device 10, in S28, searches for a safe location and processes the navigation system or the like to guide the user to that safe location.

[0106] [1-5. Correspondence of Terms] In the first embodiment, the processing S110 to S120 executed by the controller 60 corresponds to the event detection unit of this disclosure, and the processing S130 to S160 corresponds to the control instruction unit of this disclosure. In this embodiment, the instruction file corresponds to the instruction information of this disclosure.

[0107] [1-6. Effects] The first embodiment described in detail above produces the following effects.

[0108] (1a) The service provision system 1 integrates user information (i.e., profile) and situation information (i.e., context) provided from the vehicle 2 and related devices such as the mobile terminal 30 to comprehensively understand the user's characteristics and the situation in which the user is placed. Then, using LLM, it generates control instructions to realize the optimal service for the driver derived from the understood information, and controls the related devices with the generated control instructions. Therefore, it is possible to provide accurate services that take into account the user's situation and the user's characteristics.

[0109] (1b) According to the service provision system 1, when an event is detected, additional user information and status information are acquired as needed and used to generate control instructions. Therefore, even for the same event, multiple types of services can be provided to the user depending on the situation.

[0110] [2. Second Embodiment] [2-1. Differences from the First Embodiment] The second embodiment has the same basic configuration as the first embodiment, so the differences will be explained below. Note that the same reference numerals as in the first embodiment indicate the same components, and refer to the preceding description.

[0111] In the first embodiment described above, the controller 60 in the cloud server 50 collects user information and status information from the vehicle 2 and mobile terminal 30 and uses it to provide services. In contrast, the second embodiment differs from the first embodiment in that it also uses information collected from mobile terminals 30A held by passengers other than the driver of the vehicle 2 (hereinafter referred to as passengers) to provide services.

[0112] [2-2. Configuration] The service provision system 1a of the second embodiment shown in Figure 10 comprises a vehicle 2, mobile terminals 30 and 30A, and a cloud server 50. Mobile terminal 30 is held by the driver of vehicle 2. Mobile terminal 30A is held by a passenger in vehicle 2.

[0113] [2-2-1. Cloud Server] The cloud server 50 has the same configuration and functional configuration as the cloud server 50 described in the first embodiment.

[0114] However, the information management unit 61 has the function of acquiring user information and status information not only from the vehicle 2 and the driver's mobile terminal 30, but also from the passenger's mobile terminal 30A, and storing it in the integrated storage unit 62. In the integrated storage unit 62, the profile 622, context 623, and history information 624 are managed for each occupant.

[0115] The main processing performed by the controller 60 is basically the same as the main processing in the first embodiment described with reference to Figure 6. However, in the second embodiment, the user information may include the driver's user information and the passenger's user information.

[0116] [2-2-2. Vehicle] Vehicle 2 has the same configuration and functional configuration as Vehicle 2 described in the first embodiment. However, Vehicle 2 is connected to communicate not only with the driver's mobile terminal 30 but also with the passenger's mobile terminal 30A. Hereinafter, as shown in Figure 11, blocks showing the functional configuration of the mobile terminal 30A are indicated by adding the letter A to the reference numeral of the block showing the functional configuration of the mobile terminal. The mobile terminal 30A corresponds to the device of this disclosure, similar to Vehicle 2, mobile terminal 30, and cloud server 50.

[0117] The edge device 10 has the function of detecting when the driver gets in and out of the vehicle, whether there are any passengers, etc., and notifying the controller 60 of the cloud server 50. Whether the driver is in the vehicle or whether there are any passengers may be determined based on various vehicle data collected via the vehicle I / F unit 14, such as the opening and closing of the vehicle 2 doors, passenger sensors provided on each seat, and video footage taken inside the vehicle.

[0118] The edge device 10 may also have a function to acquire user information (hereinafter referred to as passenger information) about the passenger from the mobile terminal 30A held by the passenger if there is a passenger, store it in the local storage unit 25, and further synchronize the passenger information stored in the local storage unit 25 with the integrated storage unit 62 of the cloud server 50.

[0119] Passenger information may include information about their entertainment preferences, such as music, movies, dramas, games, sports, theater, concerts, and exhibitions.

[0120] [2-3. Use Cases] Use cases for service delivery system 1a will be described.

[0121] [2-3-1. Use of passenger information while the driver is in the vehicle] This scenario assumes a situation where the service is provided using passenger information, which is user information about passengers in vehicle 2.

[0122] As shown in Figure 12, when the passenger's mobile terminal 30A detects in S31 that the passenger has boarded vehicle 2, it notifies the edge device 10 of vehicle 2 that the passenger has boarded. This notification may include information that identifies the passenger who possesses the mobile terminal 30A. Detection of boarding vehicle 2 may occur through a predetermined operation on the mobile terminal 30A by the passenger, or when the mobile terminal 30A detects an inquiry from the edge device 10.

[0123] When the edge device 10 receives a notification from the mobile terminal 30A indicating the presence of a passenger, it forwards the notification to the controller 60. The edge device 10 may also store the presence of a passenger in the local storage unit 25 in accordance with the notification, and may reserve an area for storing user information and status information for the passenger.

[0124] Upon receiving notification that a passenger is on board, the controller 60 may store in the integrated storage unit 62 that there is a passenger in the vehicle 2, and may also reserve an area for storing user information and status information for the passenger.

[0125] In step S32, the mobile terminal 30 owned by the driver transmits user information and status information about the driver owned by the mobile terminal 30 to the controller 60.

[0126] In step S33, the mobile terminal 30A held by the passenger transmits user information and status information about the passenger that the mobile terminal 30A contains to the controller 60.

[0127] In S34, the edge device 10 transmits user information and status information about the driver it possesses to the controller 60.

[0128] Alternatively, the mobile terminals 30 and 30A may transmit information to the edge device 10 and then to the controller 60 via the edge device 10, instead of directly transmitting it to the controller 60.

[0129] As steps S32 to S33 are executed as appropriate, the user information and status information held by the controller 60 are maintained in a state synchronized with the information held by the mobile terminals 30, 30A and the edge device 10.

[0130] In S35, the controller 60 determines whether an event has occurred based on the user information and status information stored in the integrated storage unit 62. Here, the event occurrence condition is whether the situation is such that music playback should be performed, and the determination is made based on whether the event occurrence condition is met. Situations such as when music playback should be performed include, for example, when the conversation between the driver and passenger includes a request for music playback, or when there has been no conversation between the driver and passenger for a certain period of time or longer. Furthermore, the event occurrence condition may also include a request from a passenger via the mobile terminal 30A. In this case, the mobile terminal 30A may detect the occurrence of the event and notify the controller 60 that the event has been detected.

[0131] When the controller 60 detects the occurrence of an event, it queries the mobile terminal 30A associated with each passenger for information about the passenger, in this case, their music preferences. The query may also be sent to the mobile terminal 30 associated with the driver. Furthermore, if the information synchronized in steps S32 to S34 includes information such as each user's music preferences, the query by the controller 60 may be omitted.

[0132] Upon receiving the inquiry, the mobile terminal 30A, in S36, refers to the profile 452A, context 453A, and history information 454A, etc., stored in the local memory unit 45A, to create an answer to the inquiry, and notifies the controller 60 of the created answer. The content of the notification may include preferred music genres, preferred artists, preferred songs, etc.

[0133] Upon receiving notifications from all contact points, the controller 60, in S37, uses the controller LLM to make a situation assessment and derive a solution based on the information stored in the integrated memory unit 62 and the information acquired from the mobile terminals 30 and 30A. In this use case, music that takes into account the preferences of all passengers is extracted, and a control instruction to play the extracted music is sent to the edge device 10 of the vehicle 2. If there are multiple passengers, the preferences of all passengers may be analyzed and music that everyone is likely to like may be played, or the music that each passenger likes may be played in turn. Historical information about the passengers may be used to analyze their preferences. In addition to music, control instructions may also be sent to play sports broadcasts, movies, etc.

[0134] Upon receiving a control instruction from the controller 60, the edge device 10 of the vehicle 2 performs music playback or other actions in accordance with the content of the control instruction, according to the passenger's preferences, in S38.

[0135] [2-3-2. Use of passenger information after the driver exits the vehicle] When the driver exits vehicle 2, a situation is envisioned in which passengers remaining in vehicle 2 are provided with services appropriate to them.

[0136] In this use case, we will assume that the processes described in S31 to S34 above are performed while the driver is in the vehicle.

[0137] As shown in Figure 13, when the edge device 10 detects that the driver has disembarked in S41, it notifies the controller 60 of the detection result.

[0138] Upon receiving the information notification that a passenger has disembarked, the controller 60 determines in S42 that an event has occurred and queries the edge device 10, the source of the information notification, for information about the in-vehicle situation. In other words, in this use case, receiving the information notification that a passenger has disembarked is used as the condition for an event to occur.

[0139] When the edge device 10 receives an inquiry about the situation inside the vehicle, in S43, it detects any passengers remaining inside the vehicle based on the in-vehicle video and signals from various sensors, and then notifies the controller 60 of the detection result.

[0140] Upon receiving notification of the detection of a passenger remaining behind, the controller 60 collects information in S44 by making inquiries to the driver's mobile terminal 30 and the mobile terminal 30A of the remaining passenger. Here, the controller inquires from the driver's mobile terminal 30 about the driver's location and the estimated time required to return to vehicle 2, and from the passenger's mobile terminal 30A about the passenger's preferences regarding entertainment, which can be useful in suggesting ways to pass the time.

[0141] Upon receiving the inquiry, the mobile terminal 30A retrieves information from the local storage unit 45A, etc., in S45 and notifies the controller 60. Similarly, the mobile terminal 30 retrieves information from the local storage unit, etc., in S46 and notifies the controller 60.

[0142] Upon receiving notification of the information, the controller 60, in S47, uses the controller LLM to make a situation assessment and derive a solution based on the user information and situation information stored in the integrated storage unit 62 and the information collected from the mobile terminals 30 and 30A, and transmits a control instruction to the edge device 10. The time required for the driver to return to the vehicle may be estimated by the controller LLM.

[0143] Here, for example, if it is anticipated that the driver will need some time to return to vehicle 2, a control instruction is sent to play music that suits the preferences of the remaining passenger.

[0144] The edge device 10, upon receiving a control instruction from the controller 60, provides entertainment services to the remaining passengers, such as playing music, in accordance with the control instruction. The services provided in accordance with the control instruction may include services that adjust the indoor environment, such as adjusting the temperature of the air conditioner or opening and closing the windows. When generating control instructions for providing services that adjust the indoor environment, information obtained from an external information source 70 that provides weather forecasts may be used, for example.

[0145] [2-3. Correspondence of Terms] In the second embodiment, user information relating to a passenger corresponds to passenger information in this disclosure.

[0146] [2-4. Effects] The second embodiment described in detail above provides the effects (1a) and (1b) of the first embodiment described above, and further provides the following effects.

[0147] (2a) The service provision system 1a can provide diverse and accurate services that take into account not only user information and status information about the driver of vehicle 2, but also user information and status information about passengers in vehicle 2.

[0148] [3. Other Embodiments] Although embodiments of the present disclosure have been described above, the present disclosure is not limited to the embodiments described above and can be implemented in various modified forms.

[0149] (3a) In the above embodiment, an example was described in which the controller 60 is located on the cloud server 50, but the location where the controller 60 is located is not limited to the cloud server. For example, the controller 60 may be located on an edge device 10 of the vehicle 2 or on a related device such as a mobile terminal 30. When the controller 60 is located on the vehicle 2, the vehicle 2 does not necessarily have to be equipped with both the on-board LLM and the controller LLM; it may be equipped with only the controller LLM. Also, if the controller 60 is not located on the cloud server 50, a separate cloud LLM may be installed, which is used to reflect information obtained from an external information source 70, etc., in the processing of the controller 60.

[0150] (3b) In the above embodiment, the vehicle 2 and mobile terminals 30, 30A are given as examples of related devices, but the related devices may also include home appliances, etc., that can communicate with at least one of the vehicle 2, mobile terminals 30, 30A, and controller 60. In this case, either or both of the vehicle 2 and mobile terminals 30, 30A may be excluded from the related devices.

[0151] (3c) In the above embodiment, each of the related devices is equipped with an LLM, but LLMs other than the controller LLM may be omitted. Also, each LLM may be replaced or updated as appropriate.

[0152] (3d) In the above embodiment, the vehicle 2, mobile terminals 30, 30A, and controller 60 communicate with each other, but there are no particular restrictions on the communication protocol, and any communication protocol can be used.

[0153] (3e) In the above embodiment, the controller 60 detects the occurrence of an event, but the related device may also detect the occurrence of an event and notify the controller 60 that it has detected an event. Alternatively, detection by the controller 60 and detection by the related device may be used in combination.

[0154] (3f) The controller 60, edge device 10, and mobile terminals 30, 30A (hereinafter collectively referred to as the controller 60, etc.) and the method described herein may be implemented by a dedicated computer provided by configuring a processor and memory programmed to execute one or more functions embodied by a computer program. Alternatively, the controller 60, etc. and the method described herein may be implemented by a dedicated computer provided by configuring a processor by one or more dedicated hardware logic circuits. Alternatively, the controller 60, etc. and the method described herein may be implemented by one or more dedicated computers configured by a combination of a processor and memory programmed to execute one or more functions and a processor configured by one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by the computer on a computer-readable non-transitional tangible recording medium. The method for realizing the functions of each part included in the controller 60, etc. does not necessarily need to include software, and all of its functions may be realized using one or more hardware components.

[0155] (3g) Multiple functions of one component in the above embodiment may be realized by multiple components, or one function of one component may be realized by multiple components. Also, multiple functions of multiple components may be realized by one component, or one function realized by multiple components may be realized by one component. Furthermore, some of the configurations of the above embodiment may be omitted. Furthermore, at least some of the configurations of the above embodiment may be added to or replaced with the configurations of other above embodiments.

[0156] (3h) In addition to the service provision system 1 described above, this disclosure can also be realized in various forms, such as the edge device 10 and server 50 which are components of the service provision system 1, a program for causing a computer to function as the controller 60 of the service provision system 1, a non-transitional physical recording medium such as semiconductor memory on which this program is recorded, and a service provision method.

[0157] [4. Technical Concept Disclosed in This Specification] [Item 1] A service provision system comprising: an information management unit (61) configured to acquire passenger information relating to the passengers of the vehicle and status information indicating the status of the associated devices or the passengers, with each of the vehicle and the mobile terminals of one or more passengers of the vehicle as associated devices; an event detection unit (26: S110 to S120) configured to detect the occurrence of an event relating to the vehicle or the passengers; and a control instruction unit (S130 to S160) configured to generate control instructions using a large-scale language model to realize a service that matches the passenger information and status information at the time of the event occurrence, and to transmit these instructions to at least one of the associated devices when the event detection unit detects the occurrence of the event.

[0158] [Item 2] A service provision system according to Item 1, wherein when the occurrence of the event is detected, the control instruction unit further acquires additional information from at least one of the passenger information and the status information related to the control instruction from at least one of the associated devices, and uses the acquired additional information to generate the control instruction.

[0159] [Item 3] A service provision system as described in Item 2, wherein the control instruction unit, upon detection of the occurrence of the event, generates a prompt for querying the additional information and transmits it to the associated device.

[0160] [Item 4] A service provision system according to any one of Items 1 to 3, wherein the information management unit is configured to further acquire information relating to the occurrence of the event and information relating to the control instructions for realizing the service from an external information source (70) other than the related device.

[0161] [Item 5] A service provision system according to any one of items 1 to 4, wherein the large-scale language model used by the control instruction unit is learned and tuned using control information describing the functions of the equipment of the related device.

[0162] [Item 6] A service provision system according to any one of Items 1 to 5, wherein the large-scale language model used by the control instruction unit is learned and tuned using instruction information describing the roles of the control instruction unit and the related devices.

[0163] [Item 7] A service provision system as described in Item 6, wherein the instruction information includes information that defines a format used for conversation between the large-scale language model used by the control instruction unit and the large-scale language model possessed by the associated device.

[0164] [Item 8] A service provision system as described in Item 7, wherein the format includes information indicating the content of a conversation, information specifying the related device that will be the conversation partner, and information specifying the output format of the conversation.

[0165] [Item 9] A service provision system according to any one of items 1 to 8, wherein the control instruction includes an instruction to activate equipment on the associated device.

[0166] [Item 10] A service provision system according to any one of items 1 to 9, comprising an integrated storage unit (62) that integrates and stores the passenger information and the status information acquired from a plurality of related devices by the information management unit, and an event detection unit that detects the occurrence of the event based on the information stored in the integrated storage unit.

[0167] [Item 11] A service provision system according to any one of Items 1 to 10, wherein the related device comprises a local storage unit (45A) that stores the passenger information and the status information relating to the related device, and the event detection unit is provided on the related device and detects the occurrence of the event based on the information stored in the local storage unit.

[0168] [Item 12] A service provision system according to any one of Items 1 to 9, further comprising: an integrated storage unit (62) that integrates and stores the passenger information and the status information provided by a plurality of related devices by the information management unit; and a local storage unit (45A) provided on the related device that stores the passenger information and the status information acquired by the related device, wherein the information management unit has a function to synchronize the information stored in the integrated storage unit and the local storage unit.

[0169] [Item 13] A service provision system as described in Item 12, wherein the information management unit periodically performs the synchronization in accordance with the updates of the information held by the local storage unit.

[0170] [Item 14] A service provision system according to Item 12 or Item 13, wherein the information management unit performs the synchronization in real time in accordance with the update of information held by the local storage unit.

[0171] [Item 15] A service provision system according to any one of items 1 to 14, wherein the control instruction generated by the control instruction unit is a JOB-format instruction that can be executed by the computer of the associated device.

[0172] [Item 16] A service provision system according to any one of items 1 to 15, wherein the control instruction generated by the control instruction unit is an instruction to cause the large-scale language model of the associated device to generate a JOB-format instruction that can be executed by the computer of the associated device.

[0173] [Item 17] A service provision system according to any one of items 1 to 16, wherein the control instruction unit further acquires history information including the history of communication with the related device, and uses the acquired history information to generate the control instruction.

[0174] [Item 18] A service provision system according to any one of items 1 to 17, wherein the control instruction unit, when acquiring information from the associated device, acquires a response based on historical information including the history of communication with the control instruction unit stored in the associated device, and uses the response to generate the control instruction.

[0175] [Item 19] An edge device mounted on a vehicle, comprising: an information management unit configured to acquire passenger information relating to the passengers of the vehicle and status information indicating the status of the associated devices or the passengers, with each of the vehicle and one or more passengers' mobile terminals being associated devices; an event detection unit configured to detect the occurrence of an event relating to the vehicle or the passengers; and a control instruction unit configured to generate control instructions using a large-scale language model to realize a service that matches the passenger information and status information at the time of the event, and transmit them to at least one of the associated devices, when the event detection unit detects the occurrence of the event.

[0176] [Item 20] A server that communicates with a vehicle, comprising: an information management unit configured to acquire passenger information relating to the passengers of the vehicle and status information indicating the status of the associated devices or the passengers, with each of the vehicle and one or more passengers' mobile terminals being associated devices; an event detection unit configured to detect the occurrence of an event relating to the vehicle or the passengers; and a control instruction unit configured to generate control instructions using a large-scale language model to realize a service that matches the passenger information and status information at the time of the event, and to transmit these instructions to at least one of the associated devices, when the event detection unit detects the occurrence of the event.

[0177] [Item 21] A service provision method implemented by a vehicle or a server communicating with the vehicle, comprising: acquiring passenger information relating to the passengers of the vehicle and status information indicating the status of the associated devices or the passengers, with each of the vehicle and one or more passengers' mobile terminals being associated devices; detecting the occurrence of an event relating to the vehicle or the passengers; and, when the occurrence of the event is detected, using a large-scale language model, generating control instructions to realize a service that matches the passenger information and status information at the time of the event, and transmitting these instructions to at least one of the associated devices.

[0178] [Item 22] A program that causes a computer to function as: an information management unit configured to acquire passenger information relating to the passengers of the vehicle and status information indicating the status of the associated devices or the passengers, with each of the vehicle and the mobile terminals of one or more passengers of the vehicle as associated devices; an event detection unit configured to detect the occurrence of an event relating to the vehicle or the passengers; and a control instruction unit configured to generate control instructions using a large-scale language model to realize a service that matches the passenger information and status information at the time of the event, and to transmit these instructions to at least one of the associated devices, when the event detection unit detects the occurrence of the event.

Claims

1. A service provision system comprising: an information management unit (61) configured to acquire passenger information relating to the passengers of the vehicle and status information indicating the status of the associated devices or the passengers, with each of the vehicle and the mobile terminals of one or more passengers of the vehicle as associated devices; an event detection unit (26: S110 to S120) configured to detect the occurrence of an event relating to the vehicle or the passengers; and a control instruction unit (S130 to S160) configured to generate control instructions using a large-scale language model to realize a service that matches the passenger information and status information at the time of the event, and to transmit these instructions to at least one of the associated devices when the event detection unit detects the occurrence of the event.

2. A service provision system according to claim 1, wherein when the occurrence of the event is detected, the control instruction unit further acquires additional information from at least one of the passenger information and the status information related to the control instruction from at least one of the associated devices, and uses the acquired additional information to generate the control instruction.

3. A service provision system according to claim 2, wherein the control instruction unit, upon detection of the occurrence of the event, generates a prompt for querying the additional information and transmits it to the associated device.

4. A service provision system according to claim 1, wherein the information management unit is configured to further acquire information relating to the occurrence of the event and information relating to the control instructions for realizing the service from an external information source (70) other than the related device.

5. A service provision system according to claim 1, wherein the large-scale language model used by the control instruction unit is learned and tuned using control information describing the functions of the equipment of the associated device.

6. A service provision system according to claim 1, wherein the large-scale language model used by the control instruction unit is learned and tuned using instruction information describing the roles of the control instruction unit and the related devices.

7. A service provision system according to claim 6, wherein the instruction information includes information defining a format used for conversation between the large-scale language model used by the control instruction unit and the large-scale language model possessed by the associated device.

8. A service provision system according to claim 7, wherein the format includes information indicating the content of a conversation, information specifying the related device that is the conversation partner, and information specifying the output format of the conversation.

9. A service provision system according to claim 1, wherein the control instruction includes an instruction to activate equipment of the associated device.

10. A service provision system according to claim 1, comprising an integrated storage unit (62) that integrates and stores passenger information and status information acquired from a plurality of related devices by the information management unit, and an event detection unit that detects the occurrence of an event based on the information stored in the integrated storage unit.

11. A service provision system according to claim 1, wherein the related device comprises a local storage unit (45A) that stores the passenger information and the status information acquired by the related device, and the event detection unit is provided on the related device and detects the occurrence of the event based on the information stored in the local storage unit.

12. A service provision system according to claim 1, further comprising: an integrated storage unit (62) that integrates and stores the passenger information and status information provided by a plurality of related devices by the information management unit; and a local storage unit (45A) provided on the related device that stores the passenger information and status information acquired by the related device, wherein the information management unit has a function to synchronize the information held by the integrated storage unit with the information held by the local storage unit.

13. A service provision system according to claim 12, wherein the information management unit periodically performs the synchronization in accordance with the update of information held in the local storage unit.

14. A service provision system according to claim 12, wherein the information management unit performs the synchronization in real time in accordance with the update of information held by the local storage unit.

15. A service provision system according to claim 1, wherein the control instruction generated by the control instruction unit is a JOB-format instruction that can be executed by a computer of the associated device.

16. A service provision system according to claim 1, wherein the control instruction generated by the control instruction unit is an instruction to cause the large-scale language model of the associated device to generate a JOB-format instruction that can be executed by the computer of the associated device.

17. A service provision system according to claim 1, wherein the control instruction unit further acquires history information (624) including the history of communication with the related device, and uses the acquired history information to generate the control instruction.

18. A service provision system according to claim 1, wherein the control instruction unit, when acquiring information from the associated device, acquires a response based on history information (454A) including the history of communication with the control instruction unit stored in the associated device, and uses the response to generate the control instruction.

19. An edge device mounted on a vehicle, comprising: an information management unit configured to acquire passenger information relating to the passengers of the vehicle and status information indicating the status of the associated devices or the passengers, with the vehicle and the mobile terminals of one or more passengers of the vehicle each being associated devices; an event detection unit configured to detect the occurrence of an event relating to the vehicle or the passengers; and a control instruction unit configured to generate control instructions using a large-scale language model to realize a service that matches the passenger information and status information at the time of the event, and transmit them to at least one of the associated devices, when the event detection unit detects the occurrence of the event.

20. A server that communicates with a vehicle, comprising: an information management unit configured to acquire passenger information relating to the passengers of the vehicle and status information indicating the status of the associated devices or the passengers, with the vehicle and the mobile terminals of one or more passengers of the vehicle each being associated devices; an event detection unit configured to detect the occurrence of an event relating to the vehicle or the passengers; and a control instruction unit configured to generate control instructions using a large-scale language model to realize a service that matches the passenger information and status information at the time of the event, and to transmit these instructions to at least one of the associated devices, when the event detection unit detects the occurrence of the event.

21. A service provision method implemented by a vehicle or a server communicating with the vehicle, comprising: acquiring passenger information relating to the passengers of the vehicle and status information indicating the status of the associated devices or the passengers, with each of the vehicle and one or more passengers' mobile terminals being associated devices; detecting the occurrence of an event relating to the vehicle or the passengers; and, when the occurrence of the event is detected, generating control instructions using a large-scale language model to realize a service that matches the passenger information and status information at the time of the event, and transmitting these instructions to at least one of the associated devices.

22. A program that causes a computer to function as: an information management unit configured to acquire passenger information relating to the passengers of the vehicle and status information indicating the status of the associated devices or the passengers, with each of the vehicle and one or more mobile terminals of passengers other than the vehicle as associated devices; an event detection unit configured to detect the occurrence of an event relating to the vehicle or the passengers; and a control instruction unit configured to generate control instructions using a large-scale language model to realize a service that matches the passenger information and status information at the time the event occurred, and to transmit these instructions to at least one of the associated devices, when the event detection unit detects the occurrence of the event.

Images