Method and device for communicating data relating to a problem in an embedded system of a vehicle

Abstract

The present invention relates to a method and device for communicating data concerning a problem in a vehicle's embedded system. For this purpose, a large language model, called an LLM, is learned (41). Third data points representing a description of the problem are received (42) from a third data collection form displayed on a touchscreen. Fourth data points representing a solution to the problem are generated (43) by the LLM receiving the third data points as input. Graphic content representing the fourth data points is displayed (44) on the touchscreen. Figure 4 (for the abstract)

Description

Title of the invention: Method and device for communicating data relating to a problem in an embedded system of a vehicle. Technical field

[0001] The invention relates to methods and devices for communicating data concerning a problem in an embedded system of a vehicle, particularly, but not exclusively, a motor vehicle. The invention also relates to a method and device for data processing to automatically generate a response to a problem detected on an embedded system of a vehicle. Technological background

[0002] Modern vehicles consist of a large number of components, organs, and embedded systems configured to assist the driver in operating the vehicle or to accompany the driver and passengers during journeys. The development of these components, organs, or embedded systems is sometimes accompanied by design or manufacturing defects, defects that are only discovered once the vehicles are in circulation.

[0003] When a problem or defect is found on such a component, organ or system of a vehicle, the driver should contact an expert to obtain feedback on the problem found, for example to find out how to correct the problem or to determine the seriousness of the problem found and whether the problem can be ignored or whether it requires repair, depending on the seriousness.

[0004] Such a process requires a fairly long processing time, which can prove dangerous for the driver and his vehicle while waiting for a response. Summary of the present invention

[0005] One object of the present invention is to solve at least one of the problems of the technological background described above.

[0006] Another object of the present invention is, for example, to improve the communication of information relating to the use of a vehicle.

[0007] According to a first aspect, the present invention relates to a method for communicating data relating to a problem of an embedded system of a first vehicle, the method being implemented by at least one processor and comprising the following steps: - learning a large language model, known as LLM, by augmented generation of retrieval from initial representative usage data of a first set of vehicles including the first vehicle and second representative data of feedback on a set of problems collected for a set of embedded systems of a second set of vehicles, the learning further including learning of an interaction format with a user of the first vehicle for a generation of a response to the problem from a determined prompt; - reception of third data representative of a description of the problem, the third data being received from a third data collection form displayed in a first graphic content of a human-machine interface, called HMI, displayed on a display screen with a touch interface, called touch screen; - generation by the LLM of fourth data points representing a response to the problem, the LLM receiving the third data points as input; - control of displaying a second graphic content representing the fourth data on the touch screen.

[0008] Using a form in a user interface displayed on a screen allows a user to easily describe a problem encountered on a vehicle's embedded system. Analysis of this description by a Learning Management System (LMS) prepared with vehicle usage data and feedback data (for example, from the vehicle manufacturer) provides a response to the problem, displaying the response to provide the driver, in real time, with the data necessary to understand the problem or the steps to take to resolve it. The driver thus has all the necessary data to understand the problem, assess its severity, and take appropriate measures to ensure the safety of the vehicle, its passengers, and other road users.

[0009] According to one variant, the process further includes a step of processing the third data by a natural language processing method, a result of the processing being provided as input to the LLM to generate the fourth data.

[0010] According to another variant, the process further comprises the following steps: - generation by the LLM of at least one adjustment query for the third data; - control of display of the at least one adjustment query on the touch screen; - reception of fifth data representing the response to the at least one adjustment query from the touch screen interface, the fourth data being further generated by the LLM based on the fifth data.

[0011] According to yet another variant, the initial data include: - representative usage data of the first vehicle obtained from an on-board network of the first vehicle; - representative usage data from a set of second vehicles of the same type as the first vehicle, obtained from the on-board networks of the set of second vehicles; and - representative usage data from a set of third vehicles obtained from on-board networks of the set of third vehicles, a driving profile of said set of third vehicles corresponding to a driving profile of said first vehicle, and the second data includes: - representative data from a user manual for the first vehicle; - representative data on faults detected for the second set of vehicles and solutions implemented to correct the faults; and - representative data of diagnostic methods proposed to users of the second set of vehicles.

[0012] According to another variant, the fourth data belong to a dataset comprising: - representative data of instructions to solve the problem; - representative data of instructions to have the problem solved by a repairman; - representative data of instructions for transmitting third data to a remote device.

[0013] According to an additional variant, a set of LLM parameters is refined based on sixth data received from the remote device in response to third data according to a reinforcement learning method.

[0014] According to yet another variant, the process further includes a step of receiving seventh data representing a request to display the first graphic content.

[0015] According to another variant, the HMI is implemented via an interactive application embedded in the vehicle or via a mobile application running on a mobile communication device.

[0016] According to a second aspect, the present invention relates to a data communication system for a problem in an embedded system of a vehicle, the system comprising at least one memory associated with at least one processor configured for the implementation of the steps of the process according to the first aspect of the present invention.

[0017] According to a third aspect, the present invention relates to a vehicle, for example a motor vehicle, comprising the system according to the second aspect of the present invention.

[0018] According to a fourth aspect, the present invention relates to a computer program which includes instructions adapted for carrying out the steps of the process according to the first aspect of the present invention, in particular when the computer program is executed by at least one processor.

[0019] Such a computer program may use any programming language, and be in the form of source code, object code, or an intermediate form between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0020] According to a fifth aspect, the present invention relates to a computer-readable recording medium on which is recorded a computer program comprising instructions for carrying out the steps of the process according to the first aspect of the present invention.

[0021] On the one hand, the recording medium can be any entity or device capable of storing the program. For example, the medium can include a storage means, such as a ROM, a CD-ROM or a microelectronic circuit-type ROM, or a magnetic recording means or a hard disk drive.

[0022] On the other hand, this recording medium can also be a transmissible medium such as an electrical or optical signal, such a signal being able to be transmitted via an electrical or optical cable, by conventional or radio frequency, by self-directing laser beam, or by other means. The computer program according to the present invention can, in particular, be downloaded from an Internet-type network.

[0023] Alternatively, the recording medium may be an integrated circuit in which the computer program is incorporated, the integrated circuit being adapted to execute or to be used in the execution of the process in question. Brief description of the figures

[0024] Other features and advantages of the present invention will become apparent from the description of the particular and non-limiting embodiments of the present invention below, with reference to the attached Figures 1 to 4, in which:

[0025] [Fig.1] schematically illustrates a data communication environment, according to a particular and non-limiting embodiment of the present invention;

[0026] [Fig.2] schematically illustrates a data communication process relating to a problem of an embedded system of a vehicle in the communication environment of [Fig.1], according to a particular and non-limiting embodiment example of the present invention;

[0027] [Fig. 3] illustrates a device configured to communicate data relating to a problem of an embedded system of a vehicle in the environment of communication of [Fig.1], according to a particular and non-limiting example of the present invention.

[0028] [Fig.4] illustrates a flowchart of the different stages of a process of communication of data relating to a problem of an embedded system of a vehicle in the communication environment of [Fig.1], according to a particular and non-limiting embodiment example of the present invention. Description of examples of achievements

[0029] A method and device for communicating data relating to a problem of an embedded system of a vehicle will now be described in what follows with joint reference to Figures 1 to 4. The same elements are identified with the same reference signs throughout the description that follows.

[0030] The terms "first," "second" (or "firsts," "seconds"), etc., are used in this document by arbitrary convention to allow for the identification and distinction of different elements (such as operations, means, etc.) implemented in the embodiments described below. Such elements may be distinct or correspond to a single element, depending on the embodiment.

[0031] Fig. 1 schematically illustrates a communication environment 1, according to a particular and non-limiting embodiment of the present invention.

[0032] According to a first embodiment, the communication environment 1 comprises a first vehicle 11. The first vehicle 11 corresponds, for example, to a vehicle with an internal combustion engine, with electric motor(s), or even a hybrid vehicle with an internal combustion engine and one or more electric motors. The first vehicle 11 thus corresponds, for example, to a land vehicle, for example a car, a truck, a bus, a motorcycle.

[0033] The first vehicle 11 corresponds to a so-called connected vehicle in that it carries a communication system configured to communicate with one or more remote devices 101 via a wireless communication network infrastructure. The remote device 101 corresponds, for example, to a server or a computer in the "cloud" 100.

[0034] The communication system of a connected vehicle includes, for example, one or more communication antennas connected to a telematic control unit, known as a TCU (Telematic Control Unit), which is itself connected to one or more computers of the connected vehicle's embedded system. The antenna(s), the TCU, and the computer(s) form, for example, a multiplexed architecture for providing various services useful for the proper functioning of the connected vehicle and for assisting the driver and / or passengers of the connected vehicle in controlling the vehicle and / or for diagnosing its operation. of one or more components of the connected vehicle. The computer(s) and the TCU communicate and exchange data with each other via one or more computer buses, for example a CAN data bus (from the English "Controller Area Network" or in French "Réseau de contrôlers"), CAN FD (from the English "Controller Area Network Flexible Data-Rate" or in French "Réseau de contrôlers à débit de données flexible"), FlexRay (according to the ISO 17458 standard) or Ethernet (according to the ISO / IEC 802-3 standard).

[0035] According to a second embodiment, the environment 1 comprises a data processing device 10 corresponding, for example, to a mobile communication device such as a smartphone or a tablet. According to another embodiment, the data processing device 10 corresponds to a laptop or a computer.

[0036] The data processing device 10 is advantageously configured to communicate with the remote device(s) 101 via the wireless communication network infrastructure.

[0037] Environment 1 further comprises a set of second vehicles 12, including one or more connected second vehicles such as the first vehicle 11. The connected second vehicles of set 12 are of the same type as the first vehicle 11; for example, the second vehicles are identical or similar to the first vehicle 11 in that they carry one or more on-board systems identical or similar to those carried in the first vehicle 11. The type of a vehicle is identified by a set of technical characteristics, the vehicle brand, and / or the vehicle designation. Thus, two vehicles of the same type correspond to vehicles of the same brand and the same designation within the brand, or, in another example, vehicles of the same brand and the same technical characteristics (same engine, same equipment, and / or same on-board systems).

[0038] The environment 1 further comprises a set of third vehicles 13 comprising one or more connected third vehicles such as the first vehicle 11. The third vehicles correspond to vehicles whose driving profile corresponds to the driving profile of the first vehicle 11. A driving profile is for example characterized by the types of journeys (city, road, highway, mixed) taken with a vehicle, the frequency of use of the vehicle, an average speed, brake wear, tire wear over time, etc.

[0039] The mobile communication infrastructure enabling wireless data communication between, on the one hand, the first vehicle 11 and / or the data processing device 11 (as well as the second vehicle(s) and / or the third vehicle(s)) and, on the other hand, each of the remote devices 101 comprises, for example one or more communication equipment 102 of the relay antenna type (cellular network). In a communication mode using such a network architecture, the data is for example transmitted by the vehicle connected to the remote device 101 of the "cloud" 100 via a relay antenna 102 (the antenna 102 being for example connected to the "cloud" 100 via a wired link and the remote device 101 being itself connected to the network infrastructure of the "cloud" 100 via a wired and / or wireless network).

[0040] The wireless communication system enabling the exchange of data between, on the one hand, the first vehicle 11 and / or the second and third vehicles and / or the data processing device 11 and, on the other hand, the remote device 101 corresponds, for example, to: - a vehicle-to-infrastructure (V2I) communication system, for example based on the 3GPP LTE-V or IEEE 802.1 lp standards of ITS G5; or - a cellular network communication system, for example an LTE (Long-Term Evolution) network, LTE-Advanced (also called LTE 3G, 4G or 5G); or - a Wifi type communication system according to IEEE 802.11, for example according to IEEE 802.1 In or IEEE 802.1 lac.

[0041] A process for communicating data relating to one or more problems detected on one or more embedded systems of the vehicle 10 is described with reference to [Fig. 2]. The term "embedded system" encompasses any system, component, or part of the first vehicle 11, as well as any function, service, or feature offered in the first vehicle 11 via an embedded system, component, or part. A problem encountered with an embedded system of the first vehicle 11 corresponds, for example, to a hardware problem or hardware failure of a component or part, a software problem (e.g., a bug) in an interactive application implemented in the first vehicle 11, behavior of an embedded system that does not conform to what is expected by the driver or a passenger, etc.

[0042] The process is, for example, implemented in a system comprising the first vehicle 11, for example by one or more processors of one or more computers such as, for example, the computer of the vehicle's infotainment system, known as the IVI (In-Vehicle Infotainment) computer, and the remote device 101, for example one or more processors of the remote device 101, the first vehicle 11 and the remote device 101 being connected in communication, for example via the communication network infrastructure of environment 1. According to an alternative embodiment, the process is implemented in a system comprising the processing device data 11, for example one or more processors of this data processing device 11, and the remote device 101, for example one or more processors of the remote device 101, the first vehicle 11 and the remote device 101 being linked in communication, for example via the communication network infrastructure of environment 1.

[0043] The communication of evaluation data is implemented via an interactive application (widget) executed by the computer, and more specifically via a human-machine interface (HMI) associated with the interactive application and displayed on a touchscreen of the first vehicle 11 under the control of the IVI computer, for example. According to alternative embodiments, part of the operations of the problem data communication process is implemented by the remote device 101 or in conjunction with the remote device 101.

[0044] When the process is implemented by the system comprising the data processing device 11, the communication of evaluation data is carried out via an application (for example, a mobile application when the data processing device is a mobile communication device), and more specifically via a human-machine interface (HMI) associated with this application and displayed on a touchscreen of the data processing device 11. In one embodiment, the application is an internet browser, for example, when the data processing device is a computer. In other embodiments, part of the operations of the problem data communication process is carried out by or in conjunction with the remote device 101.

[0045] The interactive application of the first vehicle 11 and the application, for example mobile, of the data processing device 11 are linked in communication to the remote device 101 via the wireless communication network infrastructure for the exchange of data within the framework of the generation of each request, as explained below in more detail.

[0046] Fig. 2 schematically illustrates a data communication process relating to a problem of an embedded system of the first vehicle 11, according to particular and non-limiting embodiments of the present invention.

[0047] The process is, for example, implemented by one or more processors of one or more computers of the first vehicle 11 or by one or more processors of a data processing unit of the data processing device 11, in communication with the remote device 101. In the remainder of the process description, reference will be made to a processing device corresponding to a system of the first vehicle 11 comprising one or more computers when the process is implemented in a system comprising the first vehicle 11 and the remote device 101 or corresponding to the data processing device 11 when the process is implemented in a system comprising the data processing device 11 and the remote device 101.

[0048] The process comprises three sets or blocks 21, 22, or 23, each comprising several operations of the process. The first block 21 relates to the preparation of a large language model (LLM) used to converse with a user of the processing device. The second block 22 relates to the interaction between the LLM and the processing device used by the user. The third block 23 relates to the result of the interaction and a relearning of the LLM.

[0049] In a first operation 201 of the process, a blank LLM is generated. According to one variant, the LLM is selected, for example from a list of available LLMs, and its parameters are reset to obtain a blank LLM.

[0050] A blank LLM corresponds to an LLM whose parameters are generic and which are ready to be determined to obtain an LLM dedicated to generating responses to one or more problems identified on an embedded system of the first vehicle 11.

[0051] In a second operation 202 of the process, an LLM learning is implemented to calculate the LLM parameters.

[0052] The LLM learning is obtained by the implementation 2020 of a so-called augmented generation retrieval method on the basis of first data representative of use of a first set of vehicles including the first vehicle 11 and second data representative of feedback on a set of problems collected for a set of embedded systems of a second set of vehicles.

[0053] This second operation 202 makes it possible to make available to the LLM obtained in the first operation 201 all the databases necessary to respond to the user's requests for the resolution of a problem.

[0054] Retrieval Augmented Generation, or RAG (from the English "Retrieval Augmented Generation"), is a method known to those skilled in the art, for example described in the article "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", by Patrick Lewis et al., published in NeurIPS in 2020.

[0055] RAG makes it possible to exploit data sources related to the field of vehicles for example, which makes it possible to use a generic LLM model without needing to retrain the LLM model.

[0056] The first set of vehicles includes, for example, the first vehicle 11, the set of second vehicles 12 and the set of third vehicles 13.

[0057] The second set of vehicles includes, for example, any vehicle from the manufacturer of the first vehicle 11 for which feedback is available From the manufacturer, for example, following the reporting of a fault, or maintenance carried out in a garage or dealership. The second set of vehicles includes, for example, the first set of vehicles.

[0058] The initial data include, for example: - representative usage data of the first vehicle 11 obtained from an on-board network of the first vehicle 11, the on-board network being for example of the CAN type: this data corresponds to time and event data generated by the computers of the on-board network and exchanged on the data buses forming the on-board network, this data forming a history of the driving data of the first vehicle 11; - representative usage data for the set of second vehicles 12 obtained from the on-board network, for example CAN type, of each second vehicle in the set 12: this data corresponds to time and event data generated by the on-board network computers and exchanged on the data buses forming the on-board network, this data forming a driving history of each second vehicle in the set of second vehicles 12; and - representative usage data of the set of third vehicles 13 obtained from the on-board network, for example of CAN type, of each third vehicle in the set 13: this data corresponds to time and event data generated by the computers of the on-board network and exchanged on the data buses forming the on-board network, this data forming a history of driving data of each third vehicle in the set of third vehicles 13.

[0059] The first data are for example stored on a "cloud" server 100 and collected during a determined time interval or throughout the life of the first vehicle 11 and the second and third vehicles of the sets 12 and 13. This first data are for example received by the server from each vehicle via an OTA (Over The Air) wireless link.

[0060] The second set of data includes, for example: - data representative of a user manual for the first vehicle 11, that is to say data representative of advice, good practices and procedures recommended by the manufacturer for the type of vehicle corresponding to the first vehicle 11; - representative data of faults detected for the second set of vehicles and solutions implemented to correct the faults, this data being, for example, representative of credit applications (type of fault and solution applied) to correct the fault), this data allows for a history of faults on the manufacturer's customer vehicles and a history of the solutions implemented to repair the faults; and - representative data of diagnostic methods offered to users of the second set of vehicles, i.e. methods offered to the manufacturer's customers for implementation by the customer to identify and resolve a problem, when the identified problem does not require a visit to a garage or dealership.

[0061] The second data is for example stored on one or more servers controlled by the manufacturer of the first vehicle 11.

[0062] The application of RAG to LLM allows LLM to integrate the first and second data during LLM learning.

[0063] In a third operation 203 of the process, the LLM learning continues with the learning of an interaction format with a user of the first vehicle 11 for generating a response to a problem reported by the user via the data processing device. The interaction format is learned by providing 2030 one or more prompts as input to the LLM.

[0064] This third operation 203 is consecutive to the second operation 202. According to one variant, the second operation 202 and the third operation 203 are implemented simultaneously and form only one and the same learning operation.

[0065] The prompt is for example generated by the manufacturer of the first vehicle 11 to describe to the LLM how it should behave during its interaction with the user and in what form or format it should write the report of its analysis (response to the problem raised).

[0066] A prompt corresponds, for example, to a set of instructions in natural language such as, for example: "You are an expert in automotive after-sales", "You will receive remarks based on problems on vehicle users, some examples of which are below...", "From the databases "XXX" and "YYY", identify a solution to the problem raised by the user if such a solution exists", "you will write this response in a didactic manner as in the following examples...", "If you do not find a solution you will return the keyword 'no solution' so that the interface communicates to the user that the problem has been submitted to an expert of the manufacturer", etc.

[0067] XXX and YYY correspond to identifiers of the databases storing the first and second data.

[0068] The prompt is provided as input to the LLM as an instruction to guide it in generating responses (general context, response format, response instructions according to different implementation examples) to be made to the requests of the query(ies) users during the interaction between the user, via the processing device, and the LLM.

[0069] In a fourth operation 204 of the process, third data 2040 representing a description of the problem are received. This third data 2040 is received from a third data collection form displayed in a first graphic content of a human-machine interface, referred to as the HMI, displayed on a touchscreen display, referred to as the touchscreen, of the processing device.

[0070] The display of the first graphic content including the form designed to collect the third data is triggered for example by an action of the user wishing to communicate a problem observed or detected on a system of the first vehicle 11 to start an interaction with the processing device, and optionally with the remote device 101 according to certain particular embodiments described below.

[0071] The interaction is initiated, for example, by touching an icon on the home page of the HMI displayed on the touchscreen of the processing device. The touchscreen triggers, for example, the execution of the application (interactive application, mobile application, thin or thick client, web browser) enabling the interaction.

[0072] The touch press thus triggers, for example, the display of graphic content corresponding to a home page of the application's HMI, offering, for example, the execution of one or more services by pressing on a graphic object or an icon associated with each of these services.

[0073] According to one embodiment, the initiation of the interaction is triggered by the acquisition of a voice command spoken by the user via a microphone on board the first vehicle 11 or a microphone of the data processing device 10. The microphone is connected to or is part of a voice interface of the processing device.

[0074] According to another embodiment, the initiation of the interaction is triggered by the user pressing a physical button arranged in the passenger compartment of the first vehicle 11, for example on the dashboard or steering wheel of the first vehicle 11.

[0075] According to a particular embodiment, identification of the user and / or the first vehicle 11 is required during an optional operation, for example implemented only when a request is first submitted to report a problem or when the application is first executed by the processing device.

[0076] To this end, the processing device triggers the display of graphic content requiring the identification of the first vehicle 11. The graphic content The login process includes, for example, two text fields: one for entering a username and the other for entering a password. Username and password entry are implemented via the touchscreen interface on which the login graphic content is displayed.

[0077] The identifier corresponds for example to a telephone number or an email address associated with the owner of the first vehicle 11. Such an identifier is associated with an account of the user of the first vehicle 11, the account including a set of information on the first vehicle 11 allowing the first vehicle 11 to be identified, for example the unique identification number of the vehicle, called VIN (from the English "Vehicle Identification Number").

[0078] According to another example, the identifier corresponds to the vehicle identification number, known as the VIN.

[0079] An authentication request including identification data (username and password) is transmitted to the remote device 101 which in turn transmits (to the processing device) an authorization to execute the service of generating and transmitting the embedded system improvement request.

[0080] During subsequent uses of the application, identification is done automatically at the start of the application with, for example, the transmission of identification data of the identified processing device to the processing device during the first connection.

[0081] The initiation of the interaction continues with the reception of data representing a request to display the first graphic content of the GUI, the reception of this data triggering the control of displaying the first graphic content including the form for collecting third data.

[0082] This data is, for example, generated by (or representative of) a touch input by the user on a graphic object displayed in a specific graphical content (corresponding, for example, to a home page) of the application's user interface on the touchscreen of (or associated with) the processing device. This data is received following a touch input by the user on this graphic object. This graphic object corresponds, for example, to an icon or pictogram associated with the communication service for a problem observed on an embedded system.

[0083] A touch input on the graphic object sends information to the processing device so that the latter can implement one or more functions associated with the command corresponding to the touch input. The data representing the touch input is received, for example, via one or more data buses connecting the touchscreen interface and the processing device. The touch input corresponds, for example, to a short or brief touch input where the duration of the input on the graphic object is less than a threshold duration (for example, equal to 500 or 1000 ms).

[0084] According to one embodiment, this data is generated by (or representative of) a specific voice command spoken by the user and acquired by the microphone. According to another embodiment, this data is generated by (or representative of) pressing a physical button located in the passenger compartment of the first vehicle 11, for example on the dashboard or steering wheel of the first vehicle 11.

[0085] The 2040 third data collection form includes, for example, a set of fields, for example, one or more fields allowing selection of a theme from a proposed list and / or one or more fields for entering free text, uploading one or more photos and / or videos to describe and illustrate the problem.

[0086] The form thus corresponds, for example, to a questionnaire with: - one or more questions to target a particular theme or area (for example, the type of embedded system for which the problem was observed, the type of problem, the anomalies observed), with answers to these questions being guided (via a suggested answer from a list of predetermined answers) or free-form via a free text box; and / or - a free field for the user to provide further details to the answers provided previously, if applicable, and / or describe in detail the problem encountered; and / or - a field to upload one or more photos and / or videos showing the problem.

[0087] The third data 2040 representative of the problem description thus include one or more of the following data, according to all possible combinations: - representative data from responses to one or more questions; and / or - data representative of a text; and / or - representative data from one or more images; and / or - representative data from one or more videos or image sequence(s).

[0088] The user enters the third data 2040 via the screen's touch interface. In one embodiment, the third data, or part thereof, is entered via a voice interface, with the HMI generating an audio rendering of the questions or a description of the fields by speech synthesis, and the HMI filling in the fields by acquiring the user's spoken answers via a microphone.

[0089] A display control for graphic content, such as a graphic object, the first graphic content, or any other graphic content, includes a rendering of the graphic content or graphic object, such rendering corresponding to a set of operations performed by one or more processors on the pixels of one or more images of the graphic content to be displayed on the screen. For example, the rendering consists of associating with a set of pixels in an image pixel data (for example color data expressed in an RGB type space (from the English "Red, Green, Blue" or in French "rouge, vert, bleu")) associated with each graphic object.

[0090] The display control of each graphic content or object thus includes the transmission of control signals to the screen to modify the values ​​associated with the screen pixels at the location intended to display the graphic content.

[0091] In a fifth operation of the process, an interaction with the LLM 2031 obtained from the LLM learning block 21 is implemented.

[0092] The third data point 2040 is transmitted to an artificial intelligence module for processing the second data point, referred to as the AI ​​module, implementing the LLM 2031. The LLM is also called a generative LLM agent. Such a generative LLM is implemented, for example, in the form of a neural network, as understood by those skilled in the art. The AI ​​module thus corresponds, for example, to a conversational agent.

[0093] The AI ​​module is implemented by the processing device. According to one embodiment, the AI ​​module is implemented by the remote device 101, with the third data then being transmitted by the first vehicle 11 to the remote device 101 via a wireless connection. The AI ​​module is, for example, implemented in the form of one or more neural networks.

[0094] The third data is, for example, processed or analyzed using a natural language processing method, known as NLP (Natural Language Processing), to understand the text entered by the user in the form and to make this text interpretable by the AI ​​module. When the second data includes images and / or videos, the third data relating to these images and / or videos is processed to extract features from these images and / or videos in order to classify them according to their content.

[0095] According to a particular embodiment, the interaction with the LLM 2031 comprises the following operations: - generation by LLM 2031 of at least one third-party data adjustment query; - display control of at least one adjustment request on the touch screen of the processing device; - reception of fifth representative data responses to at least one adjustment request from the touchscreen interface.

[0096] According to this particular embodiment, such an interaction allows the LLM 2031 to request the user, via the touchscreen interface of the processing device, to specify or clarify their request regarding the description of the identified problem. A third-party data adjustment request is a request asking the user to specify the problem description or clarify certain aspects described via the third-party data collection form.

[0097] The LLM thus generates a set of adjustment queries or questions for the user so that the latter completes or modifies the information entered in the form and corresponding to the third data.

[0098] The generated question(s) are, for example, displayed on the screen, with a text box provided next to each displayed question so that the user can enter the answer via the touch interface. In one embodiment, this question or these questions are rendered by speech synthesis, and the answers are acquired by the microphone and interpreted via NLP.

[0099] The interaction with the conversational agent includes one or more exchanges with the generation of one or more questions at each exchange, until the LLM 2031 considers that the quality of the content of the description of the problem is sufficient, for example above a determined threshold.

[0100] The interaction with the LLM 2031 results in the generation by the LLM 2031 of fourth data representing a response to the problem as a function of the third data describing the problem, which third data are provided as input to the LLM 2031. The fourth data are, for example, further generated as a function of the fifth data when one or more fitting queries have been generated by the LLM 2031.

[0101] The generation of the fourth data then triggers a sixth operation 205 of the process according to which the display of a second graphic content representative of the response is controlled in such a way that this second graphic content is displayed on the touch screen of the processing device, for the user.

[0102] In a first embodiment 2051, the response generated by the LLM corresponds to advice provided to the user so that the user can resolve the reported problem themselves. Such a response might consist, for example, of text, optionally accompanied by graphic visuals (photos, drawings, pictograms) explaining the steps to be taken to resolve the problem. The fourth data point thus represents instructions for resolving the problem.

[0103] In this first example of embodiment 2051, the problem reported by the user corresponds, for example, to a simple problem that the user can solve on their own. According to this first example of embodiment 2051, the LLM 2031 is also able to provide an answer to the problem described via the third data. The LLM 2031 is, for example, suitable for providing an answer when the data used during the LLM learning were related to the same problem or a similar or related problem and / or related to the same embedded system.

[0104] An example of such a problem corresponds, for example, to a problem associated with the central locking of the first vehicle 11, following a visit to the dealer, where only the driver's door opens according to the door opening command instead of all the doors of the first vehicle 11. The response provided by LLM 2031 instructs the user to modify the options of the central locking system via the HMI of the first vehicle 11 displayed on the touch screen by activating the option to open all doors upon receipt of the opening command.

[0105] In a second embodiment 2052, the fourth data points represent instructions for having the problem resolved by a repairer, for example, a garage or a car dealership. In such an embodiment, the LLM is aware of the problem, either because the problem has already been reported via one or more other vehicles (from the set of second vehicles 12 and / or the set of third vehicles 13), or via the manufacturer's product knowledge, this problem then being described in the databases accessible by the LLM 2031.

[0106] Such a response corresponds, for example, to text, optionally accompanied by graphic visuals (photos, drawings, pictograms). Optionally, the response includes a graphic object offering the user the option to trigger the calculation of a route via a navigation system (from the first vehicle 11 or the data processing device 10) to the nearest garage.

[0107] In a third embodiment 2053, the fourth data points correspond to data intended to be transmitted to a remote device such as the remote device 101. Thus, when the LLM 2031 is unable to generate instructions to solve the problem, the LLM generates an instruction to transmit the third data points to the remote device 101 via a wireless connection. The remote device corresponds, for example, to a server belonging to the manufacturer, an embedded system manufacturer, or an automotive expert, which in turn provides the fourth data points representing the response. These fourth data points are, for example, entered via an HMI implemented on the remote device, the fourth data points being received by the processing device via the wireless connection linking the processing device to the remote device, for example, via the wireless communication infrastructure described opposite [Fig. 1].

[0108] The fourth data generated by the LLM 2031 includes, for example, additional text to be displayed on the touch screen in this third embodiment 2053, this text explains to the user that the description of the problem has been sent to an expert and that the system is waiting for the expert's response.

[0109] The response received from the remote device 101 corresponds, for example, to: - instructions 2054 provided to the user and displayed on the touchscreen so that they can solve the problem themselves, as in the first embodiment example 2051; or - instructions 2055 provided to the user and displayed on the touch screen inviting the user to contact a garage or dealer to have the problem corrected or repaired, as in the second embodiment example 2052.

[0110] According to a particular embodiment, the representative data of the response received from the remote device 101, referred to as the sixth data point, as well as the third data point describing the associated problem, are used to refine the parameters of the LLM using a reinforcement learning method. This data is provided to the RAG so that it can feed the LLM, in addition to the other data, in the second operation 202 of the process. This allows the LLM to be continuously improved so that it can provide more responses without relying on the remote device 101 as it is used in real-time production conditions.

[0111] According to a particular embodiment, a summary document is generated Based on all the reported problems and responses, the data representing this summary document is automatically transmitted electronically to one or more data processing devices such as computers or tablets, to be submitted to individuals identified as recipients of the summary document. For example, the summary document is automatically generated by LLM 2031 or another LLM.

[0112] Figure 3 schematically illustrates a device 3 of a system configured for communicating data relating to a problem in an embedded system of a vehicle, for example, the first vehicle 11, according to a particular and non-limiting embodiment of the present invention. The device 3 corresponds, for example, to a device embedded in the vehicle, for example, a computer. According to another example, the device 3 corresponds to a data processing device, for example, a mobile communication device. According to yet another example, the device 3 corresponds to a server.

[0113] Device 3 is, for example, configured to carry out the operations described opposite Figures 1 to 2 and / or the steps of the process described opposite [Fig. 4]. Examples of such a device 3 include, but are not limited to, embedded electronic equipment such as a vehicle's on-board computer, an electronic control unit such as an ECU (Electronic Control Unit), a A smartphone, a tablet, a laptop, a computer, a server. The elements of device 3, individually or in combination, can be integrated into a single integrated circuit, into several integrated circuits, and / or into discrete components. Device 3 can be implemented as electronic circuits or software (or computer) modules, or a combination of electronic circuits and software modules.

[0114] The device 3 comprises one (or more) processor(s) 30 configured to execute instructions for carrying out the steps of the process and / or for executing instructions from the software embedded in the device 3. The processor 30 may include integrated memory, an input / output interface, and various circuits known to those skilled in the art. The device 3 further comprises at least one memory 31, for example, volatile and / or non-volatile memory, and / or includes a memory storage device that may include volatile and / or non-volatile memory, such as EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk, or optical disk.

[0115] The computer code of the embedded software(s) including the instructions to be loaded and executed by the processor is for example stored on memory 31.

[0116] According to a particular and non-limiting embodiment, the device 3 comprises a block 32 of interface elements for communicating with external devices such as connected vehicles and / or measuring devices. The interface elements of the block 32 comprise one or more of the following interfaces: - radio frequency RF interface, for example of the Wi-Fi® type (according to IEEE 802.11), for example in the 2.4 or 5 GHz frequency bands, or of the Bluetooth® type (according to IEEE 802.15.1), in the 2.4 GHz frequency band, or of the Sigfox type using UBN (Ultra Narrow Band) radio technology, or LoRa in the 868 MHz frequency band, LTE (Long-Term Evolution), LTE-Advanced; - USB interface (from the English "Universal Serial Bus" or "Universal Serial Bus" in French); - HDMI interface (from the English "High Definition Multimedia Interface", or "High Definition Multimedia Interface" in French); - LIN interface (from the English "Local Interconnect Network", or in French "Réseau interconnecté local").

[0117] According to another particular and non-limiting embodiment, the device 3 includes a communication interface 33 which enables communication with other devices (such as other servers, databases) via a communication channel 330. The communication interface 33 corresponds, for example, to a A transmitter configured to transmit and receive information and / or data via communication channel 330. The communication interface 33 corresponds, for example, to a wired Ethernet network (standardized by ISO / IEC 802-3).

[0118] In a particular, non-limiting embodiment, the device 3 can provide output signals to one or more external devices, such as a display screen 340, touchscreen or not, one or more loudspeakers 350, and / or other peripherals 360 (projection system), respectively, via output interfaces 34, 35, and 36. In one variant, one or more of the external devices is integrated into the device 3.

[0119] Figure 4 illustrates a flowchart of the different stages of a method for communicating data relating to a problem in an embedded system of a vehicle, for example, the first vehicle 11, according to a particular and non-limiting embodiment of the present invention. The method is, for example, implemented by one or more processors of a system comprising one or more devices such as device 3 in Figure 3.

[0120] In a first step 41, a large language model, called LLM, is learned by augmented generation of retrieval from first data representative of use of a first set of vehicles including the first vehicle and second data representative of feedback on a set of problems collected for a set of embedded systems of a second set of vehicles, the learning of the LLM further including learning of an interaction format with a user of the first vehicle for a generation of a response to the problem from a determined prompt.

[0121] In a second step 42, third data representing a description of the problem are received from a third data collection form displayed in a first graphic content of a human-machine interface, called HMI, displayed on a touch interface display screen, called touch screen.

[0122] In a third step 43, fourth data representing a response to the problem are generated by the LLM, the LLM receiving the third data as input.

[0123] In a fourth step 44, a display of a second graphic content representative of the fourth data is controlled so that this second graphic content is displayed on the touch screen.

[0124] According to one variant, the variants and examples of the operations described in relation to one of Figures 1 to 2 apply to the steps of the process in [Fig.4].

Claims

Demands

1. A method for communicating data relating to a problem of an embedded system of a first vehicle (11), said method being implemented by at least one processor and comprising the following steps: - learning (41; 202) of a large language model, called LLM (2031), by augmented generation of retrieval from first data representative of use of a first set of vehicles including said first vehicle (11) and second data representative of feedback on a set of problems collected for a set of embedded systems of a second set of vehicles, said learning further comprising learning (203) of an interaction format with a user of said first vehicle (11) for a generation of a response to said problem from a determined prompt;- reception (42) of third data (2040) representing a description of said problem, said third data (2040) being received from a form for collecting said third data displayed in a first graphic content of a human-machine interface displayed on a touchscreen display, said touchscreen; - generation (43) by said LLM (2031) of fourth data representing a response to said problem, said LLM (2031) receiving as input said third data (2040); - control (44) of displaying a second graphic content representing the fourth data on said touchscreen.

2. A method according to claim 1, further comprising a step of processing said third data by a natural language processing method, a result of said processing being provided as input to said LLM (2031) to generate said fourth data.

3. A method according to claim 1 or 2, further comprising the following steps: - generation by said LLM (2031) of at least one adjustment request for said third data; - display control of said at least one adjustment request on said touch screen; - receipt of fifth data representing response to said at least one adjustment request from the touch interface of said touch screen, said fourth data being further generated by said LLM (2031) based on said fifth data.

4. A method according to any one of claims 1 to 3, wherein said first data comprises: - representative usage data of said first vehicle (11) obtained from an on-board network of said first vehicle (11); - representative usage data of a set of second vehicles (12) of the same type as said first vehicle (11) obtained from on-board networks of said set of second vehicles (12); and - representative usage data of a set of third vehicles (13) obtained from on-board networks of said set of third vehicles, a driving profile of said set of third vehicles (13) corresponding to a driving profile of said first vehicle (11), and said second data comprises: - representative data from an owner's manual of said first vehicle (11);- representative data of faults detected for said second set of vehicles and of solutions implemented to correct the faults; and - representative data of diagnostic methods proposed to users of the second set of vehicles.

5. A method according to any one of claims 1 to 4, wherein said fourth data belong to a data set comprising: - data (2051) representing instructions for solving the problem; - data (2052) representing instructions for having the problem solved by a repairer; - data (2053) representing instructions for transmitting said third data to a remote device.

6. A method according to claim 5, wherein a set of parameters of said LLM is fine-tuned as a function of sixth data points received from said remote device in response to said third data according to a reinforcement learning method.

7. A method according to any one of claims 1 to 6, further comprising a step of receiving seventh data representing a request to display said first graphic content.

8. A method according to any one of claims 1 to 7, wherein said HMI is implemented via an interactive application embedded in said first vehicle (11) or via a mobile application running on a mobile communication device (10).

9. Computer program comprising instructions for carrying out the method according to any one of the preceding claims, when such instructions are executed by a processor.

10. A data communication system relating to a problem of an embedded system of a first vehicle (10), said system comprising at least one memory (31) associated with at least one processor (30) configured for the implementation of the steps of the method according to any one of claims 1 to 8.

Images