Method and apparatus for targeted diagnostic data collection

Abstract

A method of using natural language processing on an input sequence provided by a user describing a symptom being experienced by the user relating to a vehicle issue, and the automatic identification o

Description

13 03 25 TECHNICAL FIELD The present disclosure relates to a method and apparatus for targeted diagnostic data collection on a 5 vehicle. In particular, the present disclosure relates to a method and apparatus for targeted diagnostic data collection on a vehicle in response to a natural language description of a vehicle operation characteristic. Aspects of the invention relate to a method, to an apparatus, and to a computer readable medium. 10 BACKGROUND Modern vehicles include a number of computerized sub-systems that are able to generate diagnostic signals and / or fault codes, such as diagnostic trouble codes (DTC), Controller Area Network (CAN) bus signals, etc. Such signals can be accessed during servicing or maintenance of a vehicle to allow any vehicle issues to be detected and any appropriate remedial activity undertaken in response. 15 Some vehicles may be connected to a wireless communication network and allow the generated diagnostic signals to be transmitted, for example to a maintenance server, during normal operation of the vehicle by the user. This may allow any vehicle issues to be detected without the vehicle being inspected at a dealership or maintenance facility. However, transmission of all diagnostic signals from 20 a vehicle may result in significant use of wireless network resources. Furthermore, the volume of data that would be received from a fleet of vehicles may require significant storage and processing resources to handle all of the diagnostic data received from each vehicle. It is an aim of the present invention to address one or more of the disadvantages associated with the 25 prior art. SUMMARY OF THE INVENTION Aspects and embodiments of the invention provide a method, an apparatus, and a computer readable medium as claimed in the appended claims. 30 According to an aspect of the present invention there is provided a computer implemented method comprising obtaining an input sequence, the input sequence comprising a sequence of words describing a symptom associated with a vehicle issue, processing the input sequence using a natural language processing, NLP, engine to classify the input sequence according to at least one of an on-vehicle 35 location associated with the vehicle issue and an event associated with the vehicle issue, and triggering collection of targeted diagnostic data based on the at least one of the on-vehicle location and the event. Advantageously, the computer implemented method provides for a user to describe in their own words a symptom associated with a vehicle issue being experienced by the user, this natural language 40 description can then be processed to determine a component or location on the vehicle, an event and / or a vehicle state associated with the vehicle issue, e.g. an issue may be localized to the brakes, while the 13 03 25 engine is on. Collection of targeted diagnostic data can then be based on the determined location, event, and / or vehicle state, e.g. diagnostic data associated with the brakes during engine on can be collected. This reduces the amount of irrelevant diagnostic data to be monitored, thereby reducing the amount of data transmitted from the vehicle and that must be managed by the diagnostic systems. 5 The method further comprises receiving the targeted diagnostic data. Advantageously, the targeted diagnostic data can be received once triggered to allow further processing and identify any underlying vehicle issue. 10 Optionally, the method further comprises parsing the received targeted diagnostic data to identify a diagnostic code, and in response to determining that no diagnostic code is present in the targeted diagnostic data generating a prompt to the user to request a second input sequence, obtaining the second input sequence comprising a second sequence of words describing the symptom, and wherein 15 processing the input sequence using the NLP engine further comprises processing the second input sequence using the NLP engine to classify the second input sequence according to at least one of a second on-vehicle location associated with the vehicle issue, a second event associated with the vehicle issue, and a second vehicle state associated with the vehicle issue, and triggering collection of second targeted diagnostic data based on the at least one of the second on-vehicle location, second 20 event, and second vehicle state. Advantageously, the method is operable, in the case that the received targeted diagnostic data provides insufficient information to identify any underlying cause (i.e. vehicle issue), to prompt a user for further or more detailed information / description of the vehicle issue and to attempt to collect further targeted 25 diagnostic data to allow the issue to be automatically diagnosed. The method further comprises parsing the received targeted diagnostic data to identify a diagnostic code, generating an indication of a detected vehicle issue based on the diagnostic code, and outputting the indication of the detected vehicle issue. 30 Advantageously, parsing the received targeted diagnostic data allows for actual identification of the vehicle issue causing the symptom on the basis of the received diagnostic data. Information identifying the detected issue can then be provided to the user orto a mechanic / garage to aid repair of the vehicle. 35 Optionally, the method further comprises outputting the indication of the detected vehicle issue to a user, the indication including adescription ofthe issue, and providing a prompt to the userto confirm the issue based on the included description. Advantageously, the user can be provided with a description ofthe identified vehicle issue along with a 40 description of associated symptoms or other instructions to confirm the issue has been correctly 13 03 25 identified and the user asked to confirm whether the identification is correct to ensure accurate diagnosis. The method further comprises determining an action to be performed to mitigate the detected vehicle 5 issue, and outputting an indication to a user to perform the determined action. Advantageously, following targeted diagnostic data collection, a diagnostic code could be detected which is associated with an action to be performed by the user to mitigate / fixthe vehicle issue. Providing an indication of the determined action may allow a user to quickly and easily address the issue without 10 requiring a visit to a garage, e.g. an indication could be provided to check tyre wear. Optionally, processing the input sequence using the NLP engine comprises performing multiclass classification of the input sequence, wherein the multiclass classification includes a default class. 15 Advantageously, the use of a default class allows for classification of the symptom description when insufficient information is provided to fully classify one of the on-vehicle location, vehicle state, event, e.g. example where user doesn’t clearly describe where the vehicle issue exists. Furthermore, the use of a default class may avoid the algorithm misclassifying the symptom description into one of the available classes in the absence of a default class. 20 Optionally, the method further comprises in response to the multiclass classification of the input sequence classifying the input sequence as having one of the on-vehicle location, event, or vehicle state in the default class, generating a prompt to the user to request a third input sequence, obtaining the third input sequence comprising a third sequence of words describing the symptom, and processing the 25 third input sequence using the NLP engine to generate an output comprising a further classification for the at least one of on-vehicle location associated with the vehicle issue, event associated with the vehicle issue, and vehicle state associated with the vehicle issue was classified in the default class. Advantageously, when insufficient information is provided in the description of the symptom to fully 30 classify the on-vehicle location, event and vehicle state, the user may be prompted for further information describing the vehicle issue location, car state, etc. For example, the user may identify an issue with “the handle” which could potentially relate to multiple components (interior / exterior door handles, seat adjustment handle, etc.) and further information may allow accurate identification of the relevant component. 35 Optionally, obtaining the input sequence comprises obtaining an audio signal, and performing speech-to-text conversion on the audio signal to generate the input sequence. Advantageously, the use ofspeech-to-text further increases the convenience for the user when reporting 40 a fault symptom by allowing the user to verbally announce the issue which can then be 13 03 25 recorded / captured by a microphone associated with the vehicle user input device and converted to an suitable input for the natural language processing engine. Optionally, the input sequence is obtained via text entry to one of an application executed on a mobile 5 device, a web interface, and a vehicle user input device. Advantageously, the user entry of the natural language symptom description can be achieved via multiple interfaces, allowing a user to interface with the invention via a mobile device or laptop as well as via a vehicle user input device. 10 Optionally, the NLP engine comprises a neural network including a pre-trained NLP algorithm comprising one of: a Bidirectional Encoder Representations from Transformers, BERT, model; a Generative Pre-trained Transformer, GPT, model; Text-to-Text Transfer Transformer, T5, model; or another large language model. 15 Advantageously, a wide range of pre-trained BLP algorithms can be used and have found to be accurate and easily adaptable to vehicle issue identification based on an input sequence including a description of a symptom associated with the vehicle issue. 20 Optionally, the neural network further comprises a classification layer. Advantageously, the neural network structure leverages available pre-trained NLP algorithms which can be fine-tuned using the classification layer to the targeted use case This reduces the burden of training the NLP algorithm while provide high accuracy in understanding the natural language descriptions 25 received from the user. A single pre-trained NLP algorithm may be used for different tasks when combined with different classification layers. Optionally, the natural language processing engine further comprises a tokenizer to receive the input sequence and generate a vector representation of the sequence of words for input to the pre-trained 30 NLP algorithm, wherein the tokenizer is configured to recognize automotive specific words in the input sequence. Advantageously, use of a tokenizer adapted to recognize automotive specific words and abbreviations may significantly improve the accuracy of the NLP engine in understanding and classifying natural 35 language descriptions of vehicle issue symptoms. Optionally, the neural network further comprises a prompt-table comprising a task specific set of virtual tokens comprising at least one virtual token to be embedded in the vector representation of the sequence of words, the task specific set of virtual tokens to cause the pre-trained NLP algorithm to 40 classify the input sequence as the predicted vehicle issue of the set of vehicle issues. 13 03 25 Advantageously, the incorporation of a task specific set of virtual tokens to be included in the vector representation of the input sequence allows the response of the pre-trained NLP algorithm to be influenced to provide a desired output type. The use of a prompt-table may be particularly appropriate for generative NLP algorithms. 5 Optionally, processing the input sequence using the NLP engine further comprises mapping the input sequence to a first embedding vector, the first embedding vector representing a semantic meaning of the natural language description of the symptom associated with the vehicle issue, obtaining a plurality of second embedding vectors, each second embedding vector corresponding to one of the vehicle 10 issues from the set of vehicle issues for the vehicle and representing a semantic meaning of a description a symptom associated with the vehicle issue, and wherein classifying the input sequence comprises, for each second embedding vector of the plurality of second embedding vectors, calculating a similarity score to the first embedding vector and selecting a vehicle issue corresponding to the second embedding vector having the highest similarity score. 15 Advantageously, the use of embedding vectors allows the output of a generative NLP algorithm to be constrained to one of a set of specified classes, or vehicle issues, based on a similarity between the description of the symptom provided by the userand known descriptions of vehicle issue symptoms. 20 Optionally, the NLP engine is trained to predict a classification of an on-vehicle location, event and vehicle status associated with a natural language description of a symptom associated with a vehicle issue using a set of training data comprising training example input / output pairs of natural language descriptions of symptoms and associated on-vehicle location, event and vehicle status associated with the vehicle issue. 25 Advantageously, the natural language processing engine can be trained using training examples of natural language description of symptoms and actual on-vehicle location, event and status associated with the vehicle issue to provide high accuracy in the classification. Real user inputs may be captured and used to help train the NLP engine. 30 According to another aspect of the invention, there is provided an apparatus comprising a processor, and a memory comprising computer program instructions that when executed by the processor cause the apparatus to obtain an input sequence, the input sequence comprising a sequence of words describing a symptom associated with a vehicle behaviour, process the input sequence using a natural 35 language processing, NLP, engine to generate an output comprising at least one of an on-vehicle location associated with the vehicle issue, an event associated with the vehicle issue, and a vehicle state associated with the vehicle issue, and trigger collection of targeted diagnostic data based on the at least one of the on-vehicle location, event, and vehicle state. 40 Advantageously, the apparatus is operable to receive a description of a symptom associated with a vehicle issue being experienced by the user in the users own words, this natural language description 13 03 25 can then be processed by the apparatus to determine a component or location on the vehicle, an event and / or a vehicle state associated with the vehicle issue, e.g. an issue may be localized to the brakes, while the engine is on. Collection of targeted diagnostic data can then be based on the determined location, event, and / or vehicle state, e.g. diagnostic data associated with the brakes during engine on 5 can be collected. This reduces the amount of irrelevant diagnostic data to be monitored, thereby reducing the amount of data transmitted from the vehicle and that must be managed by the diagnostic systems. This disclosure relates to a method of training a natural language processing engine for classifying an 10 input sequence comprising a sequence of words describing a symptom associated with a vehicle issue as an predicted vehicle issue of a set of vehicle issues, the natural language processing engine comprising a neural network including a pre-trained natural language processing, NLP, algorithm, and a classification layer, wherein the method comprises receiving a set of training data to train the natural language processing engine, the set of training data comprising training example input / output pairs of 15 natural language descriptions of symptoms and associated on-vehicle location, event and vehicle status associated with the issue, inputting the natural language description from one or more of the input / output pairs from the set of training data to obtain a predicted classification result, characterizing an error between the predicted classification result and the on-vehicle location, event and vehicle status corresponding to the natural language description of the training set, and using an optimisation algorithm 20 to update weights of the neural network based on the characterized error. Advantageously, the method of training a natural language processing engine allows fortraining of the natural language processing engine to receive a user’s description of a vehicle issue in their own words and to determine / classify the user’s description as relating to a particular on-vehicle location, event, 25 vehicle status associated with the occurrence of the vehicle issue. The NLP engine trained this way can be equally robust to the use of technical automotive language and acronyms or more generic nontechnical language and is therefore applicable to users of all skill / knowledge levels. Optionally, the pretrained NLP algorithm comprises one of: a Bidirectional Encoder Representations 30 from Transformers, BERT, model; a Generative Pre-trained Transformer, GPT, model; Text-to-Text Transfer Transformer, T5, model; or another large language model. Advantageously, a wide range of pre-trained BLP algorithms can be used and have found to be accurate and easily adaptable to vehicle issue identification based on an input sequence including a description 35 of a symptom associated with the vehicle issue. Optionally, the method of training a natural language processing engine further comprises performing fine-tuning training of the pre-trained NLP algorithm using a training data set comprising a corpus of automotive-related text. 40 13 03 25 Advantageously, further training of the pre-trained NLP algorithm using automotive specific language text may increase the accuracy of the NLP algorithm when processing an input sequence including automotive terms, acronyms, and vehicle specific names used by the user in the natural language description. 5 Optionally, the neural network further comprises a tokenizerto receive the input sequence and generate a token sequence, the tokenizer adapted to recognize automotive related terms in the input sequence. Advantageously, use of a tokenizer adapted to recognize automotive specific words and abbreviations 10 may significantly improve the accuracy of the NLP engine in understanding and classifying natural language descriptions of car issues / behaviours. Optionally, the neural network further comprises a prompt-table comprising a task specific set of virtual tokens comprising at least one virtual token to be embedded in the vector representation of the 15 sequence of words, the task specific set of virtual tokens to cause the pre-trained NLP algorithm to classify the input sequence as the predicted vehicle issue of the set of vehicle issues, and wherein the method of training the NLP engine further comprises training a prompt encoder neural network to generate the task specific set of virtual tokens and storing the task specific set of virtual tokens in the prompt-table. 20 Advantageously, the incorporation of a task specific set of virtual tokens to be included in the vector representation of the input sequence allows the response of the pre-trained NLP algorithm to be influenced to provide a desired output type. The use of a prompt-table may be particularly appropriate for generative NLP algorithms. 25 According to another aspect of the invention, there is provided a vehicle including the apparatus as described above. According to another aspect of the invention, there is provided computer readable instructions, which 30 when executed by a computer, are arranges to perform a method as described above. Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and / or in the following description and drawings, and in particular the individual features thereof, may be taken independently 35 or in any combination. That is, all embodiments and / or features of any embodiment can be combined in any way and / or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and / or incorporate any feature of any other claim although not originally claimed in that manner. 40 13 03 25 BRIEF DESCRIPTION OF THE DRAWINGS One or more embodiments ofthe invention will now be described, byway of example only, with reference to the accompanying drawings, in which: 5 Figure 1 shows a system suitable for implementing embodiments ofthe invention; Figure 2 illustrates a method of targeted diagnostic data collection on a vehicle according to embodiments ofthe invention; 10 Figure 3 illustrates operation of a method of targeted diagnostic data collection on a vehicle according to embodiments ofthe invention; Figure 4 is a flowchart illustrating a method of targeted diagnostic data collection according to some embodiments ofthe invention; 15 Figure 5 illustrates a computer system suitable for implementing a method of targeted diagnostic data collection on a vehicle in accordance with embodiments ofthe invention; Figure 6 illustrates a natural language processing engine according to embodiments ofthe invention; 20 Figure 7 illustrates a method of training a natural language processing engine according to embodiments ofthe invention; Figure 8 illustrates operation of a method of training a natural language processing engine according to 25 embodiments of the invention; and Figure 9 illustrates a further natural language processing engine according to embodiments of the invention. 30 DETAILED DESCRIPTION According to embodiments ofthe invention, targeted diagnostic data collection can be performed on a vehicle in response to a natural language description of one or more vehicle operating characteristics, or symptoms, associated with a vehicle issue being experienced by a user. The natural language description ofthe symptom can be processed using a natural language processing algorithm to classify 35 the symptom as associated with one or more of an on-vehicle location associated with the vehicle issue, an event associated with the vehicle issue, and a vehicle state associated with the vehicle issue. Diagnostic data collection can then be targeted based on the at least one ofthe on-vehicle location, event, and vehicle state to capture diagnostic data relevant to the vehicle issue described by the user. The collected diagnostic data may then be used to diagnose any issue present on the vehicle and, if 40 necessary, arrangements may be made for repair of the vehicle. 13 03 25 A system in accordance with an embodiment of the present invention is described herein with reference to the accompanying Figure 1. With reference to Figure 1, a vehicle 100 communicates with one or more software applications. Software applications may be executed in a cloud environment, on one or more servers connected to a network, or may be executed by a processor of the vehicle 100. In the case that 5 the software applications are not executed on the vehicle 100, communication between the vehicle and the software applications may be via a suitable communication link such as a 4G or 5G mobile communications network. The software applications include a natural language processing engine 110. A user of the vehicle 100 may optionally provide an input to the natural language processing engine 110 via a computing device 120 or mobile terminal 130 that communicate with the natural language 10 processing engine via a network. In some embodiments, the user may supply the input via an in-vehicle infotainment system or via a voice command. Figure 2 illustrates a computer implemented method 200 according to an embodiment of the present invention that can be implemented in the system illustrated in Figure 1. Figure 3 further illustrates 15 operation of method 200 in accordance with an embodiment of the invention. The method 200 begins in block 210. In block 210, the method 200 comprises obtaining an input sequence 310 comprising a sequence of words describing a symptom associated with a vehicle issue. The user may provide a description of an issue in their own words. As may be appreciated, users of 20 different levels of knowledge and experience may provide significantly different descriptions of the same issue. Some users with high-levels of domain knowledge may provide accurate and specific descriptions using precise technical terms, while less knowledgeable users may provide more ambiguous descriptions using more generic language. 25 A user may provide the input sequence 310 via text entry into a vehicle user input device, such as via an infotainment screen, an application executed on a mobile device 130, via a web interface using a computer 120, etc. or may provide the input sequence as an oral description, for example recorded as an audio signal using a microphone provided as part of an in-vehicle infotainment system. In the case of a spoken input, a speech-to-text algorithm may be used to convert the audio input signal into a text 30 input sequence. In block 220, the method 200 comprises processing the input sequence 310 using a natural language processing engine 110. The natural language processing engine 120 classifies the input sequence 310 as relating to one or more of an on-vehicle location associated with the vehicle issue, an event 35 associated with the vehicle issue, and a vehicle state associated with the vehicle issue. In response to classifying the input sequence 310 as associated with at least one of the on-vehicle location, event, and vehicle state, the natural language processing engine 110 provides an output 320 including an indication of the identified on-vehicle location 330, event 340, and / or vehicle state 350. In particular, the natural language processing engine 110 may be operable to correctly classify different descriptions of a 40 particular issue provided by users of different levels of knowledge in their own words. 13 03 25 The natural language processing, NLP, engine 110 may comprise a neural network trained to predict at least one of an on-vehicle location, an event, and a vehicle state associated with a natural language description of a symptom relating to a vehicle issue using a set of training data, the training data comprising training example input / output pairs of natural language descriptions of symptoms and 5 associated on-vehicle locations, events, and vehicle states. Further details regarding natural language processing engines suitable for classifying a sequence of words describing a symptom associated with a vehicle issue, and the training of such natural language processing engines are discussed below. In block 230, the method comprises triggering collection of targeted diagnostic data based on the at 10 least one of the on-vehicle location, event, and vehicle state predicted by the natural language processing engine 110. For example, a diagnostics application associated with NLP engine 110 may transmit a message to vehicle 100 including the predicted on-vehicle location, event, and / or vehicle state to cause a diagnostic system of the vehicle 100 to monitor for diagnostic codes relating to the identified on-vehicle location, or sub-component of the vehicle, when the identified event occurs, and 15 when the vehicle is in the predicted state. The collected diagnostic data may then be transmitted to a diagnostics monitoring application via the wireless communication link for use in diagnosing the vehicle issue being experienced by the user. 20 In some embodiments, the targeted diagnostic data received by the diagnostics monitoring application may be parsed to identify any diagnostic codes present in the data. Identified diagnostic codes may then be used to identify the vehicle issue. In embodiments, the vehicle issue may be automatically identified based on any diagnostic codes present in the targeted diagnostic data, and an output may be provided to indicate the identified vehicle issue. A description of the identified vehicle issue may be output to the 25 user and a prompt provided to request the user to confirm the identified vehicle issue matches the user’s experience. In some embodiments, an identified vehicle fault may be associated with a particular mitigation activity to be performed by the user to mitigate the detected vehicle issue. For example, the user may be 30 prompted to check the state of the tyres of the vehicle for wear. In the case that no diagnostic codes are present in the targeted diagnostic data, a prompt may be generated to prompt the user to provide a second input sequence to provide a further description of the vehicle issue being experienced by the user. The second input sequence may then be processed using 35 the NLP engine 110, as discussed above, for the initial input sequence to predict at least one of a second on-vehicle location, a second event, and a second vehicle state. The vehicle 100 may then be triggered to collect second targeted diagnostic data based on the at least one of the second on-vehicle location, the second event, and the second vehicle state. 13 03 25 Thus, in the case that no relevant diagnostic codes are found in the targeted diagnostic data, a further attempt to collect targeted diagnostic data can be performed based on a new input sequence describing the vehicle issue in the user’s own words. 5 In some embodiments, the NLP engine 110 may be arranged to classify one or more of the on-vehicle location, event and vehicle status associated with the input sequence in a default class, indicating that insufficient information is available to the NLP engine 110 to allow for an accurate prediction to be made. In response to one or more of the on-vehicle location, event, and vehicle state being classified in the 10 default class, a prompt may be generated to the user to request a third input sequence describing the symptom associated with the vehicle issue. For example, in the case that the NLP engine 110 classifies the input sequence as associated with a default event class, a prompt may be provided asking for more information on “when” the symptom is experienced by the user, similarly when an on-vehicle location is classified as default, the user may be prompted for more information as to “where” the symptom is 15 present. The third input sequence may then be processed using the NLP engine 110, as discussed above, for the initial input sequence to predict at least one of a third on-vehicle location, a third event, and a third vehicle state. The vehicle 100 may then be triggered to collect third targeted diagnostic data based on 20 the at least one of the third on-vehicle location, the third event, and the third vehicle state. Thus, the method 200 provides for automatic and remote diagnostic data collection targeted at a vehicle issue being experienced by a user of the vehicle 100. Furthermore, the collection of targeted diagnostic data can be performed without requiring involvement of a technician and with less inconvenience to the 25 user than associated with taking the vehicle to a dealership to have the issue diagnosed. In some embodiments, collected diagnostic data may be stored in a maintenance log associated with the vehicle 100 or provided to a local maintenance facility to arrange for repair of the vehicle 100. Thus, when a user contacts the maintenance facility to arrange repair, the maintenance facility may already 30 be aware of the vehicle issue and have a record of diagnostic data received from the vehicle, and may have taken steps to prepare in advance of the user making contact, for example by ordering an associated spare part. While natural processing engine 110 has been illustrated as a single entity in Figure 1, it will be 35 appreciated that the application may be implemented as software modules that may be co-located on a single server or hosted in a cloud environment. Furthermore, the functionality of the natural processing engine 110 and diagnostics monitoring application could be combined into a single application, or further divided into specialized application modules. 40 Figure 4 is a flowchart illustrating a method 400 of targeted diagnostic data collection according to some embodiments of the invention. In block 410 an input sequence comprising a description of a vehicle 13 03 25 issue, or symptoms experienced by the user, is provided by the user in their own words. In block 420, the input sequence is processed by NLP engine 110 to determine the meaning of the user’s statement and classify at least one of an on-vehicle location, event, and vehicle status associated with the input sequence. In block 430, it is determined if sufficient information has been identified to trigger targeted 5 data collection, for example have one or more of the on-vehicle location, event, and vehicle status been predicted in the default class. If it is determined that insufficient information is available, N, or one or more of the on-vehicle location, event, and vehicle status have been classified in the default class, in block 440 a prompt is generated 10 to the user requesting more details of the issue, e.g. location, what causes the issue, etc. A further input sequence is then obtained and the method returns to block 420. If it is determined that sufficient information is available, Y, in block 450 the collection of targeted diagnostic data on the vehicle 100 is triggered. In block 460, the collected diagnostic data is parsed and 15 used to attempt diagnosis of the vehicle issue. If no fault can be identified based on the collected diagnostic data, N, the method returns to block 440 and the user is prompted for further information. If the collected diagnostic data allows diagnosis of the vehicle issue, Y, for example a diagnostic code identifying a particular vehicle issue is determined to be present in the diagnostic data, then the method proceeds to block 470 when an output indicating the identified vehicle issue is provided and any 20 mitigating actions or maintenance requirement corresponding to the identified vehicle issue may be determined. As an illustrative example, a user of a vehicle 100 may provide as an input sequence 310 a symptom description of “Continuous front collision warning messages on the cluster when no impediments are in 25 front of the vehicle”. The input sequence 310 can be input to the NLP engine 110 and classified as corresponding to an on-vehicle location of a “collision warning system”, a car state of “ignition on” and an unknown or “default” event. Based on the classified on-vehicle location, event, and vehicle status the vehicle is triggered to collect diagnostic data relating to the collision warning system while the ignition is on and the collected diagnostic data provided to a diagnostic application. Any diagnostic codes present 30 in the collected diagnostic data can then be used to identify the underlying vehicle issue resulting in the symptom experienced by the user. Certain methods and systems as described herein may be implemented by a processor that processes program code that is retrieved from a non-transitory storage medium. Figure 5 illustrates an example of a computer system 500 operable to implement the described methods according to embodiments of 35 the invention. Computer system 500 includes memory 510, one or more processors 520, network interface 530, and non-transitory computer-readable medium 540. The network interface 530 forms an interface between the computer system 500 and a network 55O.The computer-readable medium 540 can be any medium that can contain, store, or maintain programs and / or data for use by or in connection with an instruction execution system or other system for giving effect to instructions. 40 Computer-readable medium can comprise any one of many physical media such as, for example, 13 03 25 electronic, magnetic, optical, electromagnetic, or semiconductor media. More specific examples of suitable machine-readable media include, but are not limited to, a hard drive, a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory, or a portable storage such as, for example, a USB drive or disk. 5 In Figure 5, the computer-readable storage medium 540 comprises program code to perform a method or implement a device as described herein. For example, the program code when executed may implement a method corresponding to the example shown in Figure 2 or Figure 4. 10 Figure 6 illustrates an example of a natural language processing, NLP, engine 600 according to an embodiment of the present invention and suitable for implementing the method 200 of Figure 2. NLP engine 600 of Figure 6 comprises a tokenizer610, a pre-trained natural language processing algorithm 620, and a classifier 630. In operation, tokenizer610 receives an input sequence comprising a sequence of words and splits the input sequence into a plurality of tokens, each token representing a word or sub-15 word piece of the input sequence. Each token is assigned a numeric value to generate a vector of values that encodes the words of the input sequence to be input to the pre-trained NLP algorithm 620. In embodiments, tokenizer610 may be adapted to recognise words and abbreviations associated with the vehicle 100. For example, tokens corresponding to names of particular components or features of 20 the vehicle, including recognized technical terminology and abbreviations, may be manually added to a list of words recognized by the tokenizer 610. Adapting the tokenizer 610 to recognize the domain specific terminology may increase the accuracy with which the NLP engine 600 is able to classify the input sequence and / or may reduce the training required for the NLP engine 600 to achieve a desired accuracy level. 25 The pre-trained NLP algorithm 620 receives the tokenized word values vector from the tokenizer 610 and generates an encoded vector that represents, or embeds, a meaning of an input sequence. For example, the NLP algorithm may receive a tokenized input sequence describing a symptom associated with a vehicle issue described in the user’s own words and generate a contextualized embedding 30 associated with the symptom being described. In operation, different descriptions of a particular vehicle issue will result in a similar encoded vector recognizing the underlying meaning of the input sequence is the same, relating to the particular vehicle issue. Pre-trained NLP algorithm 620 may comprise a neural network that has been trained on a large training 35 set of natural language texts. Examples of training corpora used to train the pre-trained NLP algorithm 620 include Wikipedia pages and BooksCorpus. One example of a pre-trained NLP algorithm 620 is a Bidirectional Encoder Representations from Transformers, BERT, model (along with related models such as RoBERTa, DistilBERT, etc.) described in “Devlin, Jacob. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." However, it will be appreciated that alternative 40 NLP algorithms could be used such as: a Generative Pre-trained Transformer, GPT, architecture model; a Text-to-TextTransferTransformer, T5, model; Megatron-LM model; or another large language model. 13 03 25 The output of the NLP algorithm 620 is then provided to classifier 630, the classifier 630 comprising one or more layers of nodes of a neural network.. The classifier 630 is trained to perform multilabel classification of the output of the NLP algorithm 620, that is the encoded vector representing the 5 symptom described by the user, as relating a particular on-vehicle location, event, and vehicle state, i.e. the input sequence is classified as relating to: one on-vehicle location of a set of possible on-vehicle locations; one event of a set of possible events; and one vehicle state of a set of possible vehicle states. In some embodiments, NLP algorithm 630 may be fine-tuned using a corpus of automotive -related 10 documents (or automotive-related text) such as automotive specific academic papers and standards documents. Furthermore, a corpus used to fine-tune the NLP algorithm 630 may include documents relating to the vehicle 100, for example owner and workshop manuals and other documentation relating to the design and operation of the vehicle 100. Such fine-tuning may significantly increase the accuracy of the NLP engine 600 in understanding language specific to the automotive domain and to the particular 15 vehicle type of which vehicle 100 is an example. Put another way, such fine tuning will configure the tokenizer 510 to recognise automotive specific words in the input sequence. Figure 7 illustrates a computer implemented method 700 according to an embodiment of the present invention to train a natural language processing engine 700 for use in the method of Figure 2. Figure 8 20 illustrates an iterative process 800 corresponding to the method 700 of training the NLP engine 600 according to an embodiment of the invention. In the compute implemented method 700 of Figure 7, a store of training data 810 is provided comprising training example input / output pairs of natural language descriptions of vehicle issue symptoms and 25 associated on-vehicle location, event, and vehicle status. In block 710 of the method 700, the set of training data 810 to be used to train the NLP engine 600 is received, the training data including training example input / output pairs of natural language descriptions of vehicle issue symptoms and associated on-vehicle location, event, and vehicle status. 30 In block 720, for one or more of the input / output pairs, the natural language description of the symptom is input to the NLP engine 600. For each of the input values used to train the NLP engine 600, an example natural language description input can be provided to NLP engine 600 and an output is generated by the NLP engine 600 in response, the output comprising predicted on-vehicle location, event, and vehicle status. 35 In block 730, the predicted classification result is compared with the associated on-vehicle location, event, and vehicle status output 840 corresponding to the natural language symptom description of the input / output pair and an error between the predicted classification result and the expected classification results of the training data is characterized. For example, the predicted vehicle issue may be compared 40 in comparator 850 with the actual associated vehicle issue 840 provided in the training data 810 to characterize an error between the predicted classification result and the corresponding on-vehicle location, event, and vehicle status identified in the training set. In block 740, an optimization algorithm, for example is used to update weights of the neural network 5 based on the characterized error. Updated weights may be calculated by a tuning algorithm 830 that receives the output of comparator 850 and in response updates weights of the neural network of the NLP engine 600 using an appropriate optimization algorithm. In embodiments, during training of the NLP engine 600, weights of the pre-trained NLP algorithm 620 10 may be frozen and weights of the classifier 630 updated to train the classifier 630 to predict the appropriate vehicle issue based on the contextualized embedding of the user’s description of the symptom. Table 1 provides a number of examples of input / output pairs suitable for use as training fortraining the 15 NLP engine 600 to classify the natural language description of a vehicle issue as one of a set of possible on-vehicle locations, events, and vehicle statuses. 13 03 25 INPUT: CUSTOMER_VERBATIM OUTPUT Location Event Status Heated / cooled seats seem to intermittently function during ignition cycles ie ... Heated seats selected upon vehicle collection, vehicle shut down at the gatehouse and return to vehicle ... no heated seat function ... then drive 30 miles ... exit vehicle and upon return the heated seat began to function at the setting selection made at the start of journey Seats default Ignition On Wipers squeak, blade knock on return with slow or light rain. Windscreen Wipers Wipers active default Absolutely incredible to drive, great manoeuvrability, comfortable and a fun drive default default default Coolant level low flagging Engine default default While driving at around 1600 RPM what can only be described as a hum from the rear of the vehicle, It was easily repeatably but difficult to recognise whether it was being generated from the chassis or exhaust system, most noticeable when the audio was muted. Engine 1600rpm In motion While driving at around 1600 RPM what can only be described as a hum from the rear of the vehicle, It was easily repeatably but difficult to recognise whether it was being generated from the chassis or exhaust system, most noticeable when the audio was muted. Engine 1600rpm In motion 13 03 25 "Park hold activates too often and holds for too long. Regularly results in abrupt release and unrefined pullaway. Need to press accelerator to release which results in a jolt as the HOLD releases. Issue repeatable and occurs every time vehicle activates and releases hold. Varies from annoying to serious, when parked on a slope and needing to press accelerator to release hold to reverse downhill in a carpark. This results in a jolt and more speed than desired to slowly manoeuvre out of a parking bay. Handbrake On handbrake release Handbrake on When the vehicle is driven at slow speeds there is very obvious front brake squeal, very annoying. Brakes default In motion rear headrest button is stuck, so headrest wont flip down. default default default The moulding trim on the outer edge of the door is loose and rattles when driving. When I press on it the rattling will stop for a day or so but then starts again. Door default In motion I was expecting a more noise cancelation cabin. It is comfortable but I was expecting something better. default default default Not a smooth pulling start. It pulls excessively. Need to apply breaks. Engine Moving from stationary Ignition on there are times when it hesitates for no apparent reason Engine Acceleration Ignition On The transmission is slightly less smooth than the transmission in my 2018 Discovery (which had 6 cylinders). Transmission default Ignition On diagnose speed limiter customer travelling on the motorway and car would not go above 40mph then after 15 mins of driving the car is able to travel at 70mph Driving aids Speed limiter active In motion customer states reverse traffic detection not available and blindspot monitoring not available Driving aids Reverse gear selected In motion TABLE 1 Figure 9 illustrates a further example of a natural language processing, NLP, engine 900 according to 5 an embodiment of the present invention and suitable for implementing the method 200 of Figure 2. NLP engine 900 of Figure 9 is similar to the NLP engine 600 illustrated in Figure 6, comprising a tokenizer 910 and pre-trained NLP algorithm 620, but further includes a prompt-table 920. Optionally, classifier 630 may be replaced by post-processing module 930 to interpret, or confine, the 10 output of NLP algorithm 620 as belonging to a particular class of the plurality of classes defined for the task being performed. Post-processing module may be particularly applicable when interpreting the output of generative natural language processing algorithms. 13 03 25 The prompt-table 920 may store one or more predefined sets of virtual tokens to be included in the token sequence provided by the tokenizer 910 as input to the pre-trained NLP algorithm, 620. Each of the predefined sets of virtual tokens may influence the NLP algorithm 620 to process the input sequence in 5 a particular way according to a particular task, i.e. to classify the input sequence as a vehicle issue of the set of vehicle issues. In particular, the prompt-table 920 may store sets of virtual tokens relating to a plurality of different tasks to be performed using the NLP engine 900. In order to perform a particular task, a set of virtual tokens associated with that task may be retrieved from the prompt-table 920 and included with the token sequence generated by the tokenizer 910. 10 A task specific set of virtual tokens may be generated for particular task by training a neural network model to predict virtual token embeddings, for example during training of the NLP engine 600 using the method illustrated in Figures 7 and 8. During training, the neural network learns a set of virtual tokens to include in the tokenized input sequence to influence the NLP algorithm 620 to process the input 15 sequence in to perform the particular task. Training of the neural network used to generate the task specific set of virtual tokens may be achieved using a relatively small set of training data, for example using a gradient decent algorithm, with the weights of the pre-trained NLP algorithm 620 frozen, reducing the computational effort required to train the NLP engine 600 as compared to further fine-tuning training of the NLP pre-trained NLP algorithm 620. Once the task specific set of virtual tokens has been 20 generated, the set of virtual tokens may be stored in prompt-table 920 to be used when the particular task is to be performed. Thus, the use of prompt-table 920 allows a static pre-trained NLP algorithm 620 to be adapted to different tasks by providing an appropriate set of virtual tokens as part of the input sequence. 25 The natural language processing engine 900 illustrated in Figure 9, including prompt-table 920, may be particularly, but not exclusively, appropriate when a generative NLP algorithm, such as a Generative Pre-Trained Transformer model, is used to implement embodiments of the invention. Such generative NLP algorithms are capable of generating responses of unlimited length in natural language in response 30 to an input prompt, but may be less suited to classification tasks in the absence of a prompt encoder. Prompt encoding is discussed in more detail in Liu, Xiao, et al. "GPT understands, too." arXiv preprint arXiv:2103.10385 (2021). In some embodiments, information relating to the classes, e.g. information describing each vehicle issue 35 of the set of vehicle issues, may be processed using the pre-trained NLP model 630 to generate an embedding of that information in the form of a class embedding vector representing a semantic meaning of the information relating to each class. Processing the input sequence 310 describing a symptom experienced by the user using the NLP 40 engine may then comprise generating an input embedding vector for the input sequence 310 and comparing the input embedding vector with each class embedding vector corresponding to the available classification categories to calculate a similarity score between the input embedding vector and each class embedding vector. A vehicle issue corresponding to the class embedding vector having the highest similarity score, that is most similar, to the input embedding vector may then be identified as the vehicle issue. It will be appreciated that various changes and modifications can be made to the present invention without departing from the scope of the present application. 13 03 25

Claims

13 03 251. A computer implemented method comprising:obtaining an input sequence, the input sequence comprising a sequence of words describing 5 a symptom associated with a vehicle issue;processing the input sequence using a natural language processing, NLP, engine to classify the input sequence according to at least one of an on-vehicle location associated with the vehicle issue and an event associated with the vehicle issue;triggering collection of targeted diagnostic data based on the at least one of the on-vehicle10 location and event;receiving the targeted diagnostic data;parsing the received targeted diagnostic data to identify a diagnostic code;generating an indication of a detected vehicle issue based on the diagnostic code;outputting the indication of the detected vehicle issue;15 determining an action to be performed to mitigate the detected vehicle issue; andoutputting an indication to a user to perform the determined action.

2. The computer implemented method of claim 1, the method further comprising:parsing the received targeted diagnostic data to identify a diagnostic code; andin response to determining that no diagnostic code is present in the targeted diagnostic data:20 generating a prompt to the user to request a second input sequence;obtaining the second input sequence comprising a second sequence of words describing the symptom; andwherein processing the input sequence using the NLP engine further comprises processing the second input sequence using the NLP engine to classify the second input sequence according to 25 at least one of a second on-vehicle location associated with the vehicle issue and a second event associated with the vehicle issue; andtriggering collection of second targeted diagnostic data based on the at least one of the second on-vehicle location and the second event.13 03 253. The computer implementing method of any preceding claim, wherein processing the input sequence using the NLP engine comprises performing multiclass classification of the input sequence, wherein the multiclass classification includes a default class.

4. The computer implemented method of claim 3, the method further comprising:5 in response to the multiclass classification of the input sequence classifying the inputsequence as having one of the on-vehicle location and the event, generating a prompt to the user to request a third input sequence;obtaining the third input sequence comprising a third sequence of words describing the symptom; and10 processing the third input sequence using the NLP engine to generate an output comprising afurther classification for the at least one of on-vehicle location associated with the vehicle issue and the event associated with the vehicle issue was classified in the default class.

5. The computer implemented method of any of claims 1 to 4, wherein the natural language processing engine comprises a neural network including:15 a tokenizerto receive an input sequence comprising a sequence of words describing asymptom associated with the vehicle issue and generate a token sequence;a pre-trained NLP algorithm to receive the token sequence; and a classification layer.

6. The computer implemented method of any of claims 1 to 5, wherein obtaining the input20 sequence comprises:obtaining an audio signal; andperforming speech-to-text conversion on the audio signal to generate the input sequence.

7. The computer implemented method of any of claims 1 to 6, wherein the natural language processing engine has been trained to predict a classification of an on-vehicle location, event and25 vehicle status associated with a natural language description of a symptom associated with a vehicle issue using a set of training data comprising training example input / output pairs of natural language descriptions of symptoms and associated on-vehicle location, event and vehicle status associated with the vehicle issue.13 03 258. An apparatus comprising:a processor; anda memory comprising computer program instructions that when executed by the processor cause the apparatus to:5 obtain an input sequence, the input sequence comprising a sequence of words describing asymptom associated with a vehicle issue;process the input sequence using a natural language processing, NLP, engine to generate an output comprising at least one of an on-vehicle location associated with the vehicle issue and an event associated with the vehicle issue;10 trigger collection of targeted diagnostic data based on the at least one of the on-vehiclelocation and the event;receive the targeted diagnostic data;parse the received targeted diagnostic data to identify a diagnostic code;generate an indication of a detected vehicle issue based on the diagnostic code;15 output the indication of the detected vehicle issue;determine an action to be performed to mitigate the detected vehicle issue; andoutput an indication to a user to perform the determined action.

9. A vehicle comprising the apparatus of claim 8.

10. Computer readable instructions which, when executed by a computer, are arranged to perform 20 a method according to claims 1 to 7.

Images