Cockpit end multi-vehicle mission adaptation method, device and vehicle

Abstract

The application provides a cockpit end multi-vehicle task adaptation method, equipment and vehicle. The cockpit end multi-vehicle task adaptation method is applied to a large model provided with a public processing layer, at least two task input adaptation layers and at least two task output adaptation layers. The method comprises task format adaptation of a corresponding first vehicle task through each task input adaptation layer; task data calculation of each task format data through the public processing layer; format conversion of intermediate feature data through the task output adaptation layer corresponding to the task input adaptation layer, and adaptation of the converted intermediate feature data and vehicle functions. In the application, the public processing layer is reused to calculate the task data of each task format data, which can effectively reduce the number of large models required in the multi-vehicle task execution process, reduce the computing power required by the vehicle chip, and avoid simultaneous execution of multiple vehicle task analysis and vehicle function adaptation.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a cockpit-side multi-vehicle mission adaptation method, device, and vehicle. Background Technology

[0002] With the rapid development of vehicle intelligence, the number of vehicle functions that can be realized within the cockpit is increasing. Consequently, the tasks corresponding to these functions are also increasing, such as user-input control commands and tasks that the vehicle needs to automatically execute based on environmental and operational parameters. During task execution, the vehicle's infotainment chip needs to parse the task and adapt it to the corresponding vehicle function to control its execution. For task parsing and function adaptation, a large-scale model is typically set up within the infotainment chip to accurately parse the task and adapt the vehicle function.

[0003] However, as vehicle functions continue to increase, more and more vehicle tasks are being executed simultaneously in the vehicle cabin. Each vehicle task requires a corresponding large model. At this time, the computing power on the vehicle chip will continue to increase with the number of large models. Consequently, with the limited computing power of the vehicle chip, some vehicle tasks cannot be parsed and adapted to vehicle functions at the same time, and the vehicle chip may even freeze due to excessive load.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a cockpit-side multi-vehicle task adaptation method, device, and vehicle, aiming to solve the technical problem in the prior art that it is difficult to simultaneously perform the parsing of multiple vehicle tasks and the adaptation of vehicle functions.

[0006] To achieve the above objectives, this invention proposes a cockpit-side multi-vehicle task adaptation method, which is applied to a large model with a common processing layer, at least two task input adaptation layers, and at least two task output adaptation layers. The cockpit-side large model scheduling methods include: The task input adaptation layers are used to adapt the corresponding first vehicle task to the task format to obtain task format data. The common processing layer performs task data calculations on each of the task format data to obtain intermediate feature data. The intermediate feature data is formatted by the task output adaptation layer corresponding to the task input adaptation layer, and the converted intermediate feature data is adapted to the vehicle functions. Perform the adapted vehicle functions.

[0007] Optionally, the cockpit-side multi-vehicle mission adaptation method further includes: The computing power of the vehicle's infotainment chip equipped with the large model and the computing power required for each vehicle's tasks are detected. If the sum of the computing power required by each vehicle task is greater than the chip computing power, the current vehicle task is determined based on the chip computing power and the computing power required by each vehicle task. The current vehicle task is adapted to the task format through the corresponding input adaptation layer to obtain task format data.

[0008] Optionally, determining the current vehicle task based on the chip's computing power and the task computing power required for each vehicle task includes: Obtain the task type for each of the first vehicle tasks; The vehicle tasks are prioritized based on the chip computing power, the task type, and the computing power required for each of the first vehicle tasks. The current vehicle task is determined based on the sorting results.

[0009] Optionally, the cockpit-side multi-vehicle mission adaptation method further includes: During the process of adapting the corresponding vehicle task to the task format through each of the task input adaptation layers, it is detected whether there is a second vehicle task being input. If so, then check the priority of the second vehicle's task; If the priority of the second vehicle task is higher than that of the first vehicle task, the format adaptation of the first vehicle task is suspended. The task input adaptation layer corresponding to the second vehicle task adapts the task format.

[0010] Optionally, pausing the first vehicle task format adaptation includes: Obtain temporary task data during the task format adaptation process of the first vehicle task; The temporary task data is formatted and stored; Accordingly, after the intermediate feature data of the second vehicle task is converted and adapted to the vehicle function, the temporary task data is extracted and the format is restored. Based on the temporary task data after format restoration, continue to execute the step of adapting the corresponding first vehicle task to the task format through each of the task input adaptation layers to obtain task format data.

[0011] Optionally, the cockpit-side multi-vehicle mission adaptation method further includes: Detect input user commands and / or sensor data, and determine a first vehicle task based on the user commands and / or sensor data; Determine the task sensitivity of the first vehicle task; When the task sensitivity is lower than the preset sensitivity, the similarity threshold between the feature fingerprint of the first vehicle task and each preset feature fingerprint in the preset feature fingerprint database is obtained. If the similarity threshold is greater than the preset similarity threshold, the vehicle function will be adjusted according to the adjustment method corresponding to the preset feature fingerprint whose similarity threshold is greater than the preset similarity threshold. If the task sensitivity is not lower than the preset sensitivity or there is no similarity threshold greater than the preset similarity threshold, the step of adapting the corresponding first vehicle task to the task format through each of the task input adaptation layers to obtain task format data is executed.

[0012] Optionally, after converting the format of the intermediate feature data through the task output adaptation layer corresponding to the task input adaptation layer, and adapting the converted intermediate feature data to the vehicle functions, the method further includes: Execute the adapted vehicle functions and detect the bandwidth utilization of the vehicle chip equipped with the large model; When the bandwidth utilization rate is lower than the preset utilization rate, the degree of execution of the vehicle function is detected; When the execution level of the vehicle function reaches the pre-termination level, the current scene state of the vehicle is obtained; Based on the current scene state, the task of the third vehicle is predicted using a preset rule base or inference prediction. The third vehicle task is input into the cache area of ​​the task input adaptation layer.

[0013] Optionally, the cockpit-side multi-vehicle mission adaptation method further includes: During the operation of the common processing layer, the task input adaptation layer, and the task output adaptation layer, the load rate of the neural network processing unit of the large model is detected. When the load rate reaches the preset load rate, the amount of vehicle task data input to the task input adaptation layer is reduced.

[0014] In addition, to achieve the above objectives, the present invention also provides a cockpit-side multi-vehicle mission adaptation device, which is used to perform the cockpit-side multi-vehicle mission adaptation method described in any of the above claims.

[0015] In addition, to achieve the above objectives, the present invention also provides a vehicle, wherein the vehicle is equipped with a data acquisition sensor, a human-machine interface terminal, a vehicle-mounted chip, and a cockpit-side multi-vehicle task adaptation device, wherein a large model is mounted on the vehicle-mounted chip, and the data acquisition sensor and the human-machine interface terminal are used to acquire data corresponding to vehicle tasks and input them into the large model in the vehicle-mounted chip.

[0016] This invention provides a cockpit-side multi-vehicle task adaptation method, device, and vehicle. The cockpit-side multi-vehicle task adaptation method is applied to a large model equipped with a common processing layer, at least two task input adaptation layers, and at least two task output adaptation layers. The cockpit-side large model scheduling method includes: adapting the corresponding first vehicle task to a task format through each of the task input adaptation layers to obtain task format data; performing task data calculation on each of the task format data through the common processing layer to obtain intermediate feature data; converting the intermediate feature data to a format through the task output adaptation layer corresponding to the task input adaptation layer, and adapting the converted intermediate feature data to vehicle functions. In this invention, by setting a specific structure for the large model, utilizing multiple task input and task output adaptation layers for vehicle function adaptation, and reusing the common processing layer to perform task data calculation on each of the task format data, the number of large models required during multi-vehicle task execution can be effectively reduced, the computing power required by the vehicle's infotainment chip can be reduced, and the difficulty of simultaneously executing multiple vehicle task parsing and vehicle function adaptation can be avoided. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of the large model proposed in this invention; Figure 2 This is a flowchart illustrating the first embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention. Figure 3 This is a flowchart illustrating the second embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention. Figure 4 This is a schematic diagram of the first process of the third embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention; Figure 5 This is a second flowchart illustrating the third embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention. Figure 6 This is a flowchart illustrating the fourth embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention. Figure 7 This is a flowchart illustrating the fifth embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention. Figure 8 This is a flowchart illustrating the sixth embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention.

[0019] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0023] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0024] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of the large model proposed in this invention.

[0025] The large model is configured with a common processing layer 20, at least two task input adaptation layers 10, and at least two task output adaptation layers 30.

[0026] It should be noted that large models can refer to large-scale artificial intelligence models deployed on the vehicle's on-device side, with a parameter count typically in the hundreds of millions (B) range, used to handle complex perception, understanding, and decision-making tasks in the smart cockpit.

[0027] It should be noted that the task input adaptation layer 10 is the vehicle task input layer of the large model. It is used to perform format adaptation, type adaptation, and other adaptation processes for the vehicle task, adapting the input vehicle task into a data format or data type that the common processing layer 20 can directly process. The task input adaptation layer 10 can also be an input feature adaptation layer, responsible for receiving different types of raw input data (such as text, speech, and images) and performing preliminary feature extraction, converting multimodal inputs into a unified feature vector format, i.e., task format data.

[0028] It should be noted that the common processing layer 20 is the feature backbone layer of the large model. As the intermediate computational base of the large model, it can be used for data computation, performing data operations on the unified data format adapted to various vehicle tasks, and outputting the calculated results, i.e., intermediate feature data. The common processing layer 20 is essentially a carrier of an intermediate data computation process. As a core processing module shared by all tasks, the common processing layer 20 uses a multilayer perceptron or attention mechanism to perform deep processing on the unified feature vector, and can maintain a resident loading state to achieve fast response.

[0029] It should be noted that the task output adaptation layer 30, as the output layer of the large model, is used to convert the intermediate feature data output by the common processing layer 20 into a format that the vehicle controller can recognize. According to different vehicle task types, corresponding adapter modules are configured to convert the intermediate feature data output by the common processing layer 20 into a task-specific output format, such as text, control commands, and confidence scores. The calculated intermediate feature data is then adapted to the corresponding vehicle functions. When the vehicle controller receives the data after the format conversion and adaptation process by the task output adaptation layer 30, it can directly recognize the data and control the execution of the corresponding vehicle functions.

[0030] In the actual operation, each task input adaptation layer 10 receives the raw data of its corresponding task, performs format standardization and feature extraction, and outputs task format data with a unified dimension. The task format data of all vehicle tasks are aggregated into a common processing layer 20 for unified deep computation to extract and enhance semantic features, generating intermediate feature data. This intermediate feature data is distributed to the task output adaptation layer 30 corresponding to the original task. Each task output adaptation layer decodes and converts the received intermediate feature data to adapt it to the input requirements of specific vehicle functions. The vehicle controller can then execute instructions or data generated through adapter conversion that can directly drive vehicle functions. For example, when performing a single vehicle task of turning on the headlights, the task input adaptation layer 10 can convert the command corresponding to the headlight turning-on task or the external light intensity parameters identified by the sensor into a data format that the common processing layer 20 can recognize, completing the format adaptation process. Then, the task format data corresponding to the headlight turning-on task is output to the common processing layer 20. Upon receiving the task format data corresponding to the headlight turning-on task, the common processing layer 20 can determine that the vehicle headlights need to be turned on through task data calculation, obtaining the corresponding intermediate feature data. Upon receiving the intermediate feature data corresponding to the headlight turning-on task, the task output adaptation layer 30 can convert the intermediate feature data into a data format that the vehicle controller can recognize, and then adapt the intermediate feature data with the vehicle function to obtain the vehicle function of turning on the headlights. Thus, after the vehicle controller receives the adapted data, it can recognize that the vehicle headlights need to be turned on. For example, when the system needs to handle two vehicle tasks simultaneously, namely voice command understanding and driver distraction detection, the two input adaptation layers 10 process the audio and image data respectively to generate feature vectors in a unified format. Then, the common processing layer 20 performs deep semantic calculations on these two feature vectors. The intermediate results generated by the calculations are handed over to the two output adaptation layers 30 to be converted into a text command to turn on the air conditioner and a distraction level of high.

[0031] In this model, the number of task input adaptation layers 10 and task output adaptation layers 30 is set to at least two, with the same number of task input adaptation layers 10 and task output adaptation layers 30. The number of common processing layers 20 is one. The large model can simultaneously execute the adaptation process for at least two vehicle tasks. Figure 1 Taking the setting of four task input adaptation layers 10 and task output adaptation layers 30 as an example, each task input adaptation layer 10 corresponds to one task output adaptation layer 30.

[0032] For example, a large model requires 7 bytes of parameters, using... Figure 1The large model described above requires 6 bytes of parameters in the common processing layer 20. The number of parameters required for the task input adaptation layer 10 and the task output adaptation layer 30 is usually related to the specific vehicle task, but their required number of parameters is typically less than 1 byte. When setting up multiple vehicles, the number of parameters corresponding to multiple large models is typically 7 bytes * N. Figure 1 The large model uses 6B + 0.1B*N parameters, which can significantly reduce the number of parameters required for the four tasks.

[0033] Figure 2 This is a flowchart illustrating the first embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention. Based on Figure 2 The present invention presents a first embodiment of a cockpit-side multi-vehicle task adaptation method.

[0034] In this embodiment, the cockpit-side multi-vehicle task adaptation method includes: Step S10: Adapt the corresponding first vehicle task to the task format through each of the task input adaptation layers to obtain task format data.

[0035] It should be understood that, in the embodiments, the executing entity can be a vehicle or a controller within the vehicle, such as a vehicle controller. A camera, a voice acquisition microphone, and the vehicle controller itself can be installed on the vehicle, and the aforementioned large model is housed within the vehicle controller. In this embodiment and the following embodiments, the vehicle controller can be used as the executing entity to discuss the cockpit-side multi-vehicle task adaptation method in detail.

[0036] It should be noted that the first vehicle task refers to the task that the current vehicle needs to perform to activate vehicle functions, such as turning on the headlights, detecting voice, and detecting images. Multiple first vehicle tasks exist simultaneously. Task format data can refer to the time feature vector with a unified dimension and data structure output by the task input adaptation layer after format adaptation. Task format adaptation refers to the process of converting raw task data from different sources and in different formats into a unified data format that can be recognized and processed by the common processing layer within the larger model, through a specific processing flow. This process can include data preprocessing, feature extraction, and vectorization. For example, for a speech task, the adaptation process might include converting the audio waveform into Mel-frequency cepstral coefficient features, and then converting it into a feature vector through an encoding network; for an image task, the adaptation process might include image scaling, normalization, and then extracting visual feature vectors through a convolutional neural network. For example, regardless of whether the original input is speech, text, or image, after processing by their respective adaptation layers, it is converted into a floating-point vector of the same dimension so that subsequent common processing layers can perform unified calculations.

[0037] In the specific implementation, when there are multiple first vehicle tasks, multiple first vehicle tasks that need to be processed are received in parallel or sequentially. Each first vehicle task is associated with a specific task input adaptation layer. Each task input adaptation layer obtains the raw input data of its corresponding first vehicle task. Each task input adaptation layer decodes, cleans, and extracts features from the raw input data according to its preset data processing logic. Then, each adaptation layer maps the extracted features to a predefined vector space of uniform dimension, generating task format data with a consistent structure. For example, if two first vehicle tasks, voice command recognition and occupant detection, occur simultaneously in the cockpit, the input adaptation layer corresponding to the voice task receives a segment of PCM-encoded audio data, performs frame segmentation and windowing on the audio, extracts MFCC features, and finally encodes the feature sequence into a feature vector of a common processing layer recognizable dimension through a lightweight fully connected network. This vector is the task format data of the voice task. Meanwhile, the input adaptation layer corresponding to the personnel detection task receives an RGB image from the in-vehicle camera. It first scales the image, then normalizes it, and extracts image features through a miniature convolutional neural network. It also outputs a feature vector with identifiable dimensions for the common processing layer, which serves as the task format data for the image task.

[0038] Step S20: The common processing layer performs task data calculations on each of the task format data to obtain intermediate feature data.

[0039] It's important to clarify that intermediate feature data refers to semantically rich feature data obtained after task-formatted data has been processed by a common processing layer. This intermediate feature data provides a general semantic interpretation for subsequent output adaptation layers corresponding to different vehicle tasks. For example, a voice command to open a car window and an image of a driver yawning might, after processing by the common processing layer, output intermediate feature data representing the intention to open the window and the driver's fatigue state, respectively. The task data computation process can refer to the deep semantic understanding and feature enhancement processing performed by the common processing layer on the input uniform-formatted feature vector. This computation process utilizes computational units within the common processing layer (such as multilayer perceptrons or attention mechanisms) to extract and fuse semantic information from the input features.

[0040] In the specific implementation, a common processing layer receives task format data in a unified vector format from the outputs of each task input adaptation layer. Utilizing its internally pre-trained neural network model, the common processing layer performs deep computation on the aggregated sets of task format data. This allows for the extraction and enhancement of semantic information from the data, such as understanding the intent of user commands, identifying the attributes of objects in images, or determining the category of a scene. After completing the computation, the common processing layer generates a set of corresponding intermediate feature data for each set of input task format data. Finally, each set of intermediate feature data is distributed to the corresponding task output adaptation layers. For example, the task format data output by the voice input adaptation layer is a 768-dimensional vector A representing audio features, and the output by the image input adaptation layer is a 768-dimensional vector B representing image features. The common processing layer receives both vectors A and B. Through its internal multi-layer attention mechanism and feedforward network computation, vector A is converted into 1024-dimensional intermediate feature data A' containing the semantics of the user wanting to listen to music; vector B is converted into 1024-dimensional intermediate feature data B' containing the semantics of a child in the car. A' and B' are the intermediate feature data obtained from the calculation, which will be sent to the music playback control output adaptation layer and the child legacy detection output adaptation layer, respectively.

[0041] Step S30: Convert the format of the intermediate feature data through the task output adaptation layer corresponding to the task input adaptation layer, and adapt the converted intermediate feature data to the vehicle function.

[0042] It's important to clarify that format conversion can refer to the process by which the task output adaptation layer decodes and maps the received intermediate feature data into an instruction or data format that can be recognized and executed by a specific vehicle function. For example, converting feature vectors into natural language text, structured control commands, probability distributions, or confidence scores. Vehicle functions can refer to specific operational or service functions performed by software or hardware in a smart cockpit to adjust the user's or vehicle's state, such as adjusting the air conditioning temperature, playing music, navigating to a destination, issuing safety warnings, or changing the ambient lighting color.

[0043] In its implementation, each task output adaptation layer contains decoding logic or a lightweight network designed for a specific task. The task output adaptation layer converts the received intermediate feature data to generate a target format. It then adapts the target format output data to the specific vehicle function, encapsulating the data into instructions or messages that the corresponding execution unit can parse, such as those recognizable by the in-vehicle infotainment system, body controller, instrument panel, or speech synthesis module. For example, the common processing layer outputs intermediate feature data representing the semantics of a user command. The corresponding task output adaptation layer for this task chain is the air conditioning control output adapter. This adapter calculates the intermediate feature data, converts it into a structured control command, adapts the command to the first vehicle function (automatic air conditioning adjustment), and then sends the control command to the vehicle controller. After receiving and parsing the control command, the vehicle controller controls the air conditioning to adjust the interior temperature.

[0044] In this embodiment, by setting a specific structure for the large model, using multiple task input adaptation layers and task output adaptation layers to adapt vehicle functions, and reusing the common processing layer to perform task data calculations on the various task format data, the number of large models required during the execution of multiple vehicle tasks can be effectively reduced, the computing power required by the vehicle chip can be reduced, and the simultaneous execution of multiple vehicle task parsing and vehicle function adaptation can be avoided.

[0045] Based on the first embodiment of the cockpit-side multi-vehicle task adaptation method described above, a second embodiment of the cockpit-side multi-vehicle task adaptation method of the present invention is proposed. (Refer to...) Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention.

[0046] In this embodiment, step S10 specifically includes: Step S101: Detect the chip computing power of the vehicle chip equipped with the large model and the task computing power required for each vehicle task.

[0047] It should be understood that during the calculation and adaptation of multiple vehicle tasks using a large model, there may be situations where the computing power of the vehicle chip is insufficient. For example, if there are too many vehicle tasks, or if some vehicle tasks require too much computing power, the large model in the vehicle chip may not be able to execute multiple vehicle tasks simultaneously.

[0048] It should be noted that the in-vehicle infotainment chip is a chip installed within the vehicle controller. This chip can house a large model for adapting and calculating input data or instructions. Chip computing power refers to the theoretical peak computing capability of the in-vehicle infotainment chip deployed on the smart cockpit side, typically measured in operations per second. The computing power of the in-vehicle infotainment chip is determined based on the specific type of chip installed within the vehicle controller. Task computing power refers to the amount of computing resources required for the large model to complete a vehicle task. It can be represented as the number of computational operations consumed or the equivalent computing power required for each execution of the task. Task computing power can be obtained by analyzing the components of the large model corresponding to the task, including its input adaptation layer, the computation graph portion occupied in the common processing layer, and the output adaptation layer.

[0049] In practical implementation, the available chip computing power of the vehicle's chip integrated into the large model can be detected or read through the vehicle chip resource monitoring module. Simultaneously, based on a pre-set task computing power model library or real-time performance monitoring data, the task computing power required for each vehicle function task awaiting execution can be obtained.

[0050] Step S102: If the sum of the computing power required by each vehicle task is greater than the chip computing power, determine the current vehicle task based on the chip computing power and the computing power required by each vehicle task.

[0051] It should be noted that the sum of the computing power required by each vehicle task refers to the arithmetic sum of the computing power required by each of the multiple tasks to be processed. The current vehicle task refers to one or more vehicle tasks that are selected to immediately enter the computing process after scheduling decisions, when the total computing demand of the system is detected to exceed the available computing power of the chip.

[0052] In practice, the computing power required by all waiting vehicle tasks can be summed to obtain the total computing power required by each vehicle task. This sum is then compared with the detected chip computing power. If the sum of the computing power required by all vehicle tasks exceeds the chip computing power, it indicates that the chip's computing resources are insufficient to support simultaneous parallel computation of all tasks. In this case, based on the chip's computing power limit and the computing power required by each vehicle task, combined with scheduling strategies such as task priority and urgency, one or more vehicle tasks that are allowed to be executed within the current period can be selected from all waiting tasks and designated as the current vehicle task.

[0053] Step S103: Adapt the current vehicle task to the task format through the corresponding input adaptation layer to obtain task format data.

[0054] In practical implementation, once the current vehicle task is determined, the subsequent task format adaptation process can be executed through the input adaptation layer corresponding to that task to generate task format data. Unselected vehicle tasks enter a waiting queue. After at least one current vehicle task has been calculated and adapted, all or some of these vehicle tasks are designated as the current task. For example, the vehicle infotainment chip has a computing power of 200 TOPS. The cockpit system simultaneously receives three task requests: a full-scene voice dialogue vehicle task requiring 120 TOPS, a driver fatigue detection vehicle task requiring 50 TOPS, and a navigation rendering vehicle task requiring 80 TOPS. The total computing power required for these three tasks is 250 TOPS, which is greater than the chip's 200 TOPS computing power. In this case, based on preset scheduling rules (e.g., prioritizing safety tasks), a decision is made to determine driver fatigue detection and AR navigation rendering as the current vehicle tasks, because their combined computing power requirement (130 TOPS) does not exceed the chip's computing power. Subsequently, the data corresponding to these two vehicle tasks are input into the corresponding input adaptation layer to adapt the current vehicle task to the task format and obtain task format data.

[0055] Furthermore, given the limited computing power of the vehicle's infotainment chip, which can only run one vehicle task at a time, to minimize the processing time of multiple vehicle tasks, the input adaptation layer can output the task format data of the first vehicle task to the common processing layer, while the input adaptation layer for the second vehicle task can perform the format adaptation process for that second vehicle task. Alternatively, when the common processing layer outputs the intermediate feature data of the first vehicle task to the output adaptation layer, the first vehicle task is in the output adaptation layer, and the common processing layer can immediately process the format conversion data of the second vehicle task. The input feature layer for the third vehicle task then performs the format conversion for that third vehicle task. The priority of the first vehicle task is higher than that of the second vehicle task, and vice versa.

[0056] Step S102 includes: Step S1021: Obtain the task type of each first vehicle task; It should be noted that task type refers to the type corresponding to the functional attributes of various vehicle functions and tasks in the smart cockpit. For example, for vehicle tasks involving adaptation and computation, task types can include voice recognition, image understanding, natural language processing, scene perception, etc. If further distinguished, they can be divided into safety tasks, such as driver status monitoring; navigation tasks, such as AR navigation; entertainment and information tasks, such as music playback and voice chat; and vehicle control tasks, such as air conditioning adjustment.

[0057] Step S1022: Prioritize each vehicle task based on the chip computing power, the task type, and the task computing power required for each first vehicle task.

[0058] It's important to note that priority refers to the order in which vehicle tasks are executed within the larger model. The priority of vehicle tasks varies depending on their task type. For example, safety-related vehicle tasks need to be executed immediately, therefore their priority is higher than other task types. On the other hand, entertainment-related vehicle tasks, primarily for leisure and not affecting driving safety, have a relatively lower priority. For instance, if four vehicle tasks are received simultaneously: driver fatigue detection, air conditioning activation, navigation rendering, and music playback, the priority order should be: driver fatigue detection > navigation rendering > air conditioning activation > music playback.

[0059] It should be understood that different chips have different computing power, and therefore can handle different number of vehicle tasks simultaneously, resulting in different priority rankings for these tasks. For example, if tasks are first sorted by type, and then if multiple vehicle tasks exist within a single task type, the task requiring less computing power will have higher priority than the task requiring more computing power. For instance, in an entertainment-related vehicle task, there might be two tasks: identifying whether music is on and identifying whether video is playing. Since the task identifying whether music is on has lower computing power, it will have higher priority than the task identifying whether video is playing.

[0060] Of course, depending on the different computing power required for the first vehicle tasks, the priority ranking of each vehicle task can also be different. Currently, the vehicle's chip has high computing power. While ensuring the execution of safe task types, the number of first vehicle tasks that can be executed can be determined by combining the computing power required for the remaining first vehicle tasks and the remaining computing power of the vehicle's chip. Prioritizing the ranking scheme that can execute more first vehicle tasks will be considered to execute as many vehicle tasks as possible as quickly as possible. Using the above example, with a chip computing power of 200 TOPS, four vehicle tasks are received simultaneously. The driver fatigue detection task requires 100 TOPS, the air conditioning activation task requires 50 TOPS, the navigation rendering task requires 80 TOPS, and the music playback task requires 50 TOPS. Although from the perspective of task type, the navigation rendering task should have a higher priority than the air conditioning and music playback tasks, from the perspective of chip computing power, three vehicle tasks can be executed simultaneously in a short time (the driver fatigue detection task is a safety-related task that must be executed). Therefore, the driver fatigue detection task, the air conditioning activation task, and the music playback task can be designated as the first priority task, and the navigation rendering task as the second priority task.

[0061] Step S102: Determine the current vehicle task based on the sorting results.

[0062] In practice, the task type of each waiting vehicle task can be identified first by using the metadata tags carried during task creation or by analyzing the configuration of the corresponding model components. Then, based on a preset scheduling strategy, a task priority evaluation model or task priority rule model is used to prioritize all waiting vehicle tasks by considering three dimensions: the upper limit of chip computing power, the task type of each task, and the specific computing power required by each task. For example, safety-related tasks are prioritized, followed by driving-related tasks, and entertainment tasks are processed last. Within the same type of task, tasks with lower computing power requirements and those that can be completed quickly with the current remaining computing power are given priority. Based on the multi-dimensional ranking results, tasks are selected sequentially from high priority to low priority, and the computing power of these selected tasks is accumulated until the accumulated computing power requirements are close to but do not exceed the chip computing power. Finally, the selected tasks are determined as the current vehicle task and allowed to enter the subsequent processing flow.

[0063] A third embodiment of the cockpit-side multi-vehicle task adaptation method of the present invention is proposed based on the first or second embodiment described above. (Refer to...) Figure 4 , Figure 4This is a schematic diagram of the first process of the third embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention.

[0064] In this embodiment, when step S10 of the cockpit-side multi-vehicle mission adaptation method is executed, it further includes: Step S111: During the process of adapting the corresponding vehicle task to the task format through each task input adaptation layer, it is detected whether there is a second vehicle task input.

[0065] It should be noted that a second vehicle task can refer to another vehicle function task that arrives while the first vehicle task is being processed through the task input adaptation layer. For example, a forward collision warning task that is suddenly triggered while the system is processing a music recommendation task is a second vehicle task.

[0066] Step S112: If yes, then detect the priority of the second vehicle task.

[0067] It should be noted that priority refers to the order in which vehicle tasks are executed within the larger model. In this embodiment, priority is determined based on the task type of the second vehicle task. For example, safety-related tasks such as fatigue detection and collision warning are typically given the highest priority, while entertainment-related tasks such as music and casual conversation have relatively lower priorities.

[0068] Step S113: If the priority of the second vehicle task is greater than the priority of the first vehicle task, pause the format adaptation of the first vehicle task.

[0069] Understandably, if the priority of the second vehicle task is higher than that of the first vehicle task, it indicates that the second vehicle task carries a higher risk and needs to be addressed in advance. The first vehicle task can continue to execute after the second vehicle task has finished. For example, if a forward collision warning is suddenly triggered during the format adaptation process of the first vehicle task (processing music recommendations), the format adaptation process of the first vehicle task will be paused, and the adaptation and calculation process of the second vehicle task will continue.

[0070] Step S114: Adapt the task format of the second vehicle task through the task input adaptation layer corresponding to the second vehicle task.

[0071] In practical implementation, when a task input adaptation layer performs task format adaptation for a first vehicle task, it can simultaneously detect whether a second vehicle task has arrived and is waiting for processing through periodic polling or event notifications triggered by the task scheduler. If a second vehicle task is detected, its priority value or level is immediately detected or obtained, and its priority is compared with the priority of the currently being processed first vehicle task. If the comparison shows that the priority of the second vehicle task is higher than that of the first vehicle task, a pause command is immediately sent to the task input adaptation layer processing the first vehicle task, interrupting its task format adaptation process. Finally, the task input adaptation layer corresponding to the second vehicle task is called to begin task format adaptation for the second vehicle task, thus ensuring that higher-priority tasks receive timely responses. For example, the voice input adaptation layer is currently performing voice feature extraction and format adaptation for a first vehicle task (a user's navigation to the company). During the adaptation process, the driver status monitoring system detects that the driver is frequently closing their eyes, generating a second vehicle task with a driver fatigue warning. Upon detecting the second vehicle task, the task priority configuration table was immediately queried. It was found that fatigue warning belonged to the highest safety priority, while navigation belonged to a high priority but lower than the safety category. Since the second vehicle task had a higher priority than the first vehicle task, the feature extraction process for the navigation to the company task was immediately suspended, and the visual input adaptation layer corresponding to the fatigue warning task was immediately activated to adapt the format of the facial images captured by the camera, so as to quickly determine fatigue and issue an alarm.

[0072] Reference Figure 5 , Figure 5 This is a second flowchart illustrating a third embodiment of the cockpit-side multi-vehicle mission adaptation method proposed in this invention. Step S113 includes: Step S1131: Obtain temporary task data during the task format adaptation process of the first vehicle task.

[0073] It should be noted that temporary task data refers to intermediate calculation results or state data generated but not yet fully adapted during the task format adaptation process of the task input adaptation layer for the first vehicle task. Temporary task data may include partially extracted feature vectors, unprocessed raw data buffers, or activation values ​​of intermediate layers in a neural network. For example, when extracting features from a speech segment, framing, windowing, and partial Mel-frequency cepstral coefficient (MFCC) calculations may have already been completed; the feature vector fragments or activation values ​​of intermediate layers in these calculations constitute temporary task data.

[0074] Step S1132: Convert the format of the temporary task data and store it.

[0075] It should be noted that format conversion refers to the process of converting temporary task data from a data format suitable for NPU processing within a large model to another data format that can be recognized by the CPU and is suitable for long-term or temporary storage in system memory. For example, converting the NPU-specific FP16 mixed-precision tensor into a flattened buffer in the CPU-wide FP32 format and compressing it to save storage space.

[0076] Step S1133: After adapting the intermediate feature data of the second vehicle task to the vehicle function, extract the temporary task data and restore its format. It should be noted that format recovery is the reverse process of format conversion. It refers to the process of decompressing and reconverting temporary task data that has been converted and compressed and stored in system memory into the original data format adapted by high-speed computing units.

[0077] Step S1134: Based on the temporary task data after format recovery, continue to execute the step of adapting the corresponding first vehicle task to the task format through each of the task input adaptation layers to obtain task format data.

[0078] In the specific implementation, when the format adaptation of the first vehicle task is paused, all currently generated temporary task data is retrieved from the computation pipeline or cache of the task input adaptation layer that is currently performing task format adaptation. Then, a format converter is invoked to process the retrieved temporary task data, converting it from a format optimized for the computation unit to a general and compact storage format. The converted data is then written to a specific cache area in system memory or a solid-state drive for storage. Correspondingly, after the higher-priority second vehicle task completes its entire process (i.e., its converted intermediate feature data has been adapted to the vehicle function), the previously saved temporary task data belonging to the first vehicle task can be extracted from the storage area. Then, a reverse format restoration operation is performed on the extracted data, i.e., decompressing and restoring the data from the general storage format to the original format adapted by the computation unit. Finally, the format-restored temporary task data is reloaded into the corresponding task input adaptation layer, and the subsequent computation steps of task format adaptation continue from the interruption point until complete task format data is obtained. For example, a continuous speech recognition task is being executed as the first vehicle task. Its corresponding speech input adaptation layer has processed the first two seconds of audio spoken by the user, generating temporary task data of a 39-dimensional MFCC feature sequence corresponding to this audio segment, which is temporarily stored in the NPU's dedicated cache. At this time, a second vehicle task with a higher priority emergency braking warning is detected. The speech recognition task is immediately paused, and the temporary task data of the MFCC feature sequence of that two-second audio segment is retrieved. This sequence is converted from the NPU's FP16 format to FP32 format and losslessly compressed, then stored in system memory. After the braking warning task is completed and the alarm is triggered, the compressed MFCC feature data temporary task data is extracted from system memory, decompressed, and converted back to FP16 format. Finally, the MFCC feature data temporary task data is input into the adaptation layer, which then processes the user's subsequent speech, seamlessly continuing the previous recognition process.

[0079] A fourth embodiment of the cockpit-side multi-vehicle task adaptation method of the present invention is proposed based on any of the above embodiments. (Refer to...) Figure 6 , Figure 6 This is a flowchart illustrating the fourth embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention.

[0080] In this embodiment, before step S10 in the cockpit-side multi-vehicle mission adaptation method, the method further includes: Step S11: Detect the input user command and / or sensor data, and determine the first vehicle task based on the user command and / or sensor data.

[0081] It should be understood that, during the execution of vehicle functions, vehicle tasks are issued primarily through user-inputted commands and sensor data. User commands are instructions actively input to the vehicle controller to execute corresponding vehicle functions. Sensor data includes information such as the current cabin environment and unexpected situations during vehicle operation. This sensor data can be directly input to the vehicle controller, which can then execute vehicle functions based on the sensor data.

[0082] In practical implementation, upon receiving user instructions and / or sensor data, the user instructions and sensor data can be parsed to obtain the first vehicle task that needs to be executed by the large model.

[0083] Step S12: Determine the task sensitivity of the first vehicle task.

[0084] It should be understood that the vehicle task adaptation and calculation process in a large model takes a certain amount of time. However, for some vehicle tasks that have already been executed and have specific adjustment methods stored, if it is determined that the vehicle task that needs to be adapted and calculated already has specific adjustment methods, the corresponding vehicle functions can be directly adjusted by calling the stored adjustment methods. This eliminates the need to use a large model to accurately adapt and calculate the corresponding adjustment methods, effectively reducing the computing power of the vehicle chip and improving the execution speed of vehicle tasks.

[0085] It should be noted that task sensitivity can refer to the classification of vehicle function tasks in terms of safety, real-time requirements, or importance, used to determine whether the task allows for acceleration strategies such as "short-circuit inference." For example, tasks involving vehicle safety control, such as AEB warning and fatigue monitoring, or critical driving decisions, are defined as high-sensitivity tasks, requiring guaranteed calculation accuracy; while tasks such as weather and stock information queries, or casual entertainment are defined as low-sensitivity tasks, where some accuracy can be sacrificed for speed while ensuring a basic user experience.

[0086] Step S13: When the task sensitivity is lower than the preset sensitivity, obtain the similarity threshold between the feature fingerprint of the first vehicle task and each preset feature fingerprint in the preset feature fingerprint database.

[0087] It should be noted that a feature fingerprint refers to a feature identifier extracted from task input data before it has undergone preliminary processing by the task input adaptation layer, representing the core characteristics of the task input. The feature fingerprint is a unique identifier for the vehicle task input. The preset feature fingerprint database can be a database stored in system memory, which stores feature fingerprints extracted from historical high-frequency task inputs using the same process, along with their corresponding directly output results or adjustment instructions. The similarity threshold refers to the degree of matching between the current task's feature fingerprint and a historical feature fingerprint in the preset feature fingerprint database; its value ranges from 0 to 1, with higher values ​​indicating greater similarity.

[0088] Of course, if the task sensitivity is not lower than the preset sensitivity, it indicates that the vehicle task requires accurate adaptation and calculation, such as a safety-related vehicle task. These vehicle tasks require accurate adaptation and calculation to avoid potential safety hazards.

[0089] Therefore, when the task sensitivity is lower than the preset sensitivity, it indicates that the vehicle task that needs to be performed is a vehicle task that will not affect driving safety, and the similarity threshold between the feature fingerprint of the first vehicle task and each preset feature fingerprint in the preset feature fingerprint database can be obtained.

[0090] Step S14: If there is a similarity threshold greater than the preset similarity threshold, adjust the vehicle function according to the adjustment method corresponding to the preset feature fingerprint whose similarity threshold is greater than the preset similarity threshold.

[0091] It should be noted that the preset similarity threshold is a pre-set threshold value used to determine whether a match is successful, for example, 0.9. When the calculated similarity value is greater than the preset similarity threshold, a match can be considered successful. The adjustment method refers to the vehicle function adjustment method corresponding to the preset feature fingerprint in the preset feature fingerprint database when the similarity threshold between the preset feature fingerprint in the preset feature fingerprint database and the feature fingerprint of the current vehicle task is greater than the preset similarity threshold. The adjustment method in the preset feature fingerprint database can be represented as a specific instruction used to drive the execution of vehicle functions. For example, for the query "What's the weather like today?", the associated adjustment method might be a piece of structured weather information data.

[0092] Step S15: If the task sensitivity is not lower than the preset sensitivity or there is no similarity threshold greater than the preset similarity threshold, perform the step of adapting the corresponding first vehicle task to the task format through each of the task input adaptation layers to obtain task format data.

[0093] In practical implementation, user input commands, such as voice and touch commands, can be continuously detected, along with sensor data collected from vehicle sensors such as cameras and radar. Based on this multimodal input information, a comprehensive decision is made regarding the first vehicle task to be processed. The task sensitivity is compared with a preset sensitivity threshold. If the task sensitivity is lower than the preset sensitivity, meaning the current vehicle task is non-safety-critical and has an acceptable approximate result, the feature fingerprint corresponding to the current first vehicle task input is obtained. Then, the similarity value between this feature fingerprint and each historical feature fingerprint stored in the preset feature fingerprint database is calculated. It is checked whether there exists a historical feature fingerprint such that the calculated similarity value is greater than the preset similarity threshold. If so, it means that the current task is highly similar to a historical high-frequency task. The subsequent large model adaptation and calculation process will not be performed. Instead, the adjustment method associated with the successfully matched historical feature fingerprint stored in the database will be directly obtained, and this adjustment method will be used to directly drive or adjust the corresponding vehicle function. If the task sensitivity is not lower than the preset sensitivity, meaning the current vehicle task to be executed is a high-sensitivity vehicle task, or if the task sensitivity is low but there are no historical fingerprints with similarity greater than the threshold during feature fingerprint matching, then the first vehicle task is adapted to the task format through the corresponding task input adaptation layer to obtain task format data, and subsequent common processing layer calculations are performed. For example, if the user's voice command "How's the weather today?" and in-vehicle time sensor data are detected, the first vehicle task is determined to be a weather information query task. This first vehicle task is judged to be a low-sensitivity information service task. Then, the feature fingerprint of the voice command after preliminary extraction by the voice input adaptation layer is obtained, and its similarity is calculated with the historical feature fingerprints stored in the feature fingerprint database. If the similarity with the feature fingerprint of the historical high-frequency query "How's the weather?" is found to be as high as 0.95, exceeding the preset threshold of 0.9, then the adjustment method associated with the record "How's the weather?" is directly retrieved from the feature fingerprint database. For example, a structured data containing today's weather, temperature, and city information is retrieved, and this information is directly pushed to the vehicle screen for display, thus skipping the time-consuming large model calculation and achieving millisecond-level response.

[0094] A fifth embodiment of the cockpit-side multi-vehicle task adaptation method of the present invention is proposed based on any of the above embodiments. (Refer to...) Figure 7 , Figure 7 This is a flowchart illustrating the fifth embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention.

[0095] In this embodiment, after step S30 in the cockpit-side multi-vehicle mission adaptation method, the method further includes: Step S40: Execute the adapted vehicle functions and detect the bandwidth utilization of the vehicle chip equipped with the large model.

[0096] It should be understood that the execution process of vehicle functions typically involves one vehicle task performing adaptation and calculation, followed by the execution of the corresponding second vehicle function; after the first vehicle function is completed, the adaptation and calculation of the second vehicle task continues, and so on. However, the next vehicle task only begins its calculation after the previous vehicle function has been completed. The input, adaptation, and calculation processes for the next vehicle task require considerable time, resulting in slow execution of multiple vehicle functions.

[0097] It should be noted that bandwidth utilization refers to the ratio of the actual usage rate of the vehicle's infotainment chip at a given moment to its maximum theoretical bandwidth. Bandwidth utilization reflects the level of activity in data computation and transmission. A lower bandwidth utilization rate indicates that there is additional bandwidth available for additional data computation or resource transmission, while a higher bandwidth utilization rate indicates that there is no additional bandwidth available for additional data computation or resource transmission.

[0098] Step S50: If the bandwidth utilization rate is lower than the preset utilization rate, detect the execution level of the vehicle function.

[0099] It should be noted that the preset utilization rate is a pre-set threshold used to determine whether the current bandwidth is idle or under low load. The preset utilization rate can be set to 30% or 40%. If the bandwidth utilization rate is lower than the preset utilization rate, it indicates that there is additional bandwidth available for additional data computation or resource transmission; if the bandwidth utilization rate is not lower than the preset utilization rate, it indicates that there is no additional bandwidth available for additional data computation or resource transmission.

[0100] It should be noted that the execution level of a vehicle function refers to the percentage of progress or stage that the currently executing task or vehicle function has reached. The execution level can be a specific percentage of completion or the current stage of the vehicle function's execution. Execution level can include: early execution level, intermediate execution level, and late execution level. For example, if a vehicle function's execution time is 10 minutes, the first minute can be defined as the early execution level, 1 to 8 minutes as the intermediate execution level, and 8 to 10 minutes as the late execution level. The pre-completion level is a progress threshold used to trigger predictive preloading, indicating that the task is nearing completion; for example, it can be set to 80% completion or late execution level.

[0101] Step S60: When the execution level of the vehicle function reaches the pre-termination level, obtain the current scene state of the vehicle.

[0102] It should be noted that the current scene state can refer to the real-time operating environment of the vehicle, as comprehensively reflected by data from vehicle sensors, user interactions, and vehicle network signals. This includes data such as vehicle speed, geographical location, time, driver status, and currently running applications. The current scene state can be used to predict the next vehicle task to be performed.

[0103] Step S70: Predict the third vehicle task based on the current scene state using a preset rule base or inference prediction.

[0104] It should be noted that the third vehicle task refers to a vehicle function task that is predicted by rules or models based on the current scene state and has a high probability of being executed in the next moment. The third vehicle task is the vehicle task corresponding to the vehicle function that the vehicle needs to execute in the next moment after the current vehicle function has finished executing. For example, if the vehicle is reversing, the predicted third vehicle task to be executed is panoramic imaging.

[0105] It should be noted that the preset rule base is a predefined set of rules that maps the relationship between scene states and prediction tasks based on explicit logical conditions. The preset rule base includes the mapping relationship between third-vehicle tasks and the current scene state. Given a fixed current scene state, there are one or more corresponding third-vehicle tasks.

[0106] Inference prediction refers to using a trained machine learning model to predict the type of task a user might initiate next, based on the feature vector of the current scene state. For example, it can predict that there is an 80% probability that the user will ask about nearby parking lots.

[0107] Step S80: Input the third vehicle task into the buffer area of ​​the task input adaptation layer.

[0108] It should be noted that the cache area can refer to a portion of the vehicle system memory (such as DDR) or a dedicated high-speed cache such as the shared cache of the GPU or NPU, which is specifically used to silently load the model components required by the third vehicle task in the background in advance to achieve a fast response when switching tasks.

[0109] In the specific implementation, during the execution of the adapted vehicle functions, a predictive preloading process can be executed in parallel. The data bandwidth utilization of the vehicle's infotainment chip equipped with the large model is monitored in real time by a resource monitoring module. The detected bandwidth utilization is compared with a preset utilization threshold. If the bandwidth utilization is lower than the preset threshold, it indicates that the system's data transmission resources are relatively idle, meeting the conditions for background preloading. The completion status of the currently executing vehicle function is detected, i.e., the execution level, and it is determined whether the execution level has reached or exceeded the pre-completion level. If the pre-completion level has been reached, it means that the current task is about to be completed. At this time, the next vehicle task, i.e., the third vehicle task, can be predicted and loaded. The comprehensive current scene state of the vehicle can be obtained through various data sources such as sensors, buses, and application programming interfaces (APIs). Then, the current scene state is used as input to query a preset rule base for matching, and may also call an inference prediction model for calculation, comprehensively deriving one or more third vehicle tasks most likely to be triggered. Finally, the model components corresponding to the predicted third vehicle task are preloaded into a designated cache area to prepare for possible immediate invocation. For example, during the process of setting the air conditioning temperature to 22 degrees Celsius, the bandwidth utilization between the NPU and memory was detected to be only 25%, lower than the preset threshold of 40%. Simultaneously, the execution progress of the air conditioning adjustment task was detected to have reached 90%. At this point, the current scenario is that the vehicle is traveling on a highway at 3 PM, the navigation indicates that the next service area is 20 kilometers away, and the driver has not rested in the past hour. Based on this scenario, combined with preset rules and inference model predictions, it is determined that the driver is highly likely to inquire about or need to find information about the nearest service area. At this point, the input adapter corresponding to this predicted third vehicle task—service area / POI search—such as a voice or map search adapter, can be silently loaded into the NPU's backup cache in the background. When the driver subsequently actually requests to find a nearby service area, there is no need to wait for the model components to load; the already prepared adapter can be directly called from the cache, achieving a millisecond-level response.

[0110] A sixth embodiment of the cockpit-side multi-vehicle task adaptation method of the present invention is proposed based on any of the above embodiments. (Refer to...) Figure 8 , Figure 8 This is a flowchart illustrating the sixth embodiment of the cockpit-side multi-vehicle task adaptation method proposed in this invention.

[0111] In this embodiment, the cockpit-side multi-vehicle mission adaptation method further includes: Step S90: During the operation of the common processing layer, the task input adaptation layer, and the task output adaptation layer, the load rate of the neural network processing unit of the large model is detected.

[0112] It should be noted that the neural network processing unit (NNPU) can refer to a core hardware module within an in-vehicle infotainment chip dedicated to performing large-scale neural network calculations, such as a neural processing unit, graphics processing unit, or tensor processing unit. In a smart cockpit environment, the NNPU is typically the main computing unit responsible for the computationally intensive parts of large models, such as matrix operations in feature enhancement layers. Load rate refers to the ratio of the actual usage of the NNPU's computing resources, such as computing cores and memory bandwidth, to its maximum theoretical capacity over a period of time; it is usually expressed as a percentage and indicates its workload.

[0113] Step S100: When the load rate reaches the preset load rate, reduce the amount of vehicle task data input to the task input adaptation layer.

[0114] It should be noted that the preset load rate is a pre-set threshold used to trigger resource adaptive adjustment strategies. When the real-time load rate of the neural network processing unit reaches or exceeds this threshold, it indicates that its computing resources are approaching saturation, and a degradation strategy needs to be implemented to ensure system stability. Data volume refers to the size or information density of the original task data input to the task input adaptation layer for processing. For different modalities, reducing data volume can be manifested in different specific operations. For example, for speech tasks, the audio sampling rate can be reduced; for image tasks, the image resolution can be reduced or compressed; for text tasks, cropping or summarizing can be performed.

[0115] The preset load rate can be divided into a first preset load rate and a second preset load rate, with the first preset load rate being lower than the second preset load rate. For example, the first preset load rate can be set to 60%, and the second preset load rate can be set to 90%. When the load rate reaches the first preset load rate, the load rate of the neural network processing unit can be reduced by decreasing a small amount of input data, such as adjusting the resolution of the input image. When the load rate reaches the second preset load rate, a large amount of input data needs to be reduced to decrease the load rate of the neural network processing unit, such as simultaneously reducing the resolution of the input image and unloading inactive adapters, such as the adapters of the input adaptation layer or the output adaptation layer that are in an idle state.

[0116] In the specific implementation, during the computational work performed in the common processing layer, task input adaptation layer, and task output adaptation layer, the vehicle chip resource monitoring module continuously or periodically monitors the real-time load rate of the neural network processing unit (NPU). The detected real-time load rate is compared with a preset load rate threshold. If the real-time load rate reaches or exceeds the preset threshold, it indicates that the neural network processing unit is under high load, posing a risk of computational resource bottleneck. At this point, to prevent system lag or task failure and to ensure that high-priority tasks can still run, a degradation strategy is triggered, dynamically reducing the amount of raw data from subsequent vehicle tasks input to the task input adaptation layer. For example, instructions may be sent to the data acquisition module or preprocessing module, requesting them to reduce the scale or precision of the data sent to the task input adaptation layer. By reducing the amount of input data, the computational burden on the task input adaptation layer and subsequent common processing layers can be reduced, thereby quickly reducing the overall load of the neural network processing unit and maintaining the system's basic service capability under resource constraints.

[0117] For example, during driving, the cockpit system simultaneously runs multiple tasks such as multimodal voice interaction, real-time in-vehicle panoramic stitching, and driver posture recognition, causing the NPU load rate to consistently exceed 90%, reaching the preset 85% load rate threshold. At this point, a dynamic degradation strategy is immediately initiated. For the computationally resource-sensitive vehicle task of real-time in-vehicle panoramic stitching, the system sends instructions to its corresponding image input adaptation layer, requesting that the original resolution of the input image be reduced from 1920x1080 to 1280x720, or that the frequency of image frames input to the large model be reduced. Due to the significant reduction in the amount of input image data, the amount of convolution and matrix operations required by the image input adaptation layer and subsequent common processing layers also decreases significantly. This reduces the overall NPU load rate from 90% to 70%, preventing task crashes or system sluggishness caused by computational overload, and ensuring the stable execution of high-priority safety tasks such as driver posture recognition.

[0118] Furthermore, in this embodiment, the data processing mechanism of the large model can be adjusted, for example, the high-precision data processing mechanism using INT8 in the large model can be adjusted to the low-precision data processing mechanism of INT4, thereby further reducing the amount of data processed in the large model, avoiding task crashes or slow system response caused by computing power overload, and ensuring the stable execution of high-priority safety tasks such as driver posture recognition.

[0119] Furthermore, to achieve the above objectives, the present invention also provides a cockpit-side multi-vehicle task adaptation device, which is used in the cockpit-side multi-vehicle task adaptation method described in any of the above embodiments. The cockpit-side multi-vehicle task adaptation device may be a vehicle controller or an additionally connected controller.

[0120] In addition, to achieve the above objectives, the present invention also provides a vehicle, wherein the vehicle is equipped with a data acquisition sensor, a human-machine interface terminal, a vehicle-mounted chip, and a cockpit-side multi-vehicle task adaptation device, wherein a large model is mounted on the vehicle-mounted chip, and the data acquisition sensor and the human-machine interface terminal are used to acquire data corresponding to vehicle tasks and input them into the large model in the vehicle-mounted chip.

[0121] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A cockpit-side multi-vehicle mission adaptation method, characterized in that, It is applicable to large models that have a common processing layer, at least two task input adaptation layers, and at least two task output adaptation layers. The cockpit-side large model scheduling methods include: The task input adaptation layers are used to adapt the corresponding first vehicle task to the task format to obtain task format data. The common processing layer performs task data calculations on each of the task format data to obtain intermediate feature data. The intermediate feature data is formatted by a task output adaptation layer corresponding to the task input adaptation layer, and the converted intermediate feature data is then adapted to the vehicle functions.

2. The cockpit-side multi-vehicle mission adaptation method as described in claim 1, characterized in that, The cockpit-side multi-vehicle mission adaptation method also includes: The computing power of the vehicle's infotainment chip equipped with the large model and the computing power required for each vehicle's tasks are detected. If the sum of the computing power required by each vehicle task is greater than the chip computing power, the current vehicle task is determined based on the chip computing power and the computing power required by each vehicle task. The current vehicle task is adapted to the task format through the corresponding input adaptation layer to obtain task format data.

3. The cockpit-side multi-vehicle mission adaptation method as described in claim 2, characterized in that, The process of determining the current vehicle task based on the chip's computing power and the task computing power required by each vehicle task includes: Obtain the task type for each of the first vehicle tasks; The vehicle tasks are prioritized based on the chip computing power, the task type, and the computing power required for each of the first vehicle tasks. The current vehicle task is determined based on the sorting results.

4. The cockpit-side multi-vehicle mission adaptation method as described in claim 1, characterized in that, The cockpit-side multi-vehicle mission adaptation method also includes: During the process of adapting the corresponding vehicle task to the task format through each of the task input adaptation layers, it is detected whether there is a second vehicle task being input. If so, then check the priority of the second vehicle's task; If the priority of the second vehicle task is higher than that of the first vehicle task, the format adaptation of the first vehicle task is suspended. The task input adaptation layer corresponding to the second vehicle task adapts the task format.

5. The cockpit-side multi-vehicle mission adaptation method as described in claim 4, characterized in that, The suspension of the first vehicle task format adaptation includes: Obtain temporary task data during the task format adaptation process of the first vehicle task; The temporary task data is formatted and stored; Accordingly, after the intermediate feature data of the second vehicle task is converted and adapted to the vehicle function, the temporary task data is extracted and the format is restored. Based on the temporary task data after format restoration, continue to execute the step of adapting the corresponding first vehicle task to the task format through each of the task input adaptation layers to obtain task format data.

6. The cockpit-side multi-vehicle mission adaptation method as described in claim 1, characterized in that, The cockpit-side multi-vehicle mission adaptation method also includes: Detect input user commands and / or sensor data, and determine a first vehicle task based on the user commands and / or sensor data; Determine the task sensitivity of the first vehicle mission; When the task sensitivity is lower than the preset sensitivity, the similarity threshold between the feature fingerprint of the first vehicle task and each preset feature fingerprint in the preset feature fingerprint database is obtained. If the similarity threshold is greater than the preset similarity threshold, the vehicle function will be adjusted according to the adjustment method corresponding to the preset feature fingerprint whose similarity threshold is greater than the preset similarity threshold. If the task sensitivity is not lower than the preset sensitivity or there is no similarity threshold greater than the preset similarity threshold, the step of adapting the corresponding first vehicle task to the task format through each of the task input adaptation layers to obtain task format data is executed.

7. The cockpit-side multi-vehicle mission adaptation method as described in claim 1, characterized in that, The step of converting the format of the intermediate feature data through the task output adaptation layer corresponding to the task input adaptation layer, and adapting the converted intermediate feature data to the vehicle functions, further includes: Execute the adapted vehicle functions and detect the bandwidth utilization of the vehicle chip equipped with the large model; When the bandwidth utilization rate is lower than the preset utilization rate, the degree of execution of the vehicle function is detected; When the execution level of the vehicle function reaches the pre-termination level, the current scene state of the vehicle is obtained; Based on the current scene state, the task of the third vehicle is predicted using a preset rule base or inference prediction. The third vehicle task is input into the cache area of ​​the task input adaptation layer.

8. The cockpit-side multi-vehicle mission adaptation method as described in claim 1, characterized in that, The cockpit-side multi-vehicle mission adaptation method also includes: During the operation of the common processing layer, the task input adaptation layer, and the task output adaptation layer, the load rate of the neural network processing unit of the large model is detected. When the load rate reaches the preset load rate, the amount of vehicle task data input to the task input adaptation layer is reduced.

9. A cockpit-side multi-vehicle mission adaptation device, characterized in that, The cockpit-side multi-vehicle mission adaptation device is used to perform the cockpit-side multi-vehicle mission adaptation method according to any one of claims 1 to 8.

10. A vehicle, characterized in that, The vehicle is equipped with a data acquisition sensor, a human-machine interface terminal, a vehicle-mounted chip, and the cockpit-side multi-vehicle task adaptation device as described in claim 9. The vehicle-mounted chip has a large model on it. The data acquisition sensor and the human-machine interface terminal are used to acquire data corresponding to vehicle tasks and input them into the large model in the vehicle-mounted chip.

Images