An adaptive voice control vehicle method and device, vehicle and storage medium

Abstract

The present application relates to the technical field of vehicle control, and discloses a self-adaptive voice control method and device, a vehicle and a storage medium, the method comprising: obtaining first load information, the first load information being used to reflect the load level of multi-dimensional hardware in the vehicle; matching a corresponding current performance level according to the first load information, and compressing the inference ability of a large language model according to the current performance level, the inference ability being used to represent the input information amount and the output control message amount of the large language model; processing the voice instruction of a user based on the compressed large language model to obtain a control message used for controlling the vehicle. The present application solves the problem of dynamic balance between the inference efficiency of a large model and hardware resources in the field of vehicles.

Description

Technical Field

[0001] This invention relates to the field of vehicle control technology, specifically to an adaptive voice control method, device, vehicle, and storage medium for vehicle control. Background Technology

[0002] With the widespread application of large language models in edge computing scenarios, using large models to infer the meaning of user speech in vehicles to assist users in vehicle control is a popular technology. However, the heterogeneous computing platforms in vehicles (including central processing units, neural network processors, memory, and graphics processors) are resource-constrained. Performing inference on resource-limited heterogeneous computing platforms poses a challenge to the hardware resources for vehicle control, and there is an urgent need to solve the dynamic balance problem between the efficiency of large model inference and hardware resources in the vehicle domain. Summary of the Invention

[0003] This invention provides an adaptive voice-controlled vehicle method, device, vehicle, and storage medium to solve the problem of dynamically balancing inference efficiency and hardware resources in large vehicle models.

[0004] In a first aspect, the present invention provides an adaptive voice-controlled vehicle method, the method comprising: acquiring first load information, the first load information being used to reflect the load level of multi-dimensional hardware in the vehicle; matching the corresponding current performance level according to the first load information, and compressing the inference capability of a large language model according to the current performance level, the inference capability being used to represent the amount of input information and output control messages of the large language model; and processing the user's voice commands based on the compressed large language model to obtain control messages for controlling the vehicle.

[0005] Based on the aforementioned technical methods, by acquiring the first load information of the vehicle's multi-dimensional hardware, matching the corresponding performance level, and dynamically compressing the inference capability of the large language model, the problem of balancing resources and efficiency in edge-side voice-controlled vehicles under limited resource environments is solved at the core level. Compared with traditional single-processor load judgment schemes, this method covers multi-dimensional hardware load states, avoiding inference delays or resource waste caused by misjudgments of single hardware loads, and ensuring that large model inference adapts to the overall hardware carrying capacity. By accurately matching performance levels to adjust the amount of input information and output control messages, the intelligence level of voice-controlled vehicles is guaranteed when hardware resources are sufficient, and real-time response needs are prioritized when resources are scarce. This avoids the impact of inference task lag on vehicle control timeliness under high load, and solves the problem of functional failure caused by unbalanced resource allocation, achieving a dynamic unity of stability, real-time performance, and intelligence in voice-controlled vehicles under different hardware load scenarios.

[0006] In some optional implementations, before obtaining the first load information, the method further includes: determining a multi-dimensional hardware experimental sample with the same hardware capacity as the vehicle; loading stress test data into the multi-dimensional hardware experimental sample and defining corresponding performance levels for different load scenarios achieved by the multi-dimensional hardware experimental sample, wherein the stress test data is used to apply load to the hardware experimental sample to achieve multiple load scenarios; performing inter-process communication tasks and large language model inference tasks through the stressed hardware experimental sample; and creating an experience database based on the performance level corresponding to each load scenario, the inter-process communication call time corresponding to each load scenario, and the inference time corresponding to each load scenario.

[0007] Based on the aforementioned technical methods, a multi-dimensional hardware experimental sample with the same capacity as the vehicle's hardware was constructed. Performance levels were defined through stress testing, and an experience database was created, providing accurate and efficient reference for subsequent load judgment and inference capability adjustments. By covering various load scenarios in advance through stress testing, a correspondence between performance levels and communication and inference times was established, avoiding the additional system overhead of real-time hardware load monitoring and reducing edge-side computational pressure. The establishment of the experience database makes load level matching more accurate and faster, eliminating the need for complex real-time load assessments during user voice interaction and shortening the response chain. Simultaneously, stress testing simulating a real hardware environment ensured the fit of the performance level definition, making subsequent inference capability adjustments based on this level more consistent with actual hardware operating conditions. This laid a solid foundation for the efficient execution of voice-controlled vehicles and improved the overall reliability and adaptability of the solution.

[0008] In some optional implementations, obtaining the first load information includes: when receiving user voice, calculating the inter-process communication call time for speech-to-speech conversion, and using the speech-to-speech inter-process communication call time as the first load information.

[0009] Based on the aforementioned technical methods, the inter-process communication call time for speech-to-semantics conversion is used as the primary load information, simplifying the acquisition of load information. Simultaneously, the use of an experience database ensures the targetedness and effectiveness of load assessment. The inter-process communication call time directly reflects the hardware's response efficiency in the critical stages of speech-to-semantics conversion, quickly mapping the comprehensive load status of multi-dimensional hardware without requiring the collection of complex multi-dimensional hardware parameters, thus reducing data acquisition complexity and system resource consumption. This method can quickly acquire load information at the initial stage of user-initiated voice commands, providing time for subsequent performance level matching and inference capability adjustments, meeting the real-time requirements of voice-controlled vehicles. Furthermore, the communication call time is directly related to the core processes of voice-controlled vehicles. Using this as the basis for load assessment accurately reflects the hardware's support capability for key aspects of voice-controlled vehicles, ensuring that subsequent inference capability adjustments are highly aligned with actual needs, avoiding decision-making biases caused by irrelevant load data interference, and improving the accuracy and timeliness of voice-controlled vehicle responses.

[0010] In some optional implementations, the corresponding current performance level is matched based on the first load information, including: determining the current performance level by comparing the first load information with the similarity of inter-process communication call times under various load scenarios in the experience database.

[0011] Based on the aforementioned technical methods, the current performance level is determined by matching the similarity between the initial load information and the communication call time in the experience database. This achieves both high efficiency and accuracy in load level judgment. The experience database already covers communication time data under various load scenarios. Similarity matching eliminates the need for complex algorithmic calculations, reducing the computational burden on the device side and ensuring rapid level judgment even in resource-constrained vehicle environments. Furthermore, the fully utilized pre-constructed experimental data avoids the uncertainties of real-time load assessment, resulting in more consistent and reliable performance level determination results.

[0012] In some optional implementations, the performance level includes an unloading level and at least one compression level. Compressing the inference capability of the large language model based on the current performance level includes: if the current performance level is an unloading level, unloading the large language model; if the current performance level is a compression level when the large language model is not loaded, determining whether the performance level matched for a previous preset number of consecutive times is an unloading level; if the performance level matched for a previous preset number of consecutive times is not an unloading level, loading the large language model and limiting the amount of input information and output control messages of the large language model according to the corresponding compression amount.

[0013] Based on the aforementioned technical methods, performance level classifications and corresponding inference capability adjustment strategies were defined, enabling refined and flexible control of large model inference capabilities to adapt to hardware environments with varying load intensities. For the unloading level, directly unloading the large model quickly releases hardware resources, prioritizing the normal execution of basic voice-controlled vehicle functions and avoiding response delays or functional failures caused by large models consuming resources under high loads. For the compression level, by judging historical continuous performance levels, frequent model starts and stops due to temporary load fluctuations are avoided, reducing system overhead and response delays caused by model loading and unloading, and improving the consistency of the user experience. Simultaneously, input information and output message volume are limited according to the compression level, maximizing the use of limited hardware resources while ensuring core vehicle control needs. This avoids functional deficiencies due to resource scarcity and prevents limitations in inference capabilities from affecting the intelligent experience when resources are plentiful, achieving an optimal balance between resource utilization and functional experience under different load scenarios.

[0014] In some optional implementations, the user's voice commands are processed based on the compressed large language model to obtain control messages for vehicle control, including: when the large language model is unloaded, calling a pre-saved command-message lookup table; if the target control message corresponding to the voice command is found through the command-message lookup table, then the target control message is output; when the large language model is not unloaded, generating inference prompts for the voice commands according to the input information limits corresponding to the compression level, and inputting the inference prompts into the large language model for inference to obtain inference control messages.

[0015] Based on the aforementioned technical means, differentiated voice command processing methods are designed for different model states, ensuring the integrity and efficiency of the voice-controlled vehicle function. When the model is unloaded, the target control message is directly output by calling the pre-stored command-message lookup table, skipping the complex inference process, ensuring the rapid response of basic vehicle control functions, and meeting the real-time requirements of voice-controlled vehicle under high load scenarios. When the model is not unloaded, inference prompts are generated according to the compression level and inference is performed, which controls the model input scale to adapt to hardware resources, and ensures the intelligence and accuracy of vehicle control commands through large model inference. The above technical means take into account the needs of different load scenarios, prioritizing the availability of functions when resources are scarce, and improving the intelligent experience when resources are abundant, avoiding the limitations of a single processing method under different loads. At the same time, the combination of the pre-stored lookup table and model inference reduces the system pressure under high load while retaining the voice control capability under low load, achieving a balance between the flexibility and practicality of the voice-controlled vehicle function.

[0016] In some optional implementations, the method further includes: at the end of inference, calculating the current inference time and using the current inference time as the second load information; updating the current performance level based on the similarity between the second load information and the inference time in each load scenario in the experience database; and adjusting the number of control messages according to the updated current performance level.

[0017] Based on the aforementioned technical means, at the end of inference, the performance level is updated and the number of control messages is adjusted according to the current inference time, forming a closed-loop feedback mechanism for load status, ensuring the dynamic adaptability of inference capability adjustments. Inference time directly reflects the actual carrying capacity of the hardware during the execution of large model inference. Updating the level in conjunction with initial load information provides a more comprehensive and accurate reflection of hardware load changes, avoiding decision lag caused by relying solely on initial load judgments. After the level update, the number of control messages is adjusted promptly to match the output results with the real-time hardware status. When the hardware load increases during inference, the number of control messages is reduced to lower system pressure; when the load decreases, the number of messages can be appropriately increased to enhance the intelligent experience. This dynamic adjustment mechanism allows the voice-controlled vehicle solution to respond to hardware load fluctuations in real time, avoiding the incompatibility of fixed control strategies in scenarios with changing loads. This further improves the stability, real-time performance, and resource utilization efficiency of voice-controlled vehicles, ensuring consistently good operating conditions in complex and ever-changing hardware load environments.

[0018] In some alternative implementations, the multidimensional hardware includes a central processing unit, a neural network processor, and memory.

[0019] Secondly, the present invention provides an adaptive voice-controlled vehicle device, comprising: a load acquisition module for acquiring first load information, the first load information being used to reflect the load level of multi-dimensional hardware in the vehicle; a performance level evaluation module for matching the corresponding current performance level according to the first load information, and compressing the reasoning ability of a large language model according to the current performance level, the reasoning ability being used to represent the amount of input information and output control messages of the large language model; and a decision module for reasoning about the user's voice commands based on the processed large language model to obtain control messages for controlling the vehicle.

[0020] Thirdly, the present invention provides a vehicle comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0021] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof.

[0022] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating an adaptive voice-controlled vehicle method according to an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of the software architecture of an adaptive voice-controlled vehicle method according to an embodiment of the present invention;

[0026] Figure 3 This is another software architecture diagram of an adaptive voice-controlled vehicle method according to an embodiment of the present invention;

[0027] Figure 4 This is another flowchart illustrating an adaptive voice-controlled vehicle method according to an embodiment of the present invention;

[0028] Figure 5 This is another flowchart illustrating an adaptive voice-controlled vehicle method according to an embodiment of the present invention;

[0029] Figure 6 This is a schematic diagram of the structure of an adaptive voice-controlled vehicle device according to an embodiment of the present invention;

[0030] Figure 7 This is a schematic diagram of the hardware structure of a vehicle according to an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0033] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0034] With the widespread application of large language models in edge computing scenarios, using large models to infer the meaning of user speech in vehicles to assist users in vehicle control is a popular technology. However, the heterogeneous computing platforms in vehicles (including central processing units, neural network processors, memory, and graphics processors) are resource-constrained. Performing inference on resource-limited heterogeneous computing platforms poses a challenge to the hardware resources for vehicle control, and there is an urgent need to solve the dynamic balance problem between the efficiency of large model inference and hardware resources in the vehicle domain.

[0035] Some technologies determine whether to use a large language model for inference based on the load of a single processor, but these solutions have the following main problems: (1) When the load of a single processor is low but the load of other processors or memory is high, the inference task of the large model may still be unable to proceed normally due to hardware limitations. In this case, performing large model inference may still cause high latency and make it impossible to control the vehicle quickly; (2) When the processor load is high, inference will be paused and the inference will continue and the vehicle will be controlled when the processor resources are idle. However, voice vehicle control has high timeliness requirements, and this method cannot meet the requirements of rapid vehicle control; (3) When the target processor load is high, in order to improve efficiency, the work that should be done by the target processor (such as tensor operations) is misassigned to other processors. However, other processors do not have the function and ability to handle this task, which makes the work impossible and makes the vehicle control efficiency even slower. In summary, it is urgent to solve the problem of dynamic balance between model inference efficiency and hardware resources for vehicles.

[0036] According to an embodiment of the present invention, an adaptive voice-controlled vehicle method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] This embodiment provides an adaptive voice control method for vehicles. Figure 1 This is a flowchart of an adaptive voice-controlled vehicle method according to an embodiment of the present invention, the process including the following steps:

[0038] Step S101: Obtain first load information, which is used to reflect the load level of multi-dimensional hardware in the vehicle.

[0039] Step S102: Match the corresponding current performance level according to the first load information, and compress the reasoning ability of the large language model according to the current performance level. The reasoning ability is used to represent the amount of input information and output control messages of the large language model.

[0040] Step S103: Process the user's voice commands based on the compressed large language model to obtain control messages for controlling the vehicle.

[0041] Specifically, this embodiment provides an adaptive voice control method for vehicles. This method is mainly applied to environments with limited resources on the vehicle side (such as scenarios where the vehicle is simultaneously running navigation, music playback, and vehicle system background services, resulting in tight hardware resource allocation). It aims to dynamically perceive the hardware load status and flexibly adjust the inference ability of the large language model to maximize the use of limited hardware resources to achieve intelligent vehicle control while ensuring the real-time performance of voice control.

[0042] The acquired "first load information" refers to key data that directly or indirectly reflects the current operating pressure of the vehicle-side multi-dimensional hardware. Its core function is to provide an objective basis for subsequent judgments on whether the hardware can support large language model inference. "Multi-dimensional hardware" specifically refers to the core hardware components on the vehicle side that support voice signal processing, semantic parsing, and large model inference, including but not limited to the central processing unit (CPU, responsible for coordinating instruction scheduling, data processing, and process management, and is the central hub of the voice control process), the neural network processor (NPU, hardware specifically designed to accelerate artificial intelligence model inference operations, which can significantly improve the processing efficiency of large language models), and memory (used for temporary storage of voice data, model parameters, and intermediate results of operations; the sufficiency of memory directly affects data reading speed). Therefore, the first load information includes, but is not limited to, the load data of the multi-dimensional hardware (such as CPU utilization, NPU utilization, and available memory capacity) and the inter-process communication call time for voice-to-semantics conversion when user voice is received.

[0043] In actual operation, the system collects core operating parameters of multi-dimensional hardware in real time through the underlying hardware monitoring interface of the vehicle operating system. For example, when the user says the voice command "turn on the driver's side air conditioning", the system immediately triggers parameter collection. Assuming that the collected CPU utilization rate is 58%, NPU utilization rate is 45%, and the remaining available memory capacity is 1.1GB, the hardware parameters are then converted into indicators that can intuitively reflect the overall hardware load (such as inter-process communication response time, i.e., the time taken to transfer voice semantic data between different processing modules). Finally, the first load information is formed. The whole process does not require complex algorithm calculations. The hardware load status can be quickly obtained through real-time parameter collection and simple conversion, avoiding excessive consumption of system resources.

[0044] Then, the corresponding current performance level is matched according to the first load information, and the reasoning ability of the large language model is compressed according to the current performance level. The performance level is a category of operating status divided according to the multi-dimensional hardware load intensity of the vehicle, which is used to clarify the boundary of the reasoning ability of the large language model under different loads. The reasoning ability of the large language model is specifically manifested in "the amount of input information" (in units of tokens, where tokens are the basic unit for the large language model to process text data, and the amount of input information determines the length of speech semantic text that the model can receive, for example, long speech commands require more tokens) and "the amount of output control messages" (i.e., the number of commands that the model can generate to control vehicle functions, such as multiple commands to control the air conditioner and seats at the same time).

[0045] In this step, the first step is to establish a correspondence between the initial load information and the performance level. For example, four performance levels are preset: when the initial load information shows CPU utilization <60%, NPU utilization <60%, and available memory ≥1GB, it is matched as a low load level (Q1); when 60%≤CPU utilization <80%, 60%≤NPU utilization <80%, and 0.5GB≤available memory <1GB, it is matched as a low-to-medium load level (Q2); when 80%≤CPU utilization <95%, 80%≤NPU utilization <95%, and 0.2GB≤available memory <0.5GB, it is matched as a medium-to-high load level (Q3); when CPU utilization ≥95%, NPU utilization ≥95%, and available memory <0.2GB, it is matched as a high load level (Q4). Then, the inference capability is adjusted according to the matched performance level. If the first load information corresponds to Q1 level (sufficient hardware resources), then the inference capability is not compressed, and the input information volume is set to 600-650 tokens (able to fully receive long voice commands containing multiple needs, such as "turn on the driver's air conditioning and set it to 24℃, and simultaneously turn on the seat ventilation at level 1"), and the output control message volume is 6-8 messages (supporting simultaneous control of multiple vehicle functions); if it corresponds to Q3 level (limited hardware resources), then the inference capability is compressed, reducing the input information volume to 100-150 tokens (only retaining the core semantics of the voice command, such as "turn on the driver's air conditioning"), and the output control message volume is limited to 1-2 messages (only responding to the most core vehicle control needs); if it corresponds to Q4 level (extremely limited hardware resources), then the large language model inference is directly stopped (i.e., the inference capability is completely compressed) to avoid the model occupying resources and causing the basic voice vehicle control function to lag.

[0046] After the above compression process, the large language model processes the user's voice commands to obtain control messages for vehicle control. The control messages are instruction data that conform to the vehicle control system data format and contain three core types of information: "controlled objects" (such as vehicle functional modules such as air conditioning, seats, and windows), "control actions" (such as opening, closing, and adjusting parameters), and "control parameters" (such as specific values ​​such as temperature 24℃ and fan speed level 2). These can be directly recognized by the vehicle actuators and the corresponding operations can be executed. Control messages need to be processed differently depending on the reasoning ability of the large language model (i.e., whether it is in a reasoning-capable state): If the model is in a reasoning-capable state (e.g., matching Q1, Q2, or Q3 levels), the user's voice command is first converted into text semantics (e.g., if the user says "The driver's seat is a bit cold," it is converted into the semantic text "The temperature in the driver's seat area is too low and needs to be increased"). Then, the semantic text is filtered according to the "inputable information limit" (retaining core information and removing redundant expressions), generating text data that meets the model's input requirements and inputting it into the model. The model generates a corresponding number of control messages according to the "output control message limit," for example, at Q1 level, it generates two control messages: "Adjust the driver's seat air conditioning temperature to 26℃; turn on the driver's seat heating to level 1." If the model is in a stopped reasoning state (e.g., matching Q4 level), the preset basic vehicle control command library is directly called. The system stores the mapping relationship between common voice commands and corresponding control messages. For example, "turn on the driver's side air conditioning" corresponds to "control object: driver's side air conditioning, control action: turn on, control parameters: temperature 25℃, fan speed level 2". It can quickly output control messages without going through the model inference process. For example, when the user says "heat up the driver's side air conditioning and turn on the seat heater" while the vehicle is running navigation, music and video in the background (CPU utilization 90%, NPU utilization 88%, memory available 0.3GB, matching Q3 level), the system first filters the semantic text to "heat up the driver's side air conditioning and turn on the seat heater" (complies with the 100-150 token limit). After inputting it into the compressed model, the model generates two control messages: "adjust the driver's side air conditioning temperature to 27℃; turn on the driver's side seat heater level 1". After the format is verified to be correct, the message is transmitted to the vehicle control system to execute the corresponding vehicle control operation.

[0047] like Figure 2The diagram shown is a software framework diagram of an embodiment of the present invention. The voice app (voice application) is used to receive the user's voice commands, and the adaptive vehicle control app (adaptive vehicle control application) is the core software module for executing the above-described method flow of the embodiment of the present invention. It includes a command sending and receiving module, a command (audio) to semantic (text) conversion module, a performance level module, a semantic discrimination module, a prompt construction module (prompt word construction module), a cabin interior and exterior environment perception module (used to collect cabin interior and exterior environment information required for large model inference, including monitoring and conversion modules for various types of data), an edge-side large model inference module, and a streaming parsing module for inference results. During the initialization phase, the adaptive vehicle control app performs the following tasks simultaneously: 1. Establishes links and registers for listening with the basic capabilities provided by CarService (the vehicle's main service), navigation service, and image service, updating the cabin and interior / exterior environmental information in real time; 2. Initializes the large-scale model on the device side. When the user initiates a voice command, the voice app sends the semantic result and timestamp to the adaptive vehicle control app via IPC (Inter-Process Communication). The performance level module updates the current performance level according to the predefined load level definition. If it is Q4, the received semantic result is returned to the voice app as is; otherwise, the speaker information is updated, and the prompt construction module is called to build the corresponding level prompt according to the load level, recording the current timestamp before performing large-scale model inference. After inference, the control message is returned to the voice app, which transmits the control message to the vehicle system through the vehicle's built-in vehicle control app (vehicle control application), calling various in-vehicle services (including but not limited to the vehicle's main service CarService, navigation service GPS-Service, cabin / passenger monitoring system services Ims / Oms-Service, etc.) to perform specific control of the vehicle.

[0048] like Figure 3 The diagram shown is another software framework diagram of an embodiment of the present invention, which illustrates in detail how the prompt building module determines which perception data from the cabin interior and exterior environment perception modules to use for inference based on the performance level determined by the performance level module.

[0049] Based on the aforementioned technical methods, this approach overcomes the limitation of traditional voice-controlled vehicle solutions that rely solely on single hardware load assessments. By collecting multi-dimensional hardware load information to generate primary load information, it more comprehensively and accurately reflects the actual hardware capacity, avoiding model inference lag or resource waste caused by misjudgments based on a single hardware load. For example, traditional solutions only consider CPU load (e.g., CPU utilization of 50%) to initiate model inference, ignoring that the NPU is already fully loaded (98%), leading to inference delays. This method avoids such problems by using multi-dimensional information assessment. Through dynamic matching of performance levels and inference capabilities, it achieves precise adaptation between "hardware resources and inference requirements." When hardware is sufficient, it ensures the intelligence of voice-controlled vehicles (supporting multiple requirements and multiple command outputs), while prioritizing real-time performance when resources are scarce (compressing inference capabilities without affecting core functions). This avoids the problems of "underutilized inference capabilities when resources are sufficient" or "functional failure due to inference consuming resources when resources are scarce." It simplifies the inference capability adjustment and command processing logic, eliminating the need for complex real-time algorithm optimization. It can achieve efficient vehicle control simply by setting preset levels and corresponding strategies. In environments with limited resources on the vehicle side, it can effectively reduce system operating overhead while ensuring stable voice control response time, significantly improving the user experience. Moreover, it does not require additional hardware costs and can be implemented simply through software logic optimization, making it highly practical and valuable for engineering promotion.

[0050] In some optional implementations, prior to step S101 described above, the method further includes:

[0051] Step a1: Determine the multidimensional hardware experimental sample with the same hardware capacity as the vehicle;

[0052] Step a2: Load the stress test data into the multi-dimensional hardware experimental sample and define the corresponding performance level for the different load scenarios reached by the multi-dimensional hardware experimental sample. The stress test data is used to apply load to the hardware experimental sample to achieve multiple load scenarios.

[0053] Step a3: Perform inter-process communication tasks and large language model inference tasks using the pressurized hardware experimental sample;

[0054] Step a4: Create an experience database based on the performance level corresponding to each load scenario, the inter-process communication call time corresponding to each load scenario, and the inference time corresponding to each load scenario.

[0055] Specifically, in some optional implementations, to ensure accurate and rapid judgment of the first load information, thereby improving the efficiency of performance level matching, embodiments of the present invention also construct an experience database for load assessment through offline experiments. The core approach employs a "static modeling method based on pre-collected multi-dimensional stress test data" to establish a mapping relationship between hardware status and performance in advance, avoiding performance losses caused by real-time monitoring during runtime.

[0056] First, a multi-dimensional hardware experimental sample with the same hardware capacity as the vehicle is determined. The multi-dimensional hardware experimental sample refers to a hardware simulation environment that is completely identical to the target vehicle's core hardware (CPU, NPU, memory) in terms of model, computing power, and capacity. For example, if the target vehicle is equipped with a quad-core CPU (2.0GHz), an NPU with 8 TOPS computing power, and 6GB of RAM, the experimental sample must completely replicate this hardware configuration to ensure that the experimental data can truly reflect the hardware operating characteristics of the target vehicle. This avoids the experimental data from being out of touch with the actual scenario due to differences in hardware specifications, and provides a realistic hardware foundation for subsequent stress testing and performance level definition.

[0057] The stress test data is then loaded into the multi-dimensional hardware experimental samples, and corresponding performance levels are defined for different load scenarios achieved by the multi-dimensional hardware experimental samples. "Stress test data" refers to a set of test instructions that can simulate the hardware load under different vehicle usage scenarios. For example, adjusting the CPU computational workload simulates the load of simultaneous navigation and music operation; adjusting the NPU inference task complexity simulates the load of multi-turn voice interaction; and adjusting the memory data read / write volume simulates the load of multiple applications running in the background. Ultimately, the experimental samples achieve four typical scenarios: "low load," "low-medium load," "medium-high load," and "high load." The performance levels are pre-defined during the establishment of the experience database based on the differentiated impact of hardware resources on key voice-controlled vehicle processes. The specific number of levels is set according to user needs, and this embodiment of the invention does not impose specific limitations. In one instance, it can be specifically set to four levels (Q1-Q4), with the division focusing on IPC communication performance and NPU inference performance. Because IPC communication performance is constrained by both CPU computing power (serialization / deserialization operations consume CPU cycles) and memory bandwidth (data migration between kernel mode and user mode depends on memory throughput), NPU inference performance, although centered on the NPU, is also affected by CPU context switching and memory access latency caused by memory swapping. Therefore, performance level thresholds need to be set comprehensively based on three types of hardware parameters. For example, Q1 (low load) corresponds to CPU utilization <60%, NPU utilization <60%, and available memory capacity ≥1GB; Q2 (medium load) corresponds to... Low load corresponds to: 60%≤CPU utilization <80%, 60%≤NPU utilization <80%, 0.5GB≤available memory capacity <1GB; Q3 (medium-high load) corresponds to: 80%≤CPU utilization <95%, 80%≤NPU utilization <95%, 0.2GB≤available memory capacity <0.5GB; Q4 (high load) corresponds to CPU utilization ≥95%, NPU utilization ≥95%, and available memory capacity <0.2GB, ensuring that the level definition can accurately match the hardware's support capability for voice control of the vehicle.

[0058] Subsequently, inter-process communication (IPC) and large language model inference tasks were executed using a pressurized hardware experimental sample. The IPC task simulates the transmission of semantic data between the voice processing module, model inference module, and vehicle control module in voice-controlled vehicles (corresponding to IPC calls in a real-world scenario). The large language model inference task simulates the semantic parsing of actual voice commands and the generation of vehicle control commands. Two types of key timing data need to be recorded during execution: one is the IPC call time (i.e., the total time spent transmitting semantic data, reflecting IPC communication performance), and the other is the large language model inference time (i.e., the total time spent from receiving semantic data to generating control messages, reflecting NPU inference performance). For example, under the Q2 load scenario, the IPC task took 15ms, and the inference task took 400ms. This data needs to be stored in association with the corresponding load scenario and performance level.

[0059] Finally, the embodiments of the present invention create an experience database based on the performance level corresponding to each load scenario, the inter-process communication call time corresponding to each load scenario, and the inference time corresponding to each load scenario, as shown in Table 1 below.

[0060] Table 1. Schematic diagram of the experience database

[0061]

[0062] The essence of an experience database is a mapping library between hardware load status and key time-series data. The stored content is based on the core entries of "performance level - IPC call time - inference time". For example, Q1 level corresponds to IPC call time <10ms and inference time <300ms, while Q3 level corresponds to IPC call time 20-30ms and inference time 500-700ms. In addition, the database needs to cover the average data from multiple rounds of stress tests to ensure data stability and reliability.

[0063] The entire process is completed through offline benchmark testing, eliminating the need for complex real-time load assessment during actual vehicle operation. This avoids the resource consumption of real-time monitoring and provides data support for rapid load level matching and precise adjustment of inference capabilities in subsequent steps. It achieves the unification of "lightweight resource scheduling" and "high-precision load assessment". At the same time, the experimental design that replicates the real hardware environment ensures the practicality of the experience database, enabling subsequent voice control strategies based on this database to be fully adapted to the hardware characteristics of the target vehicle, avoiding decision-making errors caused by data deviations, and further improving the stability and real-time performance of voice control under limited resource conditions.

[0064] In some alternative implementations, step S101 includes:

[0065] Step b1: Upon receiving user voice, the inter-process communication call time for voice-to-speech conversion is calculated, and this call time is used as the first load information.

[0066] Step S102 includes:

[0067] Step b2: Determine the current performance level by comparing the first load information with the similarity of inter-process communication call times under various load scenarios in the experience database.

[0068] Specifically, in order to further simplify the load information acquisition process and improve the accuracy and efficiency of performance level matching, the specific execution logic for matching performance levels is optimized, and the key data of the core link of voice control is focused throughout the process to avoid redundant calculations and resource consumption.

[0069] First, upon receiving user voice input, the inter-process communication (IPC) call time for voice-to-semantic conversion is calculated and used as the primary load information. Voice-to-semantic conversion is the core pre-process of voice-controlled vehicles, referring to the process by which the vehicle-side voice processing module converts the user's voice signal (e.g., "turn on the driver's side air conditioning") into machine-recognizable textual semantic data. The IPC call time is the total time spent transmitting and interacting with semantic data between the voice processing module and subsequent inference modules during this process. This time directly reflects the overall load status of multi-dimensional hardware (CPU, NPU, memory). This is because the IPC for voice-to-semantic conversion relies on the CPU to complete data serialization / deserialization operations (consuming CPU cycles) and also requires memory for data migration between modules (consuming memory bandwidth). If the NPU is under high load, it indirectly leads to CPU resources being occupied to coordinate NPU tasks, thus extending the IPC call time. In actual execution, the system calculates the startup time at the instant it detects a user initiating a voice command (such as the user triggering the wake-up phrase "Hello, XX" and then stating their vehicle control request). The starting point is the moment the voice processing module outputs semantic data (denoted as ts1), and the ending point is the moment the inference module successfully receives that semantic data (denoted as ts2). Therefore, the IPC call time = ts2 - ts1. For example, if the user says "lower the driver's seat temperature by 2 degrees," T1 is 16:00:00.000, and T2 is 16:00:00.008. The calculated IPC call time is 8ms. This time is used as the first load information, eliminating the need to collect additional scattered parameters such as CPU utilization and memory capacity. This significantly simplifies the data acquisition process while ensuring a strong correlation between the data and the actual voice control scenario, avoiding interference from non-core parameters in the judgment.

[0070] Then, by comparing the similarity of inter-process communication call times under different load scenarios in the first load information with those in the experience database, the current performance level is determined. The experience database, which was built earlier through offline stress testing, stores the mapping relationship between "performance level - IPC call time - inference time" (e.g., Q1 corresponds to IPC call time <10ms, Q2 to [10ms, 20ms), Q3 to [20ms, 40ms), and Q4 to ≥40ms). The similarity judgment is based on numerical range matching logic. Since the experience database has been tested multiple times and the range of IPC call times corresponding to different performance levels has been clearly defined, there is no need to use complex similarity algorithms (such as cosine similarity or Euclidean distance). It is only necessary to compare the statistical IPC call time with the time range of each level in the database to quickly determine the current performance level. For example, if the IPC call time counted in step b1 is 8ms, and after comparing with the experience database, it is found that this time falls within the "<10ms" range of Q1 level, then the current performance level is directly determined to be Q1; if the IPC call time is 25ms, falling within the "20~30ms" range of Q3 level, then the current performance level is determined to be Q3. On the one hand, the range matching logic only requires simple numerical magnitude judgment, without consuming too much CPU resources, adapting to the limited computing power of the vehicle side; on the other hand, from counting the IPC call time to determining the performance level, the entire process can be controlled within 10ms, avoiding the impact of level matching delay on the real-time response of voice control. For example, after the user initiates a voice command, the system can complete the load judgment in a very short time, reserving sufficient time for subsequent adjustments to the large model inference capability and generation of control messages, ensuring that the overall vehicle control process is completed within the user's acceptable latency range.

[0071] In summary, the embodiments of the present invention simplify the acquisition process of the first load information, improve the efficiency and accuracy of performance level matching, and minimize system operating overhead. This provides a timely and reliable basis for subsequent dynamic adjustment of the large model's inference capabilities, effectively avoiding the problems of "forced inference under high load leading to lag" or "idle inference capabilities under low load," further enhancing the stability and user experience of edge-side voice-controlled vehicles in resource-constrained environments.

[0072] In some optional implementations, the performance levels include an unloading level and at least one compression level, and step S102 further includes:

[0073] Step c1: If the current performance level is the unload level, then unload the large language model;

[0074] Step c2: When the large language model is not loaded, if the current performance level is the compression level, determine whether the performance level matched for the previous preset consecutive number of times is the unloading level.

[0075] Step c3: If the performance level of the previous preset consecutive number of matching is not the unloading level, then load the large language model and limit the amount of input information and output control messages of the large language model according to the corresponding compression amount.

[0076] Specifically, to achieve a fine-grained adaptation between the inference capabilities of large language models and the hardware load of vehicles, and to avoid the waste of system resources and response delays caused by frequent model starts and stops, the performance levels are clearly divided into two categories: "Unload Level" and "Compression Level". The "Unload Level" refers to a load state where hardware resources are extremely scarce and cannot support large model inference (e.g., corresponding to the Q4 level mentioned above), while the "Compression Level" refers to a load state where hardware resources can partially support model inference, but the inference scale needs to be limited to ensure real-time performance (e.g., corresponding to the Q1, Q2, and Q3 levels mentioned above).

[0077] If the current performance level is determined to be unloaded (Q4) through matching, it means that the vehicle's multi-dimensional hardware (CPU, NPU, memory) is already at a high load threshold (e.g., CPU utilization > 95%, available memory < 200MB). If the large language model continues to run at this time, it will not only cause the inference time to far exceed the user's acceptable range (e.g., > 700ms), but may also occupy hardware resources, causing basic voice control functions (e.g., simple command response) to lag or even fail. Therefore, the system will immediately trigger the large language model unloading process, releasing the NPU computing resources occupied by the large language model, terminating all currently unfinished inference tasks; clearing the model parameters and inference cache data stored in memory to release memory space; and disconnecting the communication links between the large language model and the speech processing module, vehicle control module, and environmental perception module to avoid invalid data interaction. For example, when a vehicle is simultaneously running navigation, 4K video playback, and multiple applications updating in the background, resulting in a CPU utilization of 98% and 150MB of available memory, and matching the unload level, the system will quickly unload the large language model to ensure that subsequent basic voice commands (such as "open the window") can be responded to quickly through the preset command library, avoiding command delays caused by the model occupying resources.

[0078] If the current performance level is found to be a compression level through matching, the system then checks whether the performance level matched the previous preset consecutive number of times is an unloading level. The "preset consecutive number of times" is a threshold set based on vehicle hardware characteristics and user habits (usually 2-3 times, adjustable via system parameter configuration). Its core purpose is to avoid frequent loading and unloading of large models due to temporary fluctuations in hardware load (such as a rapid increase and subsequent decrease in CPU utilization within a short period). If the model is loaded based solely on a single compression level judgment, the hardware load may drop back to the unloading level immediately after the model is loaded, resulting in wasted resources. In actual execution, without loading a large language model, the system retrieves the most recent N (N = preset consecutive number of times) performance level judgment records and compares them with the unloading level. For example, if the preset consecutive number of times is set to 2, and the current performance level is Q2 (compression level), the system will query the previous two level records: if both previous records are Q3 (compression level), it determines that "the performance level matched the previous preset consecutive number of times is not an unloading level"; if one of the previous two records is Q4 (unloading level), it determines that "an unloading level record exists."

[0079] When the first N levels are all compression levels, it indicates that the hardware load is in a stable and supportable state. At this time, loading a large model can balance intelligence and real-time performance. If there is an unloading level record, the model will not be loaded for the time being, and the state of not loading a large language model will be maintained. The basic instruction library will continue to be used to respond, so as to avoid unloading the model again due to load fluctuations after loading.

[0080] Once the model is loaded, the system will adjust its inference capabilities based on the "compression level" corresponding to the current compression level. For example... Figure 4 As shown, the current level is Q1 (e.g., providing 100% prompt words with low compression), limiting the amount of input information to 640-660 tokens (it can receive long voice commands containing multiple needs, such as "turn on the driver's air conditioning to 24°C, and at the same time turn on the seat ventilation at level 1 and close the windows"), and the amount of output control messages is 8.

[0081] 1. For example, the prompt word information for a large input model at Q1 level is:

[0082] #Role:

[0083] You are the car's intelligent assistant;

[0084] #Task:

[0085] The control items are output based on the user query and information inside and outside the vehicle;

[0086] #query:

[0087] It's a bit hot;

[0088] #Speaker:

[0089] Second row, left;

[0090] #Passenger Information:

[0091] Driver's seat: Senior citizen; Second row left: Adult; Second row right: Child;

[0092] #Environment and Vehicle Information:

[0093] Road type: Expressway; Weather: Sunny; Interior temperature: 36 degrees Celsius; Outside temperature: 25 degrees Celsius; Outside air quality: Medium; Vehicle speed: Medium; Gear: D;

[0094] #Controller Status:

[0095] Air conditioning: On, cooling, 23 degrees Celsius, fan speed level 2; Seat heating or ventilation: Driver's seat ventilation level 1, passenger seat off, second row left side off, second row right side off; Window opening: Driver's seat 0%, passenger seat 0%, second row left side 0%, second row right side 0%;

[0096] #Output".

[0097] 2. For example, the information output by the Q1 level large model is:

[0098] "Turn the air conditioner to heating mode."

[0099] Set the air conditioner to 26 degrees Celsius.

[0100] Air conditioner fan speed setting 3;

[0101] The air conditioner's airflow is directed at your face and feet;

[0102] The passenger seat heater is set to level 1.

[0103] The passenger seat massage and acupressure mode is set to 20 minutes.

[0104] If the current level is Q3 (e.g., the prompt word is compressed to 20%, high compression), the input information is limited to 120-132 tokens (only core semantics are retained, such as "driver's seat air conditioning cooling"), and the output control message quantity is 2 messages.

[0105] 1. For example, the prompt word information for a large input model at Q3 level is:

[0106] #Role:

[0107] You are the car's intelligent assistant;

[0108] #Task:

[0109] Output a maximum of two control items based on the user query;

[0110] #query:

[0111] It's a bit hot;

[0112] #Output.

[0113] 2. For example, the information output by the Q3 level large model is:

[0114] The passenger-side window was opened to 20%.

[0115] The driver's side window is opened to 20%.

[0116] This invention addresses the issue of model resource consumption under high loads while avoiding frequent model starts and stops caused by temporary load fluctuations. Furthermore, it achieves fine-grained control of inference capabilities through differentiated compression. In practical applications, it enables large models to "run efficiently when available and quickly release resources when unavailable," significantly reducing system resource consumption and ensuring stable response speeds for voice-controlled vehicles under different load scenarios. This further enhances the reliability and user experience of edge-side voice-controlled vehicles in resource-constrained environments.

[0117] In some optional implementations, step S103 above includes:

[0118] Step d1: When the large language model is unloaded, the pre-saved instruction-message lookup table is invoked.

[0119] Step d2: If the target control message corresponding to the voice command is found through the command-message lookup table, then the target control message is output.

[0120] Step d3: When the large language model is not unloaded, generate inference prompts for voice commands according to the input information limit corresponding to the compression level, and input the inference prompts into the large language model for inference to obtain inference control messages.

[0121] Specifically, in some optional implementations, a control method is provided to adapt to the two states of "unloaded" and "not unloaded" of the large language model, and to ensure that voice commands can be efficiently converted into vehicle control commands under different hardware load scenarios, so as to achieve the goal of "quickly responding to basic needs when unloaded and accurately generating intelligent commands when not unloaded".

[0122] When the large language model is unloaded, a pre-saved command-message lookup table is invoked. For example, the unloading of the large language model corresponds to the "unloading level" (such as Q4 level) in the performance level section mentioned earlier. At this time, hardware resources are extremely scarce and cannot support model inference operations. A pre-built lightweight solution is needed to respond to voice commands. The "command-message lookup table" is a mapping database built in advance based on the vehicle's high-frequency voice control needs. It stores a one-to-one correspondence between "common user voice command texts and standard vehicle control messages". The "standard vehicle control messages" must conform to the interface format of the vehicle control system (such as CarService) and include three core types of information: the controlled object (such as air conditioning, windows, seats), the control action (such as opening, closing, and adjusting parameters), and the control parameters (such as temperature 25℃, fan speed level 2, and window opening degree 20%). For example, the message for "turn on the driver's side air conditioning" is "Controlled object: driver's side air conditioning; Control action: open; Control parameters: temperature 25℃, fan speed level 2", and the message for "close the sunroof" is "Controlled object: sunroof; Control action: close; Control parameters: opening degree 0%". This lookup table is pre-stored in the vehicle's local storage module. It requires no computational resources to access and can be read in milliseconds, meeting the "fast response priority" requirement under the unloading level. In actual execution, the system immediately triggers the lookup table retrieval process after confirming that the large model has been unloaded, without any additional waiting or computation. For example, if a user says "turn on the driver's side air conditioning" while the vehicle is simultaneously running navigation, 4K video caching, and multiple applications updating in the background (matching Q4 level), the system directly retrieves the command-message lookup table, avoiding response delays caused by waiting for model loading or inference.

[0123] The system achieves rapid association between voice commands and control messages through "text matching." First, the user's voice command is converted into standard text. Then, the text is used as a query keyword to perform precise matching in the command-message lookup table. If a unique target control message is found, the message is directly sent to the vehicle control system, and the actuator performs the corresponding operation. If no matching result is found, a preset "basic response message" is output, such as "This operation cannot be performed at the moment. You can try a simple command." At the same time, the user is informed through voice feedback to avoid user confusion caused by no response.

[0124] When the large language model is not unloaded, corresponding to the "compression level" (e.g., Q1, Q2, Q3), the hardware resources can support model inference, but the inference scale needs to be controlled by the "input information limit" (in units of tokens) to avoid resource overload. The "inference prompt words" are texts generated based on the semantics of the speech commands and the compression level limit, conforming to the input format of the large model. During generation, the original text after speech conversion needs to undergo "core semantic extraction and redundancy removal". For example, at level Q1 (with an input of 640-660 tokens), if a user says, "It's a bit hot today, so I'll lower the driver's side air conditioning temperature by 3 degrees, reduce the second-row air conditioning fan speed by 1 level, and close the driver's side window," the original text is quite long. When generating prompts, the complete semantics are preserved, resulting in "User requirement: Lower the driver's side air conditioning temperature by 3 degrees, reduce the second-row air conditioning fan speed by 1 level, and close the driver's side window; output no more than 8 control messages." At level Q3 (with an input of 120-132 tokens), if a user says, "The driver's side is a bit cold, so I want to raise the air conditioning temperature by 2 degrees and turn on the seat heater by 1 level," redundant expressions are removed when generating prompts, and the core requirement is retained, resulting in "User requirement: Raise the driver's side air conditioning temperature by 2 degrees and turn on the seat heater by 1 level; output no more than 2 control messages."

[0125] After the prompt words are generated, the system inputs them into a large language model. The model combines its own semantic understanding capabilities with the vehicle control logic to infer and generate a "reasoning control message". This message not only contains basic vehicle control parameters, but can also adapt to scenario-based needs (such as in Q1 level, the model infers "the current outside temperature is 28℃, and it is more comfortable to adjust the driver's air conditioning to 24℃", and the generated message parameters are more in line with the actual environment). After the inference is completed, the system performs format verification on the message (ensuring that the controlled object and parameters conform to the vehicle hardware specifications, such as the temperature not lower than 16℃ and not higher than 32℃). After the verification is passed, it is output to the vehicle control system. For example, at level Q2 (where the input amount is 380-395 tokens), a user says, "The navigation shows congestion ahead. Turn on the driver's seat massage and set the air conditioning to 25°C." The system generates a corresponding input model with the prompt words. After inference, the model outputs two messages: "Control object: driver's seat massage; control action: turn on; control parameters: intensity level 2, mode: kneading; control object: driver's air conditioning; control action: adjust; control parameters: temperature 25°C, fan speed level 1." The vehicle executes the operation precisely, meeting the user's needs in multiple scenarios without exceeding the hardware's load capacity.

[0126] This invention addresses the issue of command response failure caused by model malfunction under high load, while ensuring the intelligence and scenario adaptability of voice-controlled vehicles under low load. Through prompt word optimization and message verification, the accuracy and security of control messages are ensured, effectively avoiding the risks of miscontrol and erroneous control, and further improving the reliability, real-time performance, and user experience of edge-side voice-controlled vehicles in resource-constrained environments.

[0127] In some optional implementations, the adaptive voice-controlled vehicle method provided in this embodiment of the invention further includes:

[0128] Step e1: At the end of inference, the current inference time is calculated and used as the second load information;

[0129] Step e2: Update the current performance level by comparing the similarity of inference time in each load scenario with the second load information and the experience database.

[0130] Step e3: Adjust the number of control messages according to the updated current performance level.

[0131] Specifically, such as Figure 5 As shown, in some optional implementations, in order to solve the problem of "latency in initial level judgment" caused by dynamic fluctuations in hardware load during large model inference, a secondary load assessment and level update are introduced after inference to achieve dynamic alignment between performance level and real-time hardware status, ensuring that control message output always adapts to the current hardware carrying capacity.

[0132] In this embodiment of the invention, at the end of inference, the current inference time is calculated and used as the second load information. The end of inference refers to the moment when the large language model completes the semantic parsing of the speech command and generates the initial control message. Unlike the "first load information" determined based on the IPC call time before inference begins, the "second load information" focuses on the actual operating state of the hardware during inference. Since inference tasks (such as semantic understanding and instruction generation) continuously occupy NPU computing power and memory bandwidth, the hardware load may change significantly during inference (for example, the CPU utilization rate is 75% before inference begins, and rises to 85% during inference due to the start of other background tasks). The inference time directly reflects the actual processing efficiency of the hardware when carrying the inference task, and can more accurately map the comprehensive load of the current multi-dimensional hardware. In actual statistics, the system takes the time when the large model receives the inference prompt word as the starting point (denoted as ts3) and the time when the model outputs the initial control message as the ending point (denoted as ts4). Then the current inference time = ts4 - ts3. For example, if the model receives the prompt word at 16:00:01.000 and outputs the initial message at 16:00:01.450, the statistical inference time is 450ms. This time is used as the second load information. There is no need to collect additional hardware parameters. It can be quickly obtained through time series statistics, which balances data accuracy and lightweight collection.

[0133] The experience database, constructed offline as described above, stores the mapping relationship between "performance level - IPC call time - inference time". For example, level Q1 corresponds to inference time <300ms, Q2 corresponds to [300ms, 500ms), Q3 corresponds to [500ms, 700ms), and Q4 corresponds to ≥700ms. Similarity judgment uses "numerical range matching" logic, requiring no complex algorithm calculations. This embodiment compares the statistically calculated inference time (second load information) with the inference time range of each performance level in the database. If the inference time falls within the range of a certain level, the current performance level is updated to that level, achieving dynamic level correction. For example, before inference begins, the initial performance level is determined to be Q1 based on an IPC call time of 8ms. However, during inference, due to a sudden increase in NPU load, the inference time is recorded as 550ms. After comparing with the experience database, it is found that this time falls within the "500~700ms" range of the Q3 performance level. Therefore, the current performance level is updated from Q1 to Q3. If the inference time is 280ms, which still falls within the Q1 range, the performance level remains unchanged. By correcting the initial judgment deviation, the problem of the initial performance level being adapted but the actual load exceeding the limit due to load fluctuations during inference is avoided, ensuring that the performance level is always consistent with the real-time hardware status.

[0134] Finally, the number of control messages is adjusted according to the updated performance level. Since the updated performance level reflects the actual hardware load at the end of inference, the number of control messages needs to be recalibrated based on this level to avoid exceeding the current hardware processing capacity (for example, initial level Q1 generates 8 messages, but updated level Q3 can only handle 2 messages; failure to adjust this will cause subsequent message transmission and execution lag). In actual adjustment, the system first obtains the upper limit of the output control message quantity corresponding to the updated level, then filters the initial control messages generated by the large model, prioritizing messages strongly related to the user's core needs (e.g., if the user says "turn on the air conditioner and adjust the seat," the "air conditioner control" message is prioritized), and discarding secondary or redundant messages. If the initial number of messages does not exceed the upper limit, all messages are output directly; if it exceeds the upper limit, messages are truncated according to the principle of prioritizing core needs. For example, when the initial level is Q1, the model generates 4 messages: "Driver's air conditioning set to 24℃; second-row air conditioning set to 25℃; driver's seat ventilation turned on; second-row seat heating turned off". After inference, the level is updated to Q3 (maximum of 2 messages). The system selects and outputs 2 core messages: "Driver's air conditioning set to 24℃; driver's seat ventilation turned on". If the level is still Q1 after the update (maximum of 8 messages), all 4 messages are output.

[0135] This invention, through the dynamic updating of second load information and levels, achieves real-time tracking of hardware dynamic load by performance levels. In practical applications, it effectively avoids vehicle control delays or functional failures caused by load fluctuations during inference, ensuring that the number of control messages always matches the hardware's carrying capacity. In scenarios with stable loads, it guarantees the integrity and intelligence of message output, further enhancing the stability, flexibility, and user experience of end-side voice vehicle control in resource-constrained environments.

[0136] This embodiment also provides an adaptive voice-controlled vehicle device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0137] This embodiment provides an adaptive voice-controlled vehicle device, such as... Figure 6 As shown, it includes:

[0138] The load acquisition module 601 is used to acquire first load information, which reflects the load level of multi-dimensional hardware in the vehicle.

[0139] The performance level evaluation module 602 is used to match the corresponding current performance level according to the first load information, and to compress the reasoning ability of the large language model according to the current performance level. The reasoning ability is used to represent the amount of input information and output control messages of the large language model.

[0140] The decision module 603 is used to reason about the user's voice commands based on the processed large language model to obtain control messages for controlling the vehicle.

[0141] In some alternative implementations, prior to the load acquisition module 601, the following is also included:

[0142] The sample module is used to determine multidimensional hardware experimental samples with the same hardware capacity as those in the vehicle.

[0143] The simulated load module is used to load stress test data onto the multi-dimensional hardware experimental sample and define corresponding performance levels for different load scenarios achieved by the multi-dimensional hardware experimental sample. The stress test data is used to apply load to the hardware experimental sample to achieve various load scenarios.

[0144] The simulated inference module is used to perform inter-process communication tasks and large language model inference tasks using pressurized hardware experimental samples.

[0145] The performance metric acquisition module is used to create an experience database based on the performance level corresponding to each load scenario, the inter-process communication call time corresponding to each load scenario, and the inference time corresponding to each load scenario.

[0146] In some alternative implementations, the load acquisition module 601 includes:

[0147] The load acquisition unit is used to count the inter-process communication call time for speech-to-speech conversion when it receives user voice, and to use the speech-to-speech inter-process communication call time as the first load information.

[0148] In some alternative implementations, the performance level evaluation module 602 includes:

[0149] The performance level determination unit is used to determine the current performance level by comparing the first load information with the similarity of inter-process communication call times under various load scenarios in the experience database.

[0150] In some alternative implementations, the performance levels include an unloading level and at least one compression level. The performance level evaluation module 602 further includes:

[0151] The unloading unit is used to unload the large language model if the current performance level is the unloading level.

[0152] The verification unit is used to determine whether the performance level matched for the previous preset number of consecutive times is the unloading level when the current performance level is the compression level, without loading the large language model.

[0153] The capacity compression unit is used to load the large language model and limit the amount of input information and output control messages of the large language model according to the corresponding compression amount if the performance level matched for a previous preset number of consecutive times is not the unloading level.

[0154] In some alternative implementations, the decision module 603 includes:

[0155] The standard table call unit is used to call a pre-saved instruction-message lookup table when the large language model is unloaded.

[0156] The message query unit is used to output the target control message if the target control message corresponding to the voice command is found through the command-message lookup table;

[0157] The message inference unit is used to generate inference prompts for voice commands according to the input information limit corresponding to the compression level when the large language model is not unloaded, and input the inference prompts into the large language model for inference to obtain inference control messages.

[0158] In some alternative implementations, it also includes:

[0159] The inference time statistics module is used to calculate the current inference time at the end of inference and use the current inference time as the second load information.

[0160] The performance level update module is used to update the current performance level based on the similarity of inference time consumption under various load scenarios in the second load information and the experience database.

[0161] The message output adjustment module is used to adjust the number of control messages according to the updated current performance level.

[0162] The apparatus provided in this embodiment of the invention can execute the method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0163] Figure 7 This is a structural schematic diagram of a vehicle provided in an embodiment of the present invention.

[0164] The following is a detailed reference. Figure 7 The diagram illustrates a structural schematic suitable for implementing a vehicle according to an embodiment of the present invention. The vehicle may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for vehicle operation. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0165] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows the vehicle to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 Vehicles with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0166] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the methods of the embodiments of the present invention.

[0167] Figure 7 The vehicle shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0168] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0169] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0170] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An adaptive voice-controlled vehicle method, characterized in that, The method includes: Obtain first load information, which is used to reflect the load level of multi-dimensional hardware in the vehicle; obtaining the first load information includes: obtaining load data of multi-dimensional hardware; when receiving user voice, calculating the inter-process communication call time for speech-to-speech, and using the speech-to-speech inter-process communication call time and the load data as the first load information; The process involves matching the current performance level based on the first load information and compressing the inference capability of a large language model according to the current performance level. The inference capability represents the amount of input information and output control messages that the large language model can generate. This process includes: determining the current performance level based on the similarity between the first load information and the inter-process communication call time in various load scenarios in the experience database; if the current performance level is an unloading level, unloading the large language model; if the current performance level is a compression level when the large language model is not loaded, determining whether the performance level matched for the previous preset number of consecutive counts is the unloading level; if the performance level matched for the previous preset number of consecutive counts is not the unloading level, loading the large language model and limiting its input information and output control messages according to the corresponding compression amount. The user's voice commands are processed based on the compressed large language model to obtain control messages for vehicle control.

2. The method according to claim 1, characterized in that, Before obtaining the first load information, the method further includes: Identify multidimensional hardware experimental samples with the same hardware capacity as those in the vehicle; The stress test data is loaded into the multi-dimensional hardware experimental sample, and corresponding performance levels are defined for different load scenarios achieved by the multi-dimensional hardware experimental sample. The stress test data is used to apply load to the hardware experimental sample to achieve multiple load scenarios. The hardware experimental sample under pressure was used to perform inter-process communication tasks and large language model inference tasks. An experience database was created based on the performance level corresponding to each load scenario, the inter-process communication call time corresponding to each load scenario, and the inference time corresponding to each load scenario.

3. The method according to claim 1, characterized in that, The process of the user's voice commands based on the compressed large language model yields control messages for vehicle control, including: When the large language model is unloaded, the pre-saved instruction-message lookup table is invoked; If the target control message corresponding to the voice command is found through the command-message lookup table, then the target control message is output. When the large language model is not unloaded, the inference prompt words for the voice command are generated according to the input information limit corresponding to the compression level, and the inference prompt words are input into the large language model for inference to obtain the inference control message.

4. The method according to claim 1, characterized in that, The method further includes: At the end of the inference, the current inference time is calculated and used as the second load information; The current performance level is updated based on the similarity between the second load information and the inference time under each load scenario in the experience database. Adjust the number of control messages according to the updated current performance level.

5. The method according to claim 1, characterized in that, The multidimensional hardware includes a central processing unit, a neural network processor, and memory.

6. An adaptive voice-controlled vehicle device, characterized in that, The device includes: A load acquisition module is used to acquire first load information, which reflects the load level of multi-dimensional hardware in the vehicle. Acquiring the first load information includes: acquiring load data of multi-dimensional hardware; when receiving user voice, calculating the inter-process communication call time for speech-to-speech conversion, and using the speech-to-speech inter-process communication call time and the load data as the first load information. The performance level evaluation module is used to match the corresponding current performance level according to the first load information, and compress the inference capability of the large language model according to the current performance level. The inference capability is used to represent the amount of input information and output control messages of the large language model. The step of matching the corresponding current performance level according to the first load information and compressing the inference capability of the large language model according to the current performance level includes: determining the current performance level by means of the similarity between the first load information and the inter-process communication call time in each load scenario in the experience database; if the current performance level is an unloading level, then unloading the large language model; when the large language model is not loaded, if the current performance level is a compression level, then determining whether the performance level matched for the previous preset consecutive number of times is the unloading level; if the performance level matched for the previous preset consecutive number of times is not the unloading level, then loading the large language model and limiting the amount of input information and output control messages of the large language model according to the corresponding compression amount. The decision module is used to reason about the user's voice commands based on the processed large language model, and obtain control messages for controlling the vehicle.

7. A vehicle, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 5.

9. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the method of any one of claims 1 to 5.

Images