Control method and apparatus, and vehicle

Abstract

A control method and apparatus, and a vehicle. The control method (300) comprises: acquiring an operation intent of a first user and first state information (S301), wherein the operation intent is obtained by recognizing a first input of the first user, and the first state information includes state information of a vehicle (100); on the basis of the operation intent, acquiring first application programming interface information from a first database (S302), the first application programming interface information comprising description information responsive to the operation intent; and inputting input data into a first machine learning model to obtain a first control instruction (S303), the input data being obtained at least on the basis of the following items: the operation intent, the first state information, and the first application programming interface information, and the first control instruction being used for scheduling a component or function in the vehicle (100). The control method (300) improves the response capability of the vehicle (100) to fuzzy intents or non-standard instructions, thereby significantly improving the convenience and intelligence level of interaction.

Description

Control methods, devices and vehicles

[0001] This application claims priority to Chinese patent application filed on January 24, 2025, with application number 202510123154.8 and title "Control Method, Apparatus and Vehicle", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of intelligent vehicles, and more specifically, to a control method, device, and vehicle. Background Technology

[0003] As smart cars become more widely used in daily life, users expect them and their related devices to bring a more comfortable and intelligent experience. Against this backdrop, intelligent voice systems, as an important human-computer interaction method, are being widely used to improve the driving experience and the level of intelligence in the in-vehicle environment.

[0004] However, as in-vehicle infotainment systems become increasingly complex, users lack understanding of many of these functions. Furthermore, in-vehicle infotainment systems typically only process commands in fixed formats or specific patterns, resulting in poor responsiveness when users express ambiguous intentions or non-standard commands. This fails to meet users' expectations for a natural and smooth interactive experience, thus impacting their human-computer interaction experience. Summary of the Invention

[0005] This application provides a control method, device, and vehicle that can improve the vehicle's responsiveness to ambiguous intentions or non-standard commands, significantly enhancing the convenience and intelligence of interaction.

[0006] In a first aspect, a control method is provided, the method comprising: acquiring an operation intention and first state information of a first user, wherein the operation intention is obtained by recognizing a first input of the first user, and the first state information includes state information of a vehicle; acquiring first application programming interface (API) information from a first database according to the operation intention, wherein the first API information includes descriptive information in response to the operation intention; inputting input data into a first machine learning model to obtain a first control instruction, wherein the input data is obtained based on at least the following: the operation intention, the first state information, and the first API information, and the first control instruction is used to schedule components or functions in the vehicle.

[0007] In one possible implementation, the first machine learning model can also be called a multimodal model, a multimodal machine learning model, or an artificial intelligence content generation model.

[0008] In one possible implementation, the first control instruction can be an instruction directly used to schedule components or functions in the vehicle, or it can be a scheduling strategy. When the first control instruction is a scheduling strategy, the components or functions in the vehicle can convert the scheduling strategy into a scheduling instruction and then execute it.

[0009] In one possible implementation, the aforementioned descriptive information responding to operational intent can be understood as: a functional explanation or introduction of the application programming interface that needs to be invoked when a component or function in the vehicle responds to an operational intent.

[0010] In one possible implementation, the first database can be a repository stored locally in the vehicle, also known as a vehicle infotainment application interface knowledge base, which can store description information of various application interfaces.

[0011] In this embodiment, a first machine learning model can be applied to process the user's operational intent and then generate a first control command. In this way, processing the user's operational intent does not require complex manual input or preset commands, improving the vehicle's responsiveness to ambiguous intents or non-standard commands, and significantly enhancing the convenience and intelligence of the interaction. Furthermore, when generating the first control command, the first machine learning model combines vehicle status information and first application programming interface information, enabling the generated first control command to accurately respond to the user's operational intent, thereby meeting diverse driving needs and improving the user's driving experience.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the input data further includes: first strategy information, which is obtained from a second database based on the operational intent, the second database being associated with an internet community; and before inputting the input data into the first machine learning model to obtain the first control instruction, the method further includes: receiving the first strategy information sent by the server.

[0013] In one possible implementation, the second database can be a database stored on a server, or an internet community knowledge base. The second database can store control commands generated when vehicles of the same or similar models respond to operational intentions.

[0014] In this embodiment, when generating the first control command, the first machine learning model incorporates first strategy information stored on the server, generated when vehicles of the same or similar vehicle models respond to operational intentions. By referencing execution data from similar vehicles, the generated first control command can be better adapted to the current vehicle characteristics and the user's operational intentions, thereby effectively improving the accuracy and generation efficiency of the first control command.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the first policy information is uploaded by the second user, or the first policy information is collected and uploaded by the vehicle after authorization by the first user.

[0016] In this embodiment, when the data stored in the second database is data uploaded by other users, the first machine learning model generates a first control command with higher accuracy and generation efficiency by referencing the data uploaded by other users. When the data stored in the second database is data collected and uploaded by the vehicle after authorization by the first user, the first machine learning model can generate a first control command that conforms to the first user's driving habits by referring to the first user's historical driving data, thereby meeting the first user's personalized driving needs. On the other hand, without the first user's authorization, the vehicle will not collect and upload the first user's historical driving data. This design can fully respect the user's privacy rights and effectively avoid potential information leakage risks.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the first state information includes: the state information of the ambient light, the first control instruction for controlling the color and brightness of the ambient light, and / or the first control instruction for controlling the content played on the head-up display in the vehicle.

[0018] In one possible implementation, the first control command is also used to indicate the duration of the ambient light flashing, or the time interval between ambient light color changes.

[0019] In one possible implementation, the status information of the ambient light may include at least one of the following: the position, brightness, color, or flashing time of the ambient light illuminated in the cabin.

[0020] In this embodiment, the vehicle can combine the user's intention to schedule ambient lighting, the status information of the ambient lighting in the cabin, and the description information of the application programming interface corresponding to scheduling the ambient lighting to control the color and brightness of the ambient lighting, and / or control the content played on the head-up display in the vehicle. In this way, the first control command can be generated accurately and efficiently, thereby flexibly meeting the user's adjustment needs for the in-vehicle ambient lighting, ensuring that the user can easily create an ideal in-vehicle atmosphere in different scenarios, further enhancing driving comfort and personalized experience.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the first state information further includes: seat occupancy information in the cabin and gear position information of the vehicle, and the first control command is used to control the color and brightness of the ambient light in the first area, the first area including the area corresponding to the occupied seat.

[0022] In this embodiment of the application, when the first state information includes seat occupancy information in the cabin and vehicle gear information, the first control command for the ambient light generated by the first machine learning model can be adapted to the seat occupancy status, ensuring that the control of the color and brightness of the ambient light in the area corresponding to the seat occupancy meets the user's operating intention, thereby further improving the user's driving experience.

[0023] In conjunction with the first aspect, in some implementations of the first aspect, before inputting the input data into the first machine learning model to obtain the first control command, the method further includes: filtering noise and performing time synchronization processing on the first state information.

[0024] In this embodiment, before inputting the input data into the first machine learning model, the first state information undergoes noise filtering and time synchronization processing. This effectively removes environmental noise or error signals that may exist during the sensor's acquisition of the first state information, thereby improving the accuracy of the first state information data. Simultaneously, time synchronization processing corrects time deviations between multiple data sources in the first state information, ensuring the consistency of the first state information in the time dimension, thus providing a reliable foundation for the analysis and calculation of the first machine learning model. This preprocessing method not only significantly improves the prediction accuracy of the first machine learning model but also enhances its adaptability to complex scenarios.

[0025] In conjunction with the first aspect, in some implementations of the first aspect, the input data further includes: second state information, which is used to indicate environmental information outside the cockpit; before inputting the input data into the first machine learning model to obtain the first control command, the method further includes: acquiring the second state information.

[0026] In this embodiment, by incorporating environmental information from outside the cockpit into the generation process of control commands, the vehicle can more accurately understand and respond to the user's operational intentions. This approach not only effectively improves the intelligence and personalization of the first control commands but also better adapts to the diverse needs of users in various driving scenarios, thereby significantly improving the overall driving experience and making the driving process safer, more efficient, and more comfortable.

[0027] In conjunction with the first aspect, in some implementations of the first aspect, the first state information includes humidity information inside the cabin, the second state information includes precipitation intensity information outside the cabin, and the first control command includes at least one of the following: controlling the cleaning frequency of the windshield wipers, controlling the radiation distance and brightness of the headlights, controlling whether the dehumidification mode of the air conditioner is turned on, or controlling whether the traction control system and the electronic stability system are activated.

[0028] In this embodiment, the vehicle can combine the user's intention to activate the rain driving mode, the humidity information inside the cabin, the precipitation intensity information outside the cabin, and the application interface information for scheduling the wipers, headlights, air conditioning, or traction control system and electronic stability system to generate a first control command that conforms to rain driving. In this way, the first control command can be generated accurately and efficiently, thereby ensuring the user's safety and driving experience during rain driving.

[0029] In conjunction with the first aspect, in some implementations of the first aspect, the first state information further includes vehicle speed information, the second state information further includes first image information and first point cloud information around the vehicle, the first point cloud information is used to indicate the distance between the vehicle and the obstacle, and the first control command includes: controlling whether the traction control system and the electronic stability system are activated.

[0030] In this embodiment, when the vehicle detects a user's intention to activate the rain driving mode while driving in rainy weather, the first machine learning model can combine vehicle speed information, image information of the vehicle's surroundings, and point cloud information to generate a first control command to control whether the traction control system and electronic stability system are activated. This ensures safe driving of the vehicle in rainy weather, thereby guaranteeing the user's driving safety.

[0031] Secondly, a control device is provided, the device comprising: an acquisition unit and a processing unit; the acquisition unit is configured to acquire an operation intention of a first user and first state information, the operation intention being obtained by recognizing a first input from the first user, and the first state information including vehicle state information; the processing unit is configured to: acquire first application programming interface (API) information from a first database according to the operation intention, the first API information including descriptive information in response to the operation intention; input input data into a first machine learning model to obtain a first control instruction, the input data being obtained based on at least the following: the operation intention, the first state information, and the first API information, the first control instruction being used to schedule components or functions in the vehicle.

[0032] In conjunction with the second aspect, in some implementations of the second aspect, the input data further includes: first policy information, which is obtained from a second database based on the operational intent, the second database being associated with an internet community, and the device further includes a transceiver unit; the transceiver unit is used to receive the first policy information sent by the server.

[0033] In conjunction with the second aspect, in some implementations of the second aspect, the first policy information is uploaded by the second user, or the first policy information is collected and uploaded by the vehicle after authorization by the first user.

[0034] In conjunction with the second aspect, in some implementations of the second aspect, the first state information includes: the state information of the ambient light, the first control instruction for controlling the color and brightness of the ambient light, and / or the first control instruction for controlling the content played on the head-up display in the vehicle.

[0035] In conjunction with the second aspect, in some implementations of the second aspect, the first state information further includes: seat occupancy information in the cabin and gear position information of the vehicle, and the first control command is used to control the color and brightness of the ambient light in the first area, the first area including the area corresponding to the occupied seat.

[0036] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to perform noise filtering and time synchronization processing on the first state information.

[0037] In conjunction with the second aspect, in some implementations of the second aspect, the input data further includes: second state information, which is used to indicate environmental information outside the cockpit; the acquisition unit is also used to acquire the second state information.

[0038] In conjunction with the second aspect, in some implementations of the second aspect, the first state information includes humidity information inside the cabin, the second state information includes precipitation intensity information outside the cabin, and the first control command includes at least one of the following: controlling the cleaning frequency of the windshield wipers, controlling the radiation distance and brightness of the headlights, controlling whether the dehumidification mode of the air conditioner is turned on, or controlling whether the traction control system and the electronic stability system are activated.

[0039] In conjunction with the second aspect, in some implementations of the second aspect, the first state information further includes vehicle speed information, the second state information further includes first image information and first point cloud information around the vehicle, the first point cloud information is used to indicate the distance between the vehicle and the obstacle, and the first control command includes: controlling whether the traction control system and the electronic stability system are activated.

[0040] Thirdly, a control device is provided, comprising: at least one processor and a memory, wherein the at least one processor is coupled to the memory for reading and executing instructions in the memory, such that the device implements the method in any of the implementations of the first aspect described above.

[0041] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing program code, which, when run on a computer, causes the computer to perform the method in any of the implementations of the first aspect described above.

[0042] Fifthly, a chip is provided, the chip including circuitry for performing the method in any of the implementations of the first aspect described above.

[0043] Sixthly, a computer program product is provided, the computer product including a computer program that, when the computer program is run by a processor, causes the method in any of the implementations of the first aspect to be executed.

[0044] In a seventh aspect, a vehicle is provided, comprising: the control device described in any of the second or third aspects above. Attached Figure Description

[0045] Figure 1 is a functional schematic diagram of a vehicle provided in an embodiment of this application;

[0046] Figure 2 is a system architecture diagram applicable to a control method provided in an embodiment of this application;

[0047] Figure 3 is a schematic flowchart of a control method provided in an embodiment of this application;

[0048] Figure 4 is another system architecture diagram to which the control method provided in the embodiments of this application is applicable;

[0049] Figure 5 is a schematic diagram of the data preprocessing module provided in the embodiment of this application performing data preprocessing;

[0050] Figure 6 is a schematic diagram of the data processing flow of the feature fusion module provided in the embodiment of this application;

[0051] Figure 7 is a schematic diagram of the data processing flow for multimodal information extraction provided in an embodiment of this application;

[0052] Figures 8 to 10 are schematic diagrams of the scenarios in which the control method provided in the embodiments of this application is applicable;

[0053] Figure 11 is another system architecture diagram to which the control method provided in the embodiments of this application is applicable;

[0054] Figure 12 is a schematic diagram of the data processing flow of the sensing system provided in an embodiment of this application;

[0055] Figure 13 is a schematic diagram of a control device provided in an embodiment of this application;

[0056] Figure 14 is a schematic diagram of another control device provided in an embodiment of this application. Detailed Implementation

[0057] In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In this application, "at least one" means one or more, and "more" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0058] The use of prefixes such as "first" and "second" in this application embodiment is solely for distinguishing different descriptive objects and does not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects. The description of the described objects is found in the claims or the context of the embodiments, and the use of such prefixes should not constitute unnecessary restrictions.

[0059] To better understand this application, the terminology used in this application will be introduced first.

[0060] (1) Multimodal model: In artificial intelligence, it can refer to a model that uses multiple different perceptual modalities (e.g., text, speech, image, etc.) for joint modeling and prediction.

[0061] (2) Generative artificial intelligence (Generative AI): An artificial intelligence system that can generate text, images or other media in response to prompts.

[0062] (3) A convolutional neural network (CNN) is a feedforward neural network in which artificial neurons can respond to a portion of the surrounding units within their coverage area. A CNN can consist of one or more convolutional layers and a fully connected layer at the top (corresponding to a classic neural network), as well as associated weights and pooling layers. This structure allows the CNN to utilize the two-dimensional structure of the input data. Compared to other deep learning structures, CNNs can deliver better results in image and speech recognition.

[0063] (4) An application programming interface (API) is a computing interface that defines the interaction between multiple software intermediaries, the types of calls or requests that can be made, how to make calls or requests, the data formats that should be used, and the conventions that should be followed.

[0064] (5) Vehicle hydroplaning mode: When the vehicle is in hydroplaning mode, the vehicle can activate the traction control system to prevent tire slippage, activate the electronic stability control system to improve vehicle stability and prevent skidding or loss of control, and activate the brake assist system to provide additional braking force during emergency braking, thereby reducing braking distance and improving driving safety.

[0065] As described in the background section, with the increasing complexity of vehicle infotainment functions, users lack understanding of many functions. In addition, vehicle infotainment systems can usually only process instructions in fixed formats or specific patterns, resulting in poor responsiveness when users express ambiguous intentions or non-standard instructions. This fails to meet users' expectations for a natural and smooth interactive experience and affects the user's human-computer interaction experience.

[0066] While the aforementioned problems can be addressed by deploying speech recognition models—for example, a speech recognition model can be used to recognize user voice requests, obtain speech recognition results, and then perform fuzzy matching between the speech recognition results and a preset text set. If a target text is matched, the vehicle can be controlled according to the target text to achieve the target scenario. Another example is to score the user's speech content, distinguishing between simple task commands and complex response commands. For simple task commands, a semantic understanding module can parse and execute them; for complex task commands, they can be converted into one or more simple task commands and then executed. However, the accuracy of speech recognition models in generating commands that match user intent is poor, making it unable to handle complex interaction scenarios and thus affecting the user's human-computer interaction experience.

[0067] This application provides a control method, device, and vehicle that can improve the vehicle's responsiveness to ambiguous intentions or non-standard commands, significantly enhancing the convenience and intelligence of interaction.

[0068] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0069] As shown in Figure 1, the vehicle 100 involved in this application may include multiple subsystems, such as a perception system 120, a computing platform 130, and a human-machine interaction module 140. Optionally, the vehicle 100 may include more or fewer subsystems, and each subsystem may include one or more components. In addition, each subsystem and component of the vehicle 100 can be interconnected via wired or wireless means.

[0070] The perception system 120 may include several sensors for sensing information about the environment surrounding the vehicle 100. For example, the perception system 120 may include a positioning system, which may be a global positioning system (GPS), a BeiDou system, or another positioning system. The perception system 120 may include one or more of the following: an inertial measurement unit (IMU), a rain sensor, a humidity and temperature sensor, a lidar, a millimeter-wave radar, an ultrasonic radar, and a camera device.

[0071] Some or all of the functions of vehicle 100 can be controlled by computing platform 130. Computing platform 130 may include processors 131 to 13n (n being a positive integer). A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 130 may also include a memory for storing instructions. Some or all of the processors 131 to 13n can call the instructions in the memory to implement the corresponding functions.

[0072] The computing platform 130 can control the functions of the vehicle 100 based on inputs received from various subsystems (e.g., the sensing system 120). In some embodiments, the computing platform 130 can be used to provide control over many aspects of the vehicle 100 and its subsystems.

[0073] The human-machine interface module 140 can intelligently generate responsive control commands based on the user's operational intentions, and effectively schedule and manage various components or functions in the vehicle through these control commands. For example, it can adjust the vehicle's driving route according to the user's navigation needs, or control the air conditioning, seat adjustment, and multimedia system according to the user's comfort needs.

[0074] Optionally, the above components are just an example. In actual applications, the components in each of the above modules may be added or deleted as needed.

[0075] The vehicle 100 in this application may include: road vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or entertainment equipment, etc. For example, vehicle 100 may be a means of transportation (such as commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (such as forklifts, trailers, tractors, etc.), engineering vehicles (such as excavators, bulldozers, cranes, etc.), agricultural equipment (such as lawnmowers, harvesters, etc.), amusement equipment, toy vehicles, etc. The embodiments of this application do not specifically limit the type of vehicle.

[0076] Figure 2 is a system architecture diagram applicable to a control method provided in an embodiment of this application.

[0077] As shown in Figure 2, the system architecture may include: system input, multimodal scene understanding module, artificial intelligence generated content (AIGC) model and in-cabin equipment. The multimodal scene module and AIGC model can be deployed in the human-computer interaction module 140 in Figure 1.

[0078] System inputs may include user input and sensor data. User input can be obtained by recognizing voice, touchscreen, or physical button input. Sensor data may include environmental information outside the vehicle cabin and / or vehicle status information. Optionally, vehicle status information may include humidity and temperature information collected by a humidity and temperature sensor, vehicle gear information, and vehicle speed information, etc. Environmental information outside the cabin may include image information collected by a camera, point cloud information collected by a lidar sensor, and precipitation intensity information collected by a rain sensor, etc.

[0079] The multimodal scene understanding module can process and fuse raw data from different sensors. This module can include: a data preprocessing module, a perception system, a feature fusion module, and a multimodal information extraction module. Specifically, the data processing module acquires raw vehicle status information and preprocesses it; the perception system preprocesses raw environmental information from outside the cabin, filtering noise to improve data quality; the feature fusion module fuses data from different sensors to form a comprehensive environmental perception; and the multimodal information extraction module extracts features from the fused data to generate scene information, including descriptive text for the current scene.

[0080] The AIGC model can generate control commands based on one or more of the following: scene information, user's operational intent, API information in the first database, and policy information in the second database, and send these commands to the execution devices within the cockpit. The AIGC model may include a rationality verification module, which verifies the generated control commands to ensure their executability, accuracy, and creativity.

[0081] Once the equipment inside the cabin receives a control command, it can execute the command, for example, by controlling various aspects such as vehicle control, navigation, multimedia, and communication.

[0082] It should be noted that the vehicle status information and external environmental information mentioned above are merely illustrative and do not constitute a limitation. Vehicle status information also includes, but is not limited to: fuel level, battery charge, braking system status, steering angle, etc., while external environmental information may also include: real-time road traffic conditions, the location and movement of surrounding vehicles, weather conditions, light intensity, and terrain features of the driving area, etc. The specific content and scope of the above information can be adjusted and expanded according to actual needs to better adapt to different scenario requirements and technical implementations.

[0083] It should also be noted that the system architecture shown in Figure 2 is merely an example, intended to provide an intuitive understanding of the implementation principle of the control method, and does not constitute any limitation on the applicable system architecture for the control method involved in this application. Those skilled in the art can flexibly adjust the system architecture shown in Figure 2 according to specific practical needs. For example, new modules can be added to support more complex functional requirements, some modules can be reduced to simplify system design, or existing modules can be replaced to use newer modules or hardware implementations. Furthermore, in different application scenarios, the system architecture may exhibit diversity due to differences in requirements, resources, or constraints.

[0084] Figure 3 is a schematic flowchart of a control method provided in an embodiment of this application. The execution subject of method 300 can be a vehicle. When the execution subject of method 300 is vehicle 100, it can be executed by computing platform 130 in vehicle 100, or by system-on-chip (SoC) in computing platform 130, or by processor in computing platform 130. The following describes method 300 with vehicle as the execution subject. Method 300 can include steps S301 to S303.

[0085] S301, Obtain the first user's operation intent and first state information.

[0086] The operation intent is obtained by recognizing the first input of the first user. The first state information includes the state information of the vehicle. The specific content of the vehicle's state information can be found in the relevant description in the text section of Figure 2.

[0087] Optionally, the first input can be the user's voice, touchscreen, or physical button input.

[0088] Optionally, the first state information can be collected by sensors. For example, if the first state information includes vehicle speed, the vehicle speed can be obtained by a vehicle speed sensor. As another example, if the first state information includes seat state information, the seat state information can be obtained by a camera deployed in the cabin. Yet another example, if the first state information includes ambient light state information, the ambient light state information can be obtained by an ambient light sensor.

[0089] S302, obtain the first application interface information from the first database according to the operation intention.

[0090] The first application programming interface information includes descriptive information in response to the operation intent.

[0091] Optionally, the descriptive information responding to the operational intent can be understood as: a functional explanation or introduction of the application programming interfaces (APIs) that need to be invoked when a component or function in the vehicle responds to the operational intent. For example, if the user's operational intent is "open the car window," the first application programming interface information may include a functional description of the specific API interface to be invoked to implement that function.

[0092] S303, input the input data into the first machine learning model to obtain the first control command.

[0093] The input data is obtained from at least the following: operation intention, first status information and first application interface information, and the first control command is used to schedule the components or functions in the vehicle.

[0094] Optionally, the first machine learning model is obtained based on training samples and training labels. The training samples include: sample operation intention, first sample state information, first sample application interface information, and first sample control instructions.

[0095] Alternatively, the first machine learning model may also be called a multimodal model, a multimodal machine learning model, or an AIGC model.

[0096] Optionally, the first control command can be a direct instruction for scheduling components or functions in the vehicle, or it can be a scheduling strategy. When the first control command is a scheduling strategy, the components or functions in the vehicle can convert the scheduling strategy into a scheduling command and execute it. For example, if the first control command instructs the air conditioning system to turn on, then the in-cabin equipment, upon receiving the first control command, can directly control the air conditioning system to turn on according to the first control command. As another example, if the first control command instructs the scheduling strategy to turn on the air conditioning, then the in-cabin equipment, upon receiving the first control command, can convert the first control command into a scheduling command to turn on the air conditioning and execute it.

[0097] Optionally, the first database can be a repository stored locally in the vehicle, or a vehicle-mounted application programming interface (API) knowledge base, which can store description information of various application programming interfaces.

[0098] Based on the aforementioned control method, a first machine learning model can be applied to process the user's operational intentions and then generate a first control command. In this way, processing the user's operational intentions eliminates the need for complex manual input or preset commands, improving the vehicle's responsiveness to ambiguous intentions or non-standard commands, and significantly enhancing the convenience and intelligence of the interaction. Furthermore, when generating the first control command, the first machine learning model combines vehicle status information and first application programming interface information, enabling the generated first control command to accurately respond to the user's operational intentions, thereby meeting diverse driving needs and improving the user's driving experience.

[0099] According to some embodiments, in step S303, the input data further includes: first policy information, which is obtained from a second database based on the operation intention. The second database is associated with the Internet community. Before step S303, method 300 further includes: receiving the first policy information sent by the server.

[0100] Based on the above control method, when generating the first control command, the first machine learning model combines the first strategy information generated when vehicles of the same or similar models respond to the operation intention stored in the server. In this way, by referring to the execution data of the same or similar vehicles, the generated first control command can be more adapted to the current vehicle characteristics and the user's operation intention, thereby effectively improving the accuracy and generation efficiency of the first control command.

[0101] Optionally, the second database can be a database stored on a server, or an internet community knowledge base. The second database can store control commands generated when vehicles of the same or similar models respond to operational intentions.

[0102] Optionally, in this embodiment, the training samples during the training of the first machine learning model may include first sample policy information.

[0103] According to some embodiments, the first policy information is uploaded by the second user, or the first policy information is collected and uploaded by the vehicle after authorization by the first user.

[0104] Based on the above control method, when the data stored in the second database is uploaded by other users, the first machine learning model generates a first control command with higher accuracy and efficiency by referencing the data uploaded by other users. When the data stored in the second database is data collected and uploaded by the vehicle after authorization from the first user, the first machine learning model can generate a first control command that conforms to the first user's driving habits by referring to the first user's historical driving data, thereby meeting the first user's personalized driving needs. On the other hand, without the first user's authorization, the vehicle will not collect or upload the first user's historical driving data. This design fully respects the user's privacy rights and effectively avoids the potential risk of information leakage.

[0105] For example, the second user can upload the first strategy information via a mobile phone, tablet, or vehicle. For instance, the second user can upload the first strategy information about the ambient lighting settings via a mobile phone, tablet, or vehicle. When the first user's intention is to turn on the ambient lighting, the first machine learning model can refer to the first strategy information when outputting the first control command.

[0106] In another example, before collecting the first user's driving data, the vehicle can display a pop-up window on the central control screen asking the first user for their consent to collect and upload the driving data. After obtaining the first user's authorization, the vehicle can collect and upload the first user's driving data, which may include first policy information.

[0107] According to some embodiments, the first state information includes: the state information of the ambient light, the first control command for controlling the color and brightness of the ambient light, and / or the first control command for controlling the content played on the head-up display in the vehicle.

[0108] Based on the above control method, the vehicle can combine the user's intention to schedule ambient lighting, the status information of the ambient lighting in the cabin, and the application programming interface information corresponding to scheduling the ambient lighting to control the color and brightness of the ambient lighting, and / or control the content played on the head-up display in the vehicle. In this way, the first control command can be generated accurately and efficiently, thereby flexibly meeting the user's adjustment needs for the in-vehicle ambient lighting and ensuring that users can easily create an ideal in-vehicle atmosphere in different scenarios, further enhancing driving comfort and personalized experience.

[0109] Optionally, the first control command may also be used to indicate the duration of the ambient light flashing, or the time interval between ambient light color changes.

[0110] Optionally, the status information of the ambient lighting may include at least one of the following: the position, brightness, color, or flashing time of the ambient lighting in the cabin.

[0111] For example, the status information of the ambient light indicates that the current ambient light is yellow and the flashing time is 10 seconds. When the first user's operation intention is "turn on the ambient light", the first control instruction is used to indicate that the color change sequence of the ambient light is yellow, blue and green in sequence, with a flashing interval of 2 seconds. In addition, the first control instruction is also used to control the head-up display to play the game theme image.

[0112] According to some embodiments, the first state information further includes: seat occupancy information in the cabin and vehicle gear status information, and the first control command is used to control the color and brightness of the ambient light in the first area of ​​the seat, the first area including the area corresponding to the occupied seat.

[0113] Based on the above control method, when the first state information includes seat occupancy information in the cabin and vehicle gear information, the first control command for ambient lighting generated by the first machine learning model can be adapted to the seat occupancy status, ensuring that the control of the color and brightness of the ambient lighting in the area where the seat is occupied conforms to the user's operating intention, thereby further improving the user's driving experience.

[0114] For example, the first status information indicates that the driver's seat is occupied and the vehicle is in P gear. When the user's intention is to "turn on the ambient lighting", the first control command can control the color and brightness of the ambient lighting in the driver's seat area to change.

[0115] In another example, the first status information indicates that the driver's seat is occupied and the vehicle is in P gear. When the user's intention is to "turn on the ambient lighting", the first control command can control the color and brightness of the ambient lighting in the driver's and passenger's seat areas to change.

[0116] According to some embodiments, before step S303, method 300 further includes: filtering noise and performing time synchronization processing on the first state information.

[0117] Based on the aforementioned control method, before inputting the input data into the first machine learning model, noise filtering and time synchronization processing are performed on the first state information. This effectively removes environmental noise or error signals that may exist during the sensor acquisition of the first state information, thereby improving the accuracy of the first state information data. Simultaneously, time synchronization processing corrects time deviations between multiple data sources in the first state information, ensuring the consistency of the first state information in the time dimension, thus providing a reliable foundation for the analysis and calculation of the first machine learning model. This preprocessing method not only significantly improves the prediction accuracy of the first machine learning model but also enhances its adaptability to complex scenarios.

[0118] According to some embodiments, the input data further includes: second state information, which is used to indicate environmental information outside the cockpit. Before step S303, method 300 further includes: acquiring the second state information.

[0119] Based on the aforementioned control method, by incorporating environmental information from outside the cockpit into the generation of control commands, the vehicle can more accurately understand and respond to the user's operational intentions. This approach not only effectively enhances the intelligence and personalization of the initial control commands but also better adapts to the diverse needs of users in various driving scenarios, thereby significantly improving the overall driving experience and making the driving process safer, more efficient, and more comfortable.

[0120] Optionally, the second state information can be acquired by sensors. For example, when the second state information includes image information outside the cockpit, this image information can be acquired by a camera. Or, for example, when the second state information includes point cloud data, the point cloud data can be acquired by LiDAR.

[0121] Optionally, in the above embodiments, the training samples of the first machine learning model during training may include second sample state information.

[0122] According to some embodiments, the first state information includes humidity information inside the cabin, the second state information includes precipitation intensity information outside the cabin, and the first control command includes at least one of the following: controlling the cleaning frequency of the windshield wipers, controlling the radiation distance and brightness of the headlights, controlling whether the dehumidification mode of the air conditioner is turned on, or controlling whether the traction control system and the electronic stability system are activated.

[0123] Based on the above control method, the vehicle can combine the user's intention to activate the rain driving mode, the humidity information inside the cabin, the precipitation intensity information outside the cabin, and the description information of the application that dispatches the wipers, lights, air conditioning, or traction control system and electronic stability system to generate a first control command that is suitable for driving in rainy weather. In this way, the first control command can be generated accurately and efficiently, thereby ensuring the user's safety and driving experience during rainy driving.

[0124] For example, the first state information indicates that the humidity inside the cabin is 20%, the second state information is used to indicate that the precipitation intensity outside the cabin is greater than or equal to a first threshold, and the first user's operation intention is to activate the rain driving mode. The first control command may include: increasing the refresh frequency of the windshield wipers and controlling the air conditioner to turn on the dehumidification function.

[0125] According to some embodiments, the first state information also includes vehicle speed information, the second state information also includes first image information and first point cloud information around the vehicle, the first point cloud information is used to indicate the distance between the vehicle and the obstacle, and the first control command includes: controlling whether the traction control system and the electronic stability system are activated.

[0126] In this embodiment, when the vehicle detects a user's intention to activate the rain driving mode while driving in rainy weather, the first machine learning model can combine vehicle speed information, image information of the vehicle's surroundings, and point cloud information to generate a first control command to control whether the traction control system and electronic stability system are activated. This ensures safe driving of the vehicle in rainy weather, thereby guaranteeing the user's driving safety.

[0127] For example, when the first image information indicates that there are multiple obstacles around the vehicle, and the first point cloud information indicates that the distance between the vehicle and the nearest obstacle is less than or equal to a safe distance, the first control command is used to instruct the traction control system and the electronic stability system to be activated.

[0128] The following sections, with reference to Figures 4 to 12, detail the control scenarios for the vehicle's ambient lighting and driving in the rain.

[0129] Figure 4 is another system architecture diagram applicable to the control method provided in the embodiments of this application. This system architecture diagram can be applied to the control scenario of ambient lighting.

[0130] As shown in Figure 4, the system architecture includes a multimodal scene understanding module, an AIGC model, and in-cabin equipment. The multimodal scene understanding module comprises a data preprocessing module, a feature fusion module, and a multimodal information extraction module.

[0131] The data preprocessing module can extract modal information from the seat occupancy status information, gear position status information, and ambient light status information. The extracted modal information can include: seat occupancy features with timestamps, gear position features with timestamps, and ambient light features with timestamps.

[0132] For example, as shown in Figure 5, seat status information, gear position information, and ambient light status information can undergo data cleaning and noise removal processes using digital filters. Since this information is collected by sensors, the data cleaning and noise removal process is to ensure that the sensors are not malfunctioning or that noise is eliminated as much as possible. When one or more sensors fail to function properly (e.g., the sensor does not detect the occupancy status of the driver's seat), default values ​​(e.g., the driver's seat is vacant) can be used to fill in the gaps, thereby avoiding data omissions. The seat status information can be obtained from data collected by a camera or seat sensors. The image information collected by the camera can be used for object detection using a CNN to determine whether the first user is in the cabin. When the first user is in the cabin, the CNN can extract the coordinates of the first user and the seat occupancy status from the image information, thereby forming seat status labels (e.g., driver's seat occupied, front passenger seat vacant, second row left seat vacant, etc.). After the data preprocessing module timestamps the seat status labels, it can obtain the seat occupancy features to be timestamped (e.g., time T1: "driver's seat occupied"). Similarly, gear position information can be processed by a CNN to convert it into standard labels (e.g., "D" indicates the vehicle is moving forward, "P" indicates the vehicle is parked, and "N" indicates neutral). After timestamp alignment, the standard labels yield timestamped gear features (e.g., time T1: "gear D"). Likewise, a CNN can determine the current ambient light color (e.g., red, green, or blue) and brightness (e.g., 30%, 50%, or 80%). After timestamp alignment, the ambient light features yield timestamped features (e.g., time T1: "ambient light is red, brightness is 50%").

[0133] After obtaining the timestamped seat occupancy features, timestamped gear position features, and timestamped ambient light features, data fusion can be performed through the feature fusion module.

[0134] As shown in Figure 6, the feature fusion module can use a fully connected neural network to fuse data from different modalities (including: timestamped seat occupancy features, timestamped gear position features, and timestamped ambient light status features) into a unified feature vector representation. For example, the fully connected neural network can concatenate various label sequences into a single large label sequence to obtain the fused feature vector, which represents the current vehicle state and user intent. For instance, the large label sequence includes: time T1: ["Driver's seat occupied", "Gear D", "Ambient light is red, brightness 50%"] and time T2: ["Driver's seat not occupied", "Gear P", "Ambient light is blue, brightness 80%"].

[0135] After obtaining the fused feature vector, the fused feature vector and other information can be input into the multimodal information extraction module for information extraction.

[0136] As shown in Figure 7, the multimodal information extraction module can extract and infer information from the fused feature vector, the API information corresponding to the ambient light, and the ambient light strategy information to obtain scene information (including text descriptions) inside and outside the cockpit; and input the scene information, ambient light strategy information, and the API information corresponding to the ambient light into the AIGC model.

[0137] The AIGC model can generate first control commands based on scene information, ambient lighting strategy information, and ambient lighting API information. During the generation of these commands, the model can consider the user's operational intent, making the generated scheduling strategies and commands more aligned with the user's intentions. For example, the AIGC model can recognize the user's voice commands (e.g., "turn on the ambient lights") and generate creative scheduling strategies for the vehicle based on the command content. This functionality allows users to interact with the system through natural language without relying on complex manual input or preset commands, significantly improving the convenience and intelligence of the interaction. Furthermore, during training, the AIGC model can utilize data from a second database covering multiple areas such as vehicle scheduling, traffic management, and user preferences, helping the model build a more comprehensive and accurate knowledge system. This model training method enables the AIGC model to generate innovative and highly adaptable scheduling strategies based on the latest industry dynamics, technological trends, and real-time traffic conditions.

[0138] For example, as shown in Figure 5, the AIGC model can include scheduling condition generation, scheduling relationship generation, scheduling time generation, and scheduling strategy generation during inference. Scheduling condition generation can include determining whether scheduling execution conditions are met based on the first user's operational intent and perceived information about the cabin environment. For example, if the first user's operational intent is "turn on the ambient lighting," the scheduling conditions can include "driver's seat occupancy and vehicle gear in D." If these conditions are met, the vehicle system can activate the ambient lighting and the head-up display (HUD). Scheduling relationship generation can include the objects to be scheduled. For example, if the first user's operational intent is "turn on the ambient lighting," the scheduling relationship is to schedule the ambient lighting and the HUD. Scheduling time generation can include the duration of the scheduling. In the context of ambient lighting, the scheduling time can be the duration of the ambient lighting changes. Scheduling strategy generation can include controlling the color and brightness changes of the lights based on the user's operational intent, the cabin atmosphere (e.g., "dynamic" or "intense"), and the scene, and controlling the HUD to display specific images based on the user's operational intent, the cabin atmosphere, and the scene. For example, when the first user's intention is to "turn on the ambient lighting," the ambient lighting scheduling strategy includes controlling the color and brightness of the ambient lighting to change sequentially to red (80% brightness), blue (80% brightness), and yellow (60% brightness), with each color change occurring at 3-second intervals, thereby creating a dynamic atmosphere. As another example, when the first user's intention is to "turn on the ambient lighting," the HUD can display a first image that matches a dynamic theme (e.g., a gaming theme). Yet another example is when the first user's intention is to "turn on the comfort ambient lighting," the HUD can display a second image that matches a comfort theme (e.g., a natural landscape theme).

[0139] After the scheduling strategy is generated, the rationality verification module in the AIGC model can verify whether the scheduling strategy is reasonable. For example, the rationality verification module can verify whether the brightness setting of the ambient light meets safety standards, whether the brightness setting of the ambient light meets the user's desired range, whether the HUD display is clear, or whether the scheduling time of the ambient light and HUD is appropriate.

[0140] If the rationality verification module passes the above verification, it can send the first control command to the in-cabin equipment. Accordingly, the in-cabin equipment can control the color and brightness of the ambient lights and the displayed image on the HUD based on the first control command. For example, the in-cabin equipment can control the color and brightness changes of the ambient lights sequentially as follows: red (80% brightness), blue (80% brightness), and yellow (60% brightness), with each color change occurring at a 3-second interval to create a dynamic range, and control the HUD to display the first image.

[0141] In this embodiment, the vehicle can obtain vehicle status information, user operation intent, ambient light strategy information, and ambient light corresponding API information in real time. By processing the above information through the AIGC model, the first control command can be generated accurately and efficiently to meet the user's needs for adjusting the ambient light, thereby improving the user's driving experience.

[0142] The following section, with reference to Figures 8 to 10, describes the human-computer interaction process for ambient lighting scheduling.

[0143] As shown in Figure 8(a), the vehicle's central control screen displays interface 800 and function bar 810. Interface 800 includes user account login information 801 (the vehicle is not currently logged into), a Bluetooth function icon 802, a Wi-Fi function icon 803, a cellular network signal icon 804, an in-vehicle map application search box 805, a card to switch to displaying all applications installed in the vehicle 806, a card to switch to displaying the in-vehicle music application 807, a display card 808 showing the vehicle's remaining battery power and remaining driving range, and a display card 809 showing the vehicle's 360-degree (°) surround view function. The in-vehicle map application search box 805 may include user-defined controls for going home 8051 and going to work 8052. The function bar 810 includes an icon 811 for switching to the central control screen desktop, an icon 812 for vehicle internal air circulation, an icon 813 for driver's seat heating function, an icon 814 for driver's area air conditioning temperature display, an icon 815 for passenger's area air conditioning temperature display, an icon 816 for passenger's seat heating function, and a volume setting icon 817. As shown in Figure 8(b), before the first user interacts with the vehicle, a prompt box 818 can be displayed on the vehicle's central control screen to prompt the first user whether they agree to the vehicle collecting the first user's driving data and uploading it to the server. When the vehicle detects that the first user clicks the "agree" control in the prompt box 818, the vehicle can collect the first user's driving data and upload it to the server. This driving data may include the first strategy information in method 300 above, which is used to assist the vehicle in generating personalized control commands for the first user.

[0144] As shown in Figure 9, a first user issues a voice command to the vehicle while driving, "Please turn on the ambient lighting." Upon receiving this voice command, the vehicle can recognize it to determine the user's intention. Then, the vehicle can interact with a server to obtain first strategy information (e.g., this first strategy information may come from a first database, indicating the vehicle's response strategy after a similar voice command was issued by the first user in historical contexts). Simultaneously, the vehicle can search a local second database to obtain the API information corresponding to the ambient lighting. This API information indicates the control logic of the ambient lighting, including on / off operations, color switching, brightness adjustment, and mode settings, as well as related parameters and functional descriptions. Finally, by inputting the user's intention, the first strategy information, and the API information into the AIGC model, a first control command is obtained. This first control command controls the ambient lighting's color, brightness, and color change interval, and also controls the HUD to display specific content.

[0145] As shown in Figure 10, after the vehicle executes the first control command, the color and brightness of the ambient lights on the vehicle change sequentially to red (80% brightness), blue (80% brightness), and yellow (60% brightness), with each color change occurring at a 3-second interval, thus creating a dynamic atmosphere. On the other hand, the HUD can display interface 819, which includes game posters, used to complement the ambient lights in creating the dynamic atmosphere.

[0146] Figure 11 is another system architecture diagram applicable to the control method provided in the embodiments of this application. This system architecture diagram can be applied to control scenarios for driving in rainy weather.

[0147] As shown in Figure 11, a camera, LiDAR, and rain sensor can be installed in the vehicle to acquire image information, point cloud information, and precipitation intensity information, respectively. A perception system is set up in the multimodal scene understanding module to process the image information, point cloud information, and precipitation intensity information. For example, as shown in Figure 12, a CNN can be used to process the image information acquired by the camera to obtain the environmental information outside the cabin (image format). The point cloud information can be processed using clustering, filtering, or feature extraction algorithms to obtain the environmental information outside the cabin (point cloud format), and a 3D model of the vehicle's surrounding environment can be generated based on this information, thereby helping to determine the distance between the vehicle and obstacles. Precipitation intensity information can be represented by capacitance signals; the precipitation intensity information outside the cabin can be obtained by recognizing capacitance signals using a CNN. The perception system can form environmental perception information outside the cabin based on the environmental information and precipitation intensity information outside the cabin, and input this information into the multimodal extraction module.

[0148] A humidity and temperature sensor, a gyroscope, and a vehicle speed sensor are installed in the cockpit to collect humidity and temperature information, vehicle angular velocity information, and vehicle speed information, respectively. After the above information is processed by the data processing module and the feature fusion module, the fused feature vector can be obtained.

[0149] The multimodal information extraction module can combine the fused feature vectors, the API information for rainy driving in the first database, the rainy strategy information in the second database, and the external environment perception information to extract and infer information, obtain scene information (including text descriptions) inside and outside the cockpit, rainy strategy information, and API information for rainy driving, and input the above information into the AIGC model.

[0150] The AIGC model can generate initial control commands based on scene information, rain strategy information, and API information to adjust the status of the windshield wipers, lights, and air conditioning system, and activate hydroplaning mode when necessary to ensure vehicle safety and comfort. When generating the initial control commands, the AIGC model can refer to the user's intentions, making the generated commands more aligned with those intentions. For example, the AIGC model can recognize user voice commands (e.g., activating rain driving mode) and generate creative dispatch strategies for the vehicle based on the command content and the aforementioned information. This functionality allows users to interact with the system through natural language without relying on complex manual input or preset commands, significantly improving the convenience and intelligence of the interaction.

[0151] For example, the AIGC model can include scheduling condition generation, scheduling relationship generation, scheduling time generation, and scheduling strategy generation during inference. Scheduling condition generation can include: detecting precipitation intensity greater than or equal to a first preset value, vehicle speed within a first preset range, or detecting in-vehicle humidity greater than or equal to a preset value, etc. Scheduling relationship generation can include the objects to be scheduled; for example, if the first user's intention is "to activate rain driving mode," the scheduling relationship would be: windshield wipers, HUD, air conditioning, headlights, and other in-cabin equipment. Scheduling time generation can include the duration of the scheduling; in a rainy scenario, the scheduling time could be: the on time of the air conditioning, the on time of the windshield wipers, or the duration of the headlights, etc.

[0152] The scheduling strategy generation can include generating strategies to control vehicle components based on one or more of the following: precipitation intensity, environmental information inside and outside the vehicle cabin, vehicle driving status, rain strategy information, or API information. For example, in the case of light precipitation (precipitation intensity greater than or equal to a second preset value), the scheduling strategy can include: increasing the cleaning frequency of the windshield wipers, and maintaining the current settings of the air conditioning system and headlights. As another example, in the case of moderate precipitation (precipitation intensity greater than or equal to a third preset value), the scheduling strategy can include: automatically adjusting the cleaning frequency of the windshield wipers to ensure cleaning effectiveness; controlling the air conditioning system to activate dehumidification mode when the humidity inside the vehicle exceeds 70%; preventing window fogging; and controlling the headlights to switch to low beam. For example, in the event of heavy precipitation (precipitation intensity greater than or equal to the fourth preset value), the scheduling strategy may include: further increasing the cleaning frequency of the windshield wipers to quickly clear the windshield; controlling the air conditioning system to turn on dehumidification mode and enhance airflow circulation inside the vehicle to ensure that the windows do not fog up; controlling the headlights to turn on automatic mode, that is, the vehicle can automatically switch to low beam or high beam based on external light conditions; turning on the window defrost function to prevent water droplets from accumulating on the glass surface; and controlling the air purification system to turn on to ensure that the air inside the cabin is clean and to remove moisture and musty smells. For example, in the event of heavy rain or extreme precipitation (precipitation intensity greater than or equal to the fourth preset value), the dispatch strategy may include: controlling the wiper cleaning frequency to the highest frequency to ensure clear visibility for the driver; controlling the air conditioning system to activate the dehumidification mode to ensure dry air in the cabin and prevent fogging of the windows; automatically switching between high beams and low beams based on vehicle speed and distance from the vehicle in front to avoid affecting other vehicles; activating traction control and electronic stability systems when the road surface is slippery or there is standing water; and activating hydroplaning mode when the vehicle speed is less than or equal to 50 km / h.

[0153] After the scheduling strategy is generated, the rationality verification module ensures that its execution is safer, more efficient, and meets user needs. Specifically, this module verifies various operations and adjustments within the scheduling strategy, assessing whether they meet the stringent safety and comfort requirements during vehicle operation. This effectively mitigates potential risks and prevents additional energy consumption due to unreasonable operations. Furthermore, the rationality verification module possesses real-time environmental awareness capabilities, continuously monitoring changes in the vehicle's external environment (such as road conditions, traffic flow, and weather changes). Based on these changes, it quickly identifies inadequacies in the scheduling strategy and makes timely optimizations. For example, it adjusts the scheduling strategy promptly when rainfall intensity outside the cabin changes or humidity inside the cabin changes.

[0154] After the rationality verification is passed, the AIGC model can send the first control command to the cabin equipment. Accordingly, the cabin equipment can control the windshield wipers, air conditioning system, headlights and other cabin equipment to work based on the first control command.

[0155] In this embodiment, the system can acquire real-time environmental information inside and outside the cockpit, user's operational intent, rainy weather strategy information, and API information corresponding to rainy weather driving, and perform scene understanding and information fusion to accurately and efficiently generate the first control command, thereby ensuring the user's safety and driving experience during rainy weather driving.

[0156] It should be understood that, in the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terms and / or descriptions between the various embodiments are consistent and can be referenced by each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0157] Figure 13 is a schematic diagram of a control device provided in an embodiment of this application. The device 1300 may include an acquisition unit 1310, a transceiver unit 1320, and a processing unit 1330. The acquisition unit 1310 is used to acquire instructions and / or data; the transceiver unit 1320 is used to receive or send instructions and / or data; and the processing unit 1330 is used to perform data processing so that the device 1300 implements the aforementioned control method.

[0158] Optionally, the device 1300 further includes a storage unit for implementing corresponding storage functions and storing corresponding instructions and / or data.

[0159] In one embodiment, the device 1300 includes: an acquisition unit 1310 and a processing unit 1330; the acquisition unit 1310 is configured to acquire the operation intention and first state information of a first user, wherein the operation intention is obtained by recognizing a first input from the first user, and the first state information includes the state information of the vehicle; the processing unit 1330 is configured to: acquire first application programming interface (API) information from a first database according to the operation intention, wherein the first API information includes descriptive information in response to the operation intention; input input data into a first machine learning model to obtain a first control instruction, wherein the input data is obtained based on at least the following: operation intention, first state information, and first API information, and the first control instruction is used to schedule components or functions in the vehicle.

[0160] In one possible implementation, the input data further includes: first policy information, which is obtained from a second database based on the operational intent. The second database is associated with an internet community. The device also includes a transceiver unit 1320, which is used to receive the first policy information sent by the server.

[0161] In one possible implementation, the first policy information is uploaded by the second user, or the first policy information is collected and uploaded by the vehicle after authorization by the first user.

[0162] In one possible implementation, the first state information includes: the state information of the ambient light, the first control command for controlling the color and brightness of the ambient light, and / or the first control command for controlling the content played on the head-up display in the vehicle.

[0163] In one possible implementation, the first state information further includes: seat occupancy information in the cabin and vehicle gear status information; the first control command is used to control the color and brightness of the ambient light in the first area, and the first area includes the area corresponding to the occupied seat.

[0164] In one possible implementation, the processing unit 1330 is further configured to perform noise filtering and time synchronization processing on the first state information.

[0165] In one possible implementation, the input data further includes: second state information, which is used to indicate environmental information outside the cockpit; and the acquisition unit 1310, which is also used to acquire the second state information.

[0166] In one possible implementation, the first state information includes humidity information inside the cabin, the second state information includes precipitation intensity information outside the cabin, and the first control command includes at least one of the following: controlling the cleaning frequency of the windshield wipers, controlling the radiation distance and brightness of the headlights, controlling whether the dehumidification mode of the air conditioner is turned on, or controlling whether the traction control system and the electronic stability system are activated.

[0167] In one possible implementation, the first state information also includes vehicle speed information, and the second state information also includes first image information and first point cloud information around the vehicle. The first point cloud information is used to indicate the distance between the vehicle and the obstacle. The first control command includes: controlling whether the traction control system and the electronic stability system are activated.

[0168] Figure 14 is a schematic diagram of another control device provided in an embodiment of this application.

[0169] The device 1400 includes a memory 1410, a processor 1420, and a communication interface 1430. The memory 1410, processor 1420, and communication interface 1430 are connected via an internal connection path. The memory 1410 stores instructions, and the processor 1420 executes the instructions stored in the memory 1410 to control the communication interface 1430 to acquire information, thereby enabling the device 1400 to implement the aforementioned control method. Optionally, the memory 1410 can be coupled to the processor 1420 via an interface, or it can be integrated with the processor 1420.

[0170] It should be noted that the communication interface 1430 described above uses a transceiver device, such as, but not limited to, a transceiver. The communication interface 1430 may also include an input / output interface.

[0171] The processor 1420 stores one or more computer programs, which include instructions. When the instructions are executed by the processor 1420, the control device 1400 performs the control methods described in the above embodiments.

[0172] In implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 1420 or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 1410, and the processor 1420 reads the information in memory 1410 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0173] Optionally, the communication interface 1430 in FIG14 can implement the acquisition unit 1310 or the transceiver unit 1320 in FIG13, and the processor 1420 in FIG14 can implement the processing unit 1330 in FIG13.

[0174] This application also provides a computer-readable storage medium storing program code that, when executed on a computer, causes the computer to perform the control method shown in FIG3 above.

[0175] This application also provides a computer program product, which includes a computer program that, when run, causes the computer to execute the control method shown in FIG3 above.

[0176] This application embodiment also provides a chip, including: a circuit for performing the method shown in FIG3 above.

[0177] This application also provides a vehicle, which includes a control device as shown in FIG13 or FIG14.

[0178] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0179] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0183] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0184] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A control method, characterized in that, The method includes: The operation intention and first state information of the first user are obtained. The operation intention is obtained by recognizing the first input of the first user. The first state information includes the state information of the vehicle. According to the operation intention, first application interface information is obtained from the first database, and the first application interface information includes description information in response to the operation intention; Input data is fed into a first machine learning model to obtain a first control instruction. The input data is obtained from at least the following: the operation intention, the first state information, and the first application programming interface information. The first control instruction is used to schedule components or functions in the vehicle.

2. The method as described in claim 1, characterized in that, The input data further includes: first strategy information, which is obtained from a second database based on the operational intent. The second database is associated with internet communities. Before inputting the input data into the first machine learning model to obtain the first control command, the method further includes: Receive the first policy information sent by the server.

3. The method as described in claim 2, characterized in that, The first policy information is uploaded by the second user, or the first policy information is collected and uploaded by the vehicle after authorization by the first user.

4. The method according to any one of claims 1 to 3, characterized in that, The first status information includes: the status information of the ambient light, the first control command used to control the color and brightness of the ambient light, and / or the first control command used to control the content played on the head-up display in the vehicle.

5. The method as described in claim 4, characterized in that, The first status information also includes: seat occupancy information in the cabin and gear status information of the vehicle. The first control command is used to control the color and brightness of the ambient light in the first area, which includes the area corresponding to the occupied seat.

6. The method as described in claim 5, characterized in that, Before inputting the input data into the first machine learning model to obtain the first control command, the method further includes: The first state information is processed for noise filtering and time synchronization.

7. The method according to any one of claims 1 to 3, characterized in that, The input data further includes: second state information, which indicates environmental information outside the cockpit. Before inputting the input data into the first machine learning model to obtain the first control command, the method further includes: Obtain the second status information.

8. The method as described in claim 7, characterized in that, The first status information includes humidity information inside the cabin, the second status information includes precipitation intensity information outside the cabin, and the first control command includes at least one of the following: controlling the cleaning frequency of the windshield wipers, controlling the radiation distance and brightness of the headlights, controlling whether the dehumidification mode of the air conditioner is turned on, or controlling whether the traction control system and electronic stability system are activated.

9. The method as described in claim 8, characterized in that, The first status information also includes vehicle speed information, and the second status information also includes first image information and first point cloud information around the vehicle. The first point cloud information is used to indicate the distance between the vehicle and the obstacle. The first control command includes: controlling whether the traction control system and the electronic stability system are activated.

10. A control device, characterized in that, The device includes: an acquisition unit and a processing unit; The acquisition unit is used to acquire the operation intention and first status information of the first user. The operation intention is obtained by recognizing the first input of the first user, and the first status information includes the status information of the vehicle. The processing unit is used for: According to the operation intention, first application interface information is obtained from the first database, and the first application interface information includes description information in response to the operation intention; Input data is fed into a first machine learning model to obtain a first control instruction. The input data is obtained from at least the following: the operation intention, the first state information, and the first application programming interface information. The first control instruction is used to schedule components or functions in the vehicle.

11. The apparatus as claimed in claim 10, characterized in that, The input data further includes: first strategy information, which is obtained from a second database based on the operation intention, the second database being associated with internet communities, and the device further includes a transceiver unit; The transceiver unit is used to receive the first policy information sent by the server.

12. The apparatus as claimed in claim 11, characterized in that, The first policy information is uploaded by the second user, or the first policy information is collected and uploaded by the vehicle after authorization by the first user.

13. The apparatus as claimed in any one of claims 10 to 12, characterized in that, The first status information includes: the status information of the ambient light, the first control command used to control the color and brightness of the ambient light, and / or the first control command used to control the content played on the head-up display in the vehicle.

14. The apparatus as claimed in claim 13, characterized in that, The first status information also includes: seat occupancy information in the cabin and gear status information of the vehicle. The first control command is used to control the color and brightness of the ambient light in the first area, which includes the area corresponding to the occupied seat.

15. The apparatus as claimed in claim 14, characterized in that, The processing unit is also used to filter noise and perform time synchronization processing on the first state information.

16. The apparatus as claimed in any one of claims 10 to 12, characterized in that, The input data also includes: second status information, which is used to indicate environmental information outside the cockpit; The acquisition unit is further configured to acquire the second status information.

17. The apparatus as claimed in claim 16, characterized in that, The first status information includes humidity information inside the cabin, the second status information includes precipitation intensity information outside the cabin, and the first control command includes at least one of the following: controlling the cleaning frequency of the windshield wipers, controlling the radiation distance and brightness of the headlights, controlling whether the dehumidification mode of the air conditioner is turned on, or controlling whether the traction control system and electronic stability system are activated.

18. The apparatus as claimed in claim 17, characterized in that, The first status information also includes vehicle speed information, and the second status information also includes first image information and first point cloud information around the vehicle. The first point cloud information is used to indicate the distance between the vehicle and the obstacle. The first control command includes: controlling whether the traction control system and the electronic stability system are activated.

19. A control device, characterized in that, The device includes a processor and a memory, the processor being coupled to the memory, the memory being used to store computer programs or instructions, and the processor being used to execute the computer programs or instructions in the memory, such that the method of any one of claims 1 to 9 is performed.

20. A chip, characterized in that, The chip includes circuitry for performing the method as described in any one of claims 1 to 9.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that, when executed on a computer, causes the computer to perform the method as described in any one of claims 1 to 9.

22. A computer program product, characterized in that, The computer product includes a computer program that, when run, causes the computer to perform the method as described in any one of claims 1 to 9.

23. A vehicle, characterized in that, Includes the control device as described in any one of claims 10 to 19.

Images