A vehicle air conditioner control method and system, computer equipment and medium

Abstract

The application provides a vehicle-mounted air conditioner control method and system, computer equipment and medium, comprising: obtaining multi-modal data, encoding the multi-modal data, obtaining word tokens corresponding to the multi-modal data, and determining the intermediate representation of the multi-modal data based on the word tokens corresponding to the multi-modal data; forming an action-conditioned world model based on the intermediate representation; and finally determining an air conditioner control action according to the action-conditioned world model and a preset constraint condition, and controlling the vehicle-mounted air conditioner according to the determined air conditioner control action. The application can more completely represent the vehicle cabin thermal environment and the user thermal comfort state by using multi-modal data for vehicle-mounted air conditioner control, and supports individualized thermal comfort control. Meanwhile, the application adopts a model predictive control mechanism with constraints, which can explicitly meet the safety constraints of vehicle air conditioner control and improve the feasibility of deploying the action-conditioned world model on a vehicle.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a vehicle air conditioning control method and system, computer equipment and media. Background Technology

[0002] In some related technologies, vehicle air conditioning systems are typically controlled based on structured signals, or through rule-based strategies or calibration tables. However, a user's thermal comfort is not only related to the ambient temperature inside and outside the vehicle, but is also affected by many other factors. Relying solely on information such as temperature sensors and the temperature settings on the air conditioning panel may not adequately represent the user's true thermal comfort state, thus failing to accurately control the vehicle air conditioning. Summary of the Invention

[0003] This application provides a vehicle air conditioning control method and system, computer equipment and medium to solve or alleviate the problems described above.

[0004] This application provides a vehicle air conditioning control method, comprising the following steps: acquiring multimodal data, wherein the multimodal data is determined based on a target vehicle and a user using the vehicle air conditioning in the target vehicle; encoding the multimodal data to obtain lexical units corresponding to the multimodal data, and determining intermediate representations of the multimodal data based on the lexical units corresponding to the multimodal data; forming an action conditional world model based on the intermediate representations, wherein the action conditional world model is used to predict at least one of cabin state, thermal comfort state, energy consumption state, user intervention probability, and air conditioning action within a future preset time range; determining air conditioning control actions according to the action conditional world model and preset constraints, and controlling the vehicle air conditioning according to the determined air conditioning control actions.

[0005] In one embodiment of this application, the process of encoding the multimodal data to obtain the corresponding lexical units, and determining the intermediate representation of the multimodal data based on the corresponding lexical units includes: performing structured encoding on the multimodal data according to the data type to generate image modal lexical units, structured temporal lexical units, human static lexical units, thermal field lexical units, and action lexical units; encoding the image modal lexical units, structured temporal lexical units, human static lexical units, thermal field lexical units, and action lexical units to obtain the intermediate representations of the image modal lexical units, the structured temporal lexical units, the human static lexical units, the thermal field lexical units, and the action lexical units, respectively, and using them as the intermediate representations of the multimodal data.

[0006] In one embodiment of this application, the process of forming an action conditional world model based on the intermediate representation includes: decomposing the intermediate representation of the multimodal data into shared state representation and private state representation, and optimizing the shared state representation and the private state representation using a target loss; wherein the shared state representation is used to perform cross-modal representation of the multimodal data, and the private state representation is used to retain the unique information of each modality; wherein the target loss is obtained by weighting multiple losses; and forming the action conditional world model and a comfort prediction head based on the optimized shared state representation and private state representation; wherein the comfort prediction head is used to perform regression output on the prediction results of the action conditional world model.

[0007] In one embodiment of this application, the process of forming an action condition world model based on the intermediate representation further includes: using the current state representation, historical air conditioning actions, all air conditioning action sequences, external environment state, weather forecast state, current vehicle state, and air conditioning state as inputs to the action condition world model; and using the state representation within a future preset time range, the whole-body thermal comfort score within the future preset time range, the local thermal comfort score within the future preset time range, the air conditioning energy consumption estimate, the probability of user manual intervention, the probability of user readjustment, the defogging rate, the temperature change rate, and the adjustment action amplitude as outputs to construct the action condition world model; wherein, the state representation includes shared state representation and / or private state representation, the cabin state within the future preset time range is represented by the state representation within the future preset time range, the thermal comfort state within the future preset time range is represented by the local thermal comfort score within the future preset time range, the energy consumption state is represented by the air conditioning energy consumption estimate, the user intervention probability is represented by the probability of user manual intervention and the probability of user readjustment, and the air conditioning action is represented by the defogging rate, the temperature change rate, and the adjustment action amplitude.

[0008] In one embodiment of this application, the method further includes: designating the action conditional world model trained with all modal data as a first model, and designating the action conditional world model trained with some modal data as a second model; using the second model to learn the distillation objective of the first model, and using the learned second model to determine the state representation at the current moment; wherein the distillation objective includes at least one of: shared state representation, current thermal comfort prediction distribution, future thermal comfort prediction distribution, future state representation, latent state, and uncertainty estimation.

[0009] In one embodiment of this application, the process of determining the air conditioning control action based on the action condition world model and preset constraints includes: Obtain candidate air conditioner action sequences, which are predicted by the action condition world model and represent air conditioner actions within a preset time range in the future; The optimization objective is determined based on the preset constraints, and the air conditioning control action is determined from the candidate air conditioning action sequence based on the optimization objective; wherein, the preset constraints include at least one of the following: upper and lower limits of temperature, upper and lower limits of air volume, range of air outlet direction selection, action change rate limit, defogging rule, temperature change rate, air conditioning actuator constraint, user explicit constraint, user explicit preference constraint, and uncertainty backoff strategy.

[0010] In one embodiment of this application, the multimodal data includes vehicle interior image data, vehicle controller local area network (VLAN) bus data, vehicle interior environment data, vehicle exterior environment data, vehicle exterior weather data, user static human body attribute data, cabin temperature sensor deployment point temperature, cabin thermal field data, user thermal comfort score data, and user active air conditioning adjustment action data; the VLAN bus data includes at least one of vehicle speed, air conditioning set temperature, actual air outlet temperature, blower speed, air conditioning internal / external circulation status, compressor status, outside temperature, sunlight intensity, door status, and seat status; the user thermal comfort score data includes at least one of whole-body thermal comfort score, head thermal comfort score, back thermal comfort score, and foot thermal comfort score.

[0011] This application also provides an in-vehicle air conditioning control system, the system comprising: a data acquisition module for acquiring multimodal data, the multimodal data being determined based on a target vehicle and a user using an in-vehicle air conditioner in the target vehicle; an encoding module for encoding the multimodal data to obtain lexical units corresponding to the multimodal data, and determining intermediate representations of the multimodal data based on the lexical units corresponding to the multimodal data; a model building module for forming an action conditional world model based on the intermediate representations, the action conditional world model being used to predict at least one of cabin state, thermal comfort state, energy consumption state, user intervention probability, and air conditioning action within a future preset time range; and an in-vehicle air conditioning control module for determining air conditioning control actions according to the action conditional world model and preset constraints, and controlling the in-vehicle air conditioner according to the determined air conditioning control actions.

[0012] This application also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the vehicle air conditioning control method described in any one of the above.

[0013] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the vehicle air conditioning control method described in any one of the above.

[0014] The beneficial effects of this application are as follows: This application proposes an in-vehicle air conditioning control method and system, computer equipment, and medium. It acquires multimodal data, encodes the multimodal data to obtain corresponding lexical units, and determines intermediate representations of the multimodal data based on these lexical units. Then, it forms an action conditional world model based on these intermediate representations. This model predicts at least one of the following within a preset time range: cabin state, thermal comfort state, energy consumption state, user intervention probability, and air conditioning action. Finally, it determines the air conditioning control action based on the action conditional world model and preset constraints, and controls the in-vehicle air conditioning according to the determined action. Therefore, this application, by utilizing multimodal data for in-vehicle air conditioning control, can more completely represent the vehicle cabin thermal environment and user thermal comfort state, supporting individualized thermal comfort control. Furthermore, this application employs a constrained model predictive control mechanism, which can explicitly satisfy vehicle air conditioning control safety constraints, improving the feasibility of deploying the action conditional world model in vehicles. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0016] Figure 1 This is a schematic flowchart of a vehicle air conditioning control method provided in one embodiment of this application; Figure 2 This is a flowchart illustrating a vehicle air conditioning control method provided in another embodiment of this application. Figure 3 This is a schematic diagram of the hardware structure of an in-vehicle air conditioning control system provided in one embodiment of this application; Figure 4 This is a schematic diagram of the hardware structure of a computer device suitable for implementing one or more embodiments of this application. Detailed Implementation

[0017] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0018] It is understood that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0019] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0020] Thermal comfort is a person's subjective evaluation of their satisfaction with the surrounding thermal environment. The concept is defined by the ISO 7730 standard and involves physical, physiological and psychological dimensions.

[0021] Hidden states are the memory carriers for neural networks to process temporal information.

[0022] Uncertainty is a parameter that characterizes the dispersion of values ​​reasonably assigned to a measurand. It is a quantitative, non-negative indicator of the quality of measurement results. The smaller the value, the higher the quality and the greater the value of the measurement results; the larger the value, the lower the reliability of the results.

[0023] A token is the smallest unit of information that a model processes.

[0024] With the development of intelligent cockpits and new energy vehicles, in-vehicle air conditioning systems are no longer limited to traditional cooling, heating, defrosting, and defogging functions, but are gradually evolving towards personalization, intelligence, and energy efficiency. Existing vehicle air conditioning control systems typically operate based on structured signals (such as interior temperature, exterior temperature, sunlight sensor, set temperature, blower speed, etc.) or through rule-based strategies or calibration tables. However, the actual thermal comfort of a user is not only related to the ambient temperature inside and outside the vehicle, but is also influenced by various factors (such as user body size, clothing, posture, localized heat distribution within the cabin, airflow direction, vehicle operating status, sunlight exposure, historical air conditioning actions, and individual user preferences).

[0025] Meanwhile, traditional vehicle air conditioning systems primarily rely on temperature sensors and air conditioning panel settings, providing limited input information. They lack the ability to comprehensively model multi-source information such as user visual state, infrared thermal state, human attributes, local thermal field, CAN (Controller Area Network) status, and external weather, thus failing to fully represent the user's true thermal comfort state. Directly reconstructing or fusing all modalities can easily lead to the learning of information irrelevant to thermal comfort. For example, if model training aims only to reconstruct all modal data for the next moment, the model may learn more about short-term statistical patterns in temperature, CAN status, or visual appearance, rather than learning state representations truly useful for thermal comfort prediction and air conditioning control. Furthermore, during the data acquisition and model training phases, the model may be able to obtain privileged information such as human height, weight, clothing, cabin temperature sensor placement points, and questionnaire scores. However, in the actual mass production or deployment phase, vehicles typically only receive limited modal data such as RGB (red, green, and blue) image data, infrared image data, CAN bus data, and weather data. This leads to inconsistencies between the information available to the model during training and deployment. Existing solutions rarely systematically address this issue of full modality data during training and lack of modality data during deployment. Moreover, user adjustment behavior is influenced by current discomfort, individual preferences, the original vehicle's air conditioning strategy, environmental disturbances, and unobserved factors. If user actions from logs are directly used as supervisory labels to train the controller, the model may learn "when the user will adjust" rather than "how to adjust to improve future comfort," meaning the user adjustment actions collected by the model may not necessarily represent the optimal control actions. In addition, air conditioning control not only needs to optimize user comfort, but also needs to take into account constraints such as energy consumption, smoothness of operation, defogging safety, temperature change rate, and air volume change range. Existing methods are difficult to explicitly handle these engineering constraints, and there are deployment risks, thus lacking an air conditioning operation optimization mechanism oriented towards safety constraints.

[0026] Based on the above description, in an exemplary embodiment of this application, as follows: Figure 1As shown, this embodiment provides a vehicle air conditioning control method, including the following steps: S110, Acquire multimodal data, which is determined based on the target vehicle and the users using the vehicle's air conditioning. In some examples, the users using the vehicle's air conditioning include, but are not limited to, drivers or passengers using the air conditioning inside the target vehicle. The target vehicle includes, but is not limited to, new energy vehicles and fuel vehicles.

[0027] S120, Encode the multimodal data to obtain the corresponding word units of the multimodal data, and determine the intermediate representation of the multimodal data based on the corresponding word units of the multimodal data; S130, a motion condition world model is formed based on intermediate representations. This model is used to predict at least one of the following: cabin state, thermal comfort state, energy consumption state, user intervention probability, and air conditioning action within a future preset time range. In some examples, the value of the future preset time range can be selected or set according to actual conditions; no specific numerical limit is specified here. For example, the future preset time range can be 30 seconds, 60 seconds, 300 seconds, etc.

[0028] S140, determine the air conditioning control action based on the action condition world model and preset constraints, and control the vehicle air conditioning according to the determined air conditioning control action.

[0029] Therefore, this method, by utilizing multimodal data for vehicle air conditioning control, can more completely characterize the vehicle cabin thermal environment and user thermal comfort state, supporting individualized thermal comfort control. At the same time, this application adopts a constrained model predictive control mechanism, which can explicitly satisfy the vehicle air conditioning control safety constraints and improve the feasibility of deploying the action condition world model in vehicles.

[0030] In some embodiments, the multimodal data in step S110 includes, but is not limited to: vehicle interior image data, vehicle controller area network (CAN) bus data, vehicle interior environment data, vehicle exterior environment data, vehicle exterior weather data, user static human body attribute data, cabin temperature sensor deployment point temperature, cabin thermal field data, user thermal comfort score data, and user active air conditioning adjustment action data; the vehicle controller area network bus data includes at least one of vehicle speed, air conditioning set temperature, actual air outlet temperature, blower speed, air conditioning internal / external circulation status, compressor status, outside temperature, sunlight intensity, door status, and seat status; the user thermal comfort score data includes at least one of whole-body thermal comfort score, head thermal comfort score, back thermal comfort score, and foot thermal comfort score. In some examples, the user thermal comfort score data can be in the form of discrete levels, such as -3 to +3 points, or continuous values ​​or other comfort evaluation scales. In some examples, the vehicle interior image data includes in-vehicle RGB (red, green, blue) image data and / or in-vehicle infrared image data.

[0031] In some embodiments, when acquiring multimodal data, since different modal data have different sampling frequencies, a unified timestamp can be used as the primary index to construct a canonical timeline (standard timeline). For each time anchor point, corresponding visual segments, vehicle CAN bus data sequences, vehicle interior and exterior environmental data sequences, weather data sequences, user static human attribute data, and cabin temperature sensor deployment point temperature or cabin thermal field data can be extracted based on historical time windows. Modal missing markers and time interval features are then generated to address common issues in multimodal data acquisition such as asynchronous sampling, data missingness, partial frame loss, incomplete modal data, and time jitter.

[0032] In an exemplary embodiment of this application, step S120, which encodes multimodal data to obtain tokens corresponding to the multimodal data and determines intermediate representations of the multimodal data based on the tokens corresponding to the multimodal data, may include: performing structured encoding on the multimodal data according to the data type to generate image modal tokens, structured temporal tokens, human static tokens, thermal field tokens, and action tokens; encoding the image modal tokens, structured temporal tokens, human static tokens, thermal field tokens, and action tokens to obtain intermediate representations of the image modal tokens, structured temporal tokens, human static tokens, thermal field tokens, and action tokens, respectively; and using the intermediate representations of the image modal tokens, structured temporal tokens, human static tokens, thermal field tokens, and action tokens as intermediate representations of the multimodal data.

[0033] In some examples, image modal data such as in-vehicle RGB image data and in-vehicle infrared image data can be generated as image modal tokens. For non-image modal data such as vehicle CAN bus data, external weather data, user static attribute data, and cabin thermal field data, they can be encoded as structured tokens, time series tokens, or observation tuple tokens. For example, for regularly sampled structured time series, the numerical sequence of each variable within a historical window can be divided into time patches to form a structured time series token. A time patch divides the input data into a series of short time segments in chronological order; each time segment is called a patch. As another example, for irregularly sampled event stream data, observation tuple tokens can be formed, where each observation can be represented as: (time, variable_id, value, missing_mask, delta_t), where time represents the observation time, variable_id represents the variable identity, value represents the variable value, missing_mask represents the missing state, and delta_t represents the time interval from the anchor point. For example, for static human attributes, modal data such as gender, age, height, weight, and clothing can be encoded into static human tokens. These static human tokens can serve as input to the teacher model, but can be removed from the student model or replaced by individualized embeddings. The teacher model is the model trained using all modal data, including in-vehicle RGB image data, in-vehicle infrared image data, vehicle CAN bus data, external weather data, user static human attribute data, cabin thermal field data, cabin temperature sensor placement point temperatures, and historical questionnaire information. The student model can only use some modal data, such as in-vehicle RGB image data, in-vehicle infrared image data, vehicle CAN bus data, and weather data—modal data that can be stably obtained by the vehicle. For modal data such as height, weight, clothing, and cabin temperature sensor placement point temperatures that cannot be stably obtained, the student model cannot use them directly. Instead, it learns the hidden states and predictive capabilities of the teacher model through distillation. For example, cabin thermal field data or the temperatures of cabin temperature sensor deployment points can be encoded as thermal field tokens. If the cabin thermal field data is a two-dimensional matrix, it can be encoded using a convolutional neural network, a small visual Transformer, or a patch encoder; if it is the temperature of multiple cabin temperature sensor deployment points, it can be encoded using a structured time-series encoder.In some examples, the patch encoder divides an image or time series into fixed-size blocks, converts them into vector sequences through linear projection and positional encoding, and then inputs them into the Transformer Encoder for feature extraction and encoding. The Transformer Encoder is the encoding module in the Transformer architecture responsible for converting the input sequence into a high-dimensional vector representation rich in contextual semantics.

[0034] Therefore, this application can comprehensively utilize multimodal data such as in-vehicle RGB image data, in-vehicle infrared image data, vehicle CAN bus data, vehicle external weather data, user human static attribute data, cabin thermal field data, cabin temperature sensor deployment point temperature and historical questionnaire information, which can more completely characterize the vehicle cabin thermal environment and user thermal comfort state.

[0035] In an exemplary embodiment of this application, step S130, which involves forming an action conditional world model based on intermediate representations, may include: decomposing the intermediate representations of multimodal data into shared latents and private latents, and optimizing the shared latents and private latents using a target loss; wherein the shared latents are used to represent multimodal data across modalities, and the private latents are used to retain the unique information of each modality; wherein the target loss is obtained by weighting multiple losses; and forming an action conditional world model and a comfort prediction head based on the optimized shared latents and private latents; wherein the comfort prediction head is used to perform regression output on the prediction results of the action conditional world model.

[0036] In some examples, shared latent can be used to represent cabin thermal environment, user thermal state, air conditioning operation status, and thermal comfort-related status that are reflected across modalities, while private latent can be used to represent information unique to each modal data, such as posture and clothing appearance information in in-vehicle RGB image data, local thermal distribution information in in-vehicle infrared image data, and control status information in vehicle CAN bus data.

[0037] In some examples, the target loss can be obtained by weighting the cross-modal contrast loss, characterization prediction loss, key physical quantity prediction loss, and thermal comfort supervision loss, i.e.: L = λ1 * L_align + λ2 * L_future + λ3 * L_key + λ4 * L_comfort + λ5 * L_distill + λ6 * L_modality_drop, where L represents the target loss, L_align represents the cross-modal alignment loss, used to bring shared latents of different modalities closer together within the same time window and to separate shared latents of different time windows or different samples. `latent` represents the future representation prediction loss, used to predict the latent (state representation) for several future time steps based on the current multimodal state; `L_key` represents the key physical quantity prediction loss, used to predict future key temperatures, air conditioning status, low-dimensional representations of the thermal field, or other physical quantities related to thermal comfort; `L_comfort` represents the thermal comfort supervision loss, used to predict current and future thermal comfort scores on sparse questionnaire anchors; `L_distill` represents the teacher-student distillation loss; `L_modality_drop` represents the modality missing robustness training loss; `λ1`, `λ2`, `λ3`, `λ4`, `λ5`, and `λ6` are the corresponding weighting coefficients, where `λ1+λ2+λ3+λ4+λ5+λ6=1`. The future representation prediction loss means that instead of forcing the model to reconstruct all original images or all sensor columns, it predicts future state representations useful for thermal comfort and control, thereby improving the effectiveness of the state representation for downstream tasks.

[0038] In some examples, for the thermal comfort prediction loss, user thermal comfort ratings can be used as a low-frequency sparse supervision target instead of being directly incorporated into the main encoder as a high-frequency common input modality. For each questionnaire anchor point, multimodal inputs are extracted from the historical time window prior to that point, and the following are predicted: (1) the current whole-body thermal comfort rating; (2) the current local thermal comfort rating, including at least one of the head, back, and feet; (3) the whole-body thermal comfort rating within a specified future time range; and (4) the local thermal comfort rating within a specified future time range. The thermal comfort prediction loss can employ regression loss, classification loss, ordinal relation loss, or multi-task joint loss. For ordered ratings from -3 to +3, they can be simultaneously treated as ordered classification and regression tasks to improve the model's ability to characterize thermal comfort levels.

[0039] Therefore, this application enables the model to focus on learning states related to thermal comfort prediction and air conditioning control, rather than simply reconstructing all the original inputs, by decomposing the intermediate representations of each modality, aligning across modalities, predicting future representations, and supervising sparse comfort.

[0040] In an exemplary embodiment of this application, step S130, which forms an action condition world model based on intermediate representations, may further include: using the current state representation, historical air conditioning actions, all air conditioning action sequences, external environment state, weather forecast state, current vehicle state, and air conditioning state as inputs to the action condition world model; and using the state representation within a future preset time range, the whole-body thermal comfort score within a future preset time range, the local thermal comfort score within a future preset time range, the air conditioning energy consumption estimate, the probability of user manual intervention, the probability of user readjustment, the defogging rate, the temperature change rate, and the adjustment action amplitude as outputs to construct the action condition world model; wherein, the state representation includes shared state representation and / or private state representation, the cabin state within a future preset time range is represented by the state representation within a future preset time range, the thermal comfort state within a future preset time range is represented by the local thermal comfort score within a future preset time range, the energy consumption state is represented by the air conditioning energy consumption estimate, the user intervention probability is represented by the probability of user manual intervention and the probability of user readjustment, and the air conditioning action is represented by the defogging rate, the temperature change rate, and the adjustment action amplitude. Since a user's active adjustment of the air conditioning is not equivalent to the optimal air conditioning control action, this application does not directly use the user actions in the logs as the sole supervisory label to train the controller. Instead, it constructs an action-conditional world model. The action-conditional world model can be used to predict the cabin state, thermal comfort state, energy consumption state, user intervention probability, and air conditioning actions over a future period of time under the current state and candidate action sequences. In some examples, when constructing the action-conditional world model, the corresponding inputs may include: current multimodal state representation, historical air conditioning actions, air conditioning action sequence, external environment state, weather forecast state, current vehicle state, and air conditioning state; the corresponding outputs may include: state representation within a future preset time range, whole-body thermal comfort score within a future preset time range, local thermal comfort score within a future preset time range, air conditioning energy consumption estimate, probability of user manual intervention, probability of user readjustment, defogging speed exceeding preset speed, temperature change value exceeding preset temperature value, and adjustment action amplitude exceeding preset amplitude. The constructed action-conditional world model can be represented as: p(z_{t+1:t+H},c_{t+1:t+H},e_{t+1:t+H},o_{t+1:t+H} | z_t,a_{t:t+H-1}), where z represents the state representation, c represents the thermal comfort state, e represents energy consumption, o represents the probability of user intervention, a represents the air conditioning action, and t and H represent time. The air conditioning operation may include at least one of the following: set temperature, outlet air temperature, air volume, outlet air direction, face blowing mode, foot blowing mode, defogging mode, internal and external circulation status, and compressor and heat pump operating status.Therefore, this application does not directly regard the user's active adjustment action as the optimal control label, but predicts the future results under different candidate actions through the action conditional world model, so as to make more reasonable air conditioning action decisions.

[0041] In an exemplary embodiment of this application, the vehicle air conditioning control method may further include: designating a motion conditional world model trained using all modal data as a first model, and designating a motion conditional world model trained using partial modal data as a second model; using the second model to learn the distillation objective of the first model, and using the learned second model to determine the state representation at the current moment; wherein, the distillation objective includes at least one of: shared state representation, current thermal comfort prediction distribution, future thermal comfort prediction distribution, future state representation, latent state, and uncertainty estimation. In some examples, the first model may also be called the teacher model, and the second model may also be called the student model. During the training phase, the teacher model can use all available modalities, including in-vehicle RGB image data, in-vehicle infrared image data, vehicle CAN bus data, vehicle external weather data, user human static attribute data, cabin thermal field data, cabin temperature sensor deployment point temperature, and historical questionnaire information, etc. During the deployment phase, the student model only uses modal data that can be stably obtained by the vehicle, such as in-vehicle RGB image data, in-vehicle infrared image data, vehicle CAN bus data, weather data, and other modal data that can be stably obtained by the vehicle. For modal data such as human height, weight, clothing, and cabin temperature sensor deployment points that cannot be stably obtained during the deployment phase, these are not mandatory inputs to the student model. Instead, a distillation process is used to enable the student model to learn the hidden states and predictive capabilities of the teacher model. The distillation objectives include at least one of the following: the teacher model's shared latent state representation, the teacher model's current thermal comfort prediction distribution, the teacher model's future thermal comfort prediction distribution, the teacher model's future latent state representation, the teacher model's hidden state, and the teacher model's uncertainty estimate. During student model training, modal dropout can be further performed, randomly occluding or deleting certain input modalities, allowing the student model to adapt to situations such as missing cameras, incomplete CAN bus data, missing weather data, time jitter, and infrared anomalies during the training phase. Therefore, this application's training and deployment lacks a modality adaptation mechanism. Specifically, it uses all available modalities during training and only the actual, stably obtainable modal data of the vehicle during deployment. This addresses the problem of abundant information during training but limited information during deployment, enabling the action conditional world model to maintain good thermal comfort prediction performance in mass production deployment scenarios. Furthermore, this application employs a distillation learning mechanism between the teacher and student models. This mechanism utilizes all information from the training phase to improve the teacher model's representation quality while ensuring that the student model can operate solely on the actual, obtainable modal data of the vehicle during deployment.

[0042] In an exemplary embodiment of this application, step S140, which determines the air conditioning control action based on the action condition world model and preset constraints, includes: obtaining a candidate air conditioning action sequence, which is predicted by the action condition world model and represents the air conditioning action within a preset time range in the future; determining an optimization objective based on the preset constraints, and determining the air conditioning control action from the candidate air conditioning action sequence based on the optimization objective; wherein the preset constraints include at least one of the following: upper and lower limits of temperature, upper and lower limits of fan speed, range of air outlet direction selection, action change rate limit, defogging rule, temperature change rate, air conditioning actuator constraint, user explicit constraint, user explicit preference constraint, and uncertainty backoff strategy. In some examples, based on the action condition world model, this application adopts a model predictive control method to generate a candidate action sequence in each control cycle, and evaluate the thermal comfort, energy consumption, and risk of each air conditioning action in the candidate action sequence within a preset time range in the future, and select the optimal air conditioning action from them. The optimization objective can be expressed as: J = α*C_comfort + β*C_energy + γ*C_smooth + δ*C_override + η*C_safety, where C_comfort represents the thermal comfort deviation cost, C_energy represents the air conditioning energy consumption cost, C_smooth represents the smoothness cost of action change, C_override represents the user manual takeover probability cost, and C_safety represents the safety constraint violation cost. α, β, γ, δ, and η are the corresponding weighting coefficients, and α + β + γ + δ + η = 1. C_comfort, C_energy, C_smooth, C_override, and C_safety are all determined based on constraints, which include at least one of the following: setting upper and lower temperature limits, upper and lower limits for fan speed, selectable range of air outlet direction, action change rate limit, defogging safety priority rule, temperature change rate limit, automotive-grade air conditioning actuator constraints, explicit user restrictions or preference constraints, and a fallback strategy under high uncertainty conditions. When the uncertainty predicted by the action condition world model exceeds a preset threshold, aggressive control actions may not be executed; instead, the system may revert to a rule-based air conditioning strategy, the original vehicle air conditioning strategy, or a conservative recommended action strategy. Therefore, this application employs a constrained model predictive control mechanism that can simultaneously consider constraints such as comfort, energy consumption, action smoothness, user takeover probability, and defogging safety, thereby improving the feasibility of deploying the action condition world model in vehicles.

[0043] In another exemplary embodiment of this application, such as Figure 2 As shown, a vehicle air conditioning control method is provided, including the following steps: Multimodal data from within the vehicle cabin is collected and transformed to a unified timeline. This multimodal data includes, but is not limited to: in-vehicle RGB image data, in-vehicle infrared image data, vehicle CAN bus data, vehicle interior environmental data, vehicle exterior environmental data, vehicle exterior weather data, user static human attribute data, cabin temperature sensor placement point temperatures, cabin thermal field data, user thermal comfort score data, and user-initiated air conditioning adjustment data. Vehicle controller area network (VLAN) bus data includes at least one of the following: vehicle speed, air conditioning set temperature, actual air outlet temperature, blower motor setting, air conditioning internal / external circulation status, compressor status, outside temperature, sunlight intensity, door status, and seat status. User thermal comfort score data includes at least one of the following: full-body thermal comfort score, head thermal comfort score, back thermal comfort score, and foot thermal comfort score. In some examples, the user thermal comfort score data can be in the form of discrete levels, such as -3 to +3, or continuous values ​​or other comfort evaluation scales. The process of transforming multimodal data to a unified timeline can be as follows: using a unified timestamp as the primary index, constructing a canonical timeline (standard timeline). For each time anchor point, the corresponding visual segments, vehicle CAN bus data sequences, vehicle interior and exterior environment data sequences, weather data sequences, user human static attribute data, and cabin temperature sensor deployment point temperature or cabin thermal field data can be extracted based on the historical time window. Modal missing markers and time interval features are generated to handle common problems in multimodal data acquisition such as asynchronous sampling, data missing, local frame loss, incomplete modality, and time jitter.

[0044] Different types of modal data are structured and encoded to generate image modal tokens, structured temporal tokens, static human body tokens, thermal field tokens, and motion tokens. In some examples, RGB or IR (infrared) video can be encoded using a visual encoder to form image modal tokens. Vehicle CAN bus data, vehicle interior environment data, vehicle exterior environment data, and weather data can be encoded using a structured temporal encoder to form structured temporal tokens. User static human attribute data, such as age, height, weight, and clothing, can be encoded using a static covariate encoder to form static human body tokens. Temperature data from cabin temperature sensor deployment points and cabin thermal field data can be encoded using a thermal field encoder to form thermal field tokens.

[0045] Each modal token is encoded using multiple modal encoders to obtain the intermediate representation of each modality; The intermediate representations of each modality are decomposed into shared latent representations and private latent representations. Shared latent representations are used for cross-modal representation of multimodal data, while private latent representations preserve the unique information of each modality. In some examples, shared latent representations can be used to represent the cabin thermal environment, user thermal state, air conditioning operating status, and thermal comfort-related states that are commonly reflected across modalities, while private latent representations can be used to represent information unique to each modality, such as posture and clothing appearance information in in-vehicle RGB image data, local thermal distribution information in in-vehicle infrared image data, and control status information in vehicle CAN bus data.

[0046] Based on shared state representation and private state representation, a current thermal comfort prediction head and a future thermal comfort prediction head are trained using sparse thermal comfort labels. The current comfort prediction head is used to regress the prediction results of the current action condition world model, and the future thermal comfort prediction head is used to regress the prediction results of the action condition world model within a future preset time range.

[0047] A teacher model is constructed to learn the complete thermal comfort state using full-modal data available during the training phase. During training, the teacher model can use all available modalities, including in-vehicle RGB image data, in-vehicle infrared image data, vehicle CAN bus data, external weather data, user static human attribute data, cabin thermal field data, cabin temperature sensor deployment point temperatures, and historical questionnaire information.

[0048] A student model is constructed that utilizes only stable modal data obtainable during the deployment phase, and learns the shared state representation, prediction distribution, and future hidden states of the teacher model through distillation. During deployment, the student model uses only stable modal data obtainable from the vehicle, such as in-vehicle RGB image data, in-vehicle infrared image data, vehicle CAN bus data, and weather data. Modal data that is not stably obtainable during deployment, such as human height, weight, clothing, and cabin temperature sensor placement point temperatures, are not forced as input to the student model. Instead, distillation is used to enable the student model to learn the hidden states and predictive capabilities of the teacher model.

[0049] A motion-conditional world model is constructed, and the output of the student model is used as the input of the motion-conditional world model. The motion-conditional world model is used to predict the cabin state, thermal comfort state, energy consumption state, user intervention probability and air conditioning action within a future preset time range under different candidate air conditioning action sequences. In some examples, when constructing the action-conditional world model, the corresponding inputs may include: current multimodal state representation, historical air conditioning actions, air conditioning action sequence, external environment state, weather forecast state, current vehicle state, and air conditioning state; the corresponding outputs may include: state representation within a future preset time range, whole-body thermal comfort score within a future preset time range, local thermal comfort score within a future preset time range, air conditioning energy consumption estimate, probability of user manual intervention, probability of user readjustment, defogging speed exceeding preset speed, temperature change value exceeding preset temperature value, and adjustment action amplitude exceeding preset amplitude. The constructed action-conditional world model can be represented as: p(z_{t+1:t+H},c_{t+1:t+H},e_{t+1:t+H},o_{t+1:t+H} | z_t,a_{t:t+H-1}), where z represents the state representation, c represents the thermal comfort state, e represents energy consumption, o represents the probability of user intervention, and a represents the air conditioning action. The air conditioning operation may include at least one of the following: set temperature, outlet air temperature, air volume, outlet air direction, face blowing mode, foot blowing mode, defogging mode, internal and external circulation status, and compressor and heat pump operating status.

[0050] Based on the action-conditional world model, a model predictive control method with constraints is used to select the optimal air conditioning action. In some examples, based on the action-conditional world model and the model predictive control method, candidate action sequences are generated in each control cycle, and the thermal comfort, energy consumption, and risk of each air conditioning action in the candidate action sequence are evaluated within a future preset time range, from which the optimal air conditioning action is selected. The optimization objective can be expressed as: J = α*C_comfort + β*C_energy + γ*C_smooth + δ*C_override + η*C_safety, where C_comfort represents the thermal comfort deviation cost, C_energy represents the air conditioning energy consumption cost, C_smooth represents the action change smoothness cost, C_override represents the user manual takeover probability cost, C_safety represents the safety constraint violation cost, and α, β, γ, δ, and η are the corresponding weighting coefficients, with α + β + γ + δ + η = 1. Among them, C_comfort, C_energy, C_smooth, C_override, and C_safety are all determined based on constraints, which include at least one of the following: setting upper and lower limits for temperature, upper and lower limits for fan speed, selectable range for air outlet direction, limit on rate of change of action, priority rule for defogging safety, limit on rate of temperature change, constraints on automotive-grade air conditioning actuators, explicit user restrictions or preference constraints, and fallback strategy under high uncertainty. When the uncertainty predicted by the world model of action conditions exceeds a preset threshold, aggressive control actions may not be executed, and instead, the system may fall back to the rule-based air conditioning strategy, the original vehicle air conditioning strategy, or the conservative action recommendation strategy.

[0051] The optimal air conditioning action is output to the vehicle air conditioning execution system to control at least one of the following: air outlet temperature, air outlet direction, air volume, and air conditioning mode.

[0052] In summary, this application proposes an in-vehicle air conditioning control method. It acquires and encodes multimodal data to obtain corresponding lexical units, and determines intermediate representations of the multimodal data based on these lexical units. Then, it forms an action conditional world model based on these intermediate representations. This model predicts at least one of the following within a preset time range: cabin state, thermal comfort state, energy consumption state, user intervention probability, and air conditioning action. Finally, it determines the air conditioning control action based on the action conditional world model and preset constraints, and controls the in-vehicle air conditioning according to the determined action. Therefore, this method, by utilizing multimodal data for in-vehicle air conditioning control, can more completely represent the vehicle cabin thermal environment and user thermal comfort state, supporting individualized thermal comfort control. Furthermore, this method employs a constrained model predictive control mechanism, which can simultaneously consider constraints such as comfort, energy consumption, action smoothness, user takeover probability, and defogging safety, explicitly satisfying vehicle air conditioning control safety constraints and improving the feasibility of deploying the action conditional world model in vehicles. Furthermore, this method does not directly treat user-initiated adjustments as the optimal control label. Instead, it predicts future outcomes under different candidate actions using an action-conditional world model, thus making more reasonable air conditioning action decisions. Simultaneously, this method addresses issues such as inconsistent sampling frequencies across different modalities, missing data, and time jitter by unifying the time axis. Moreover, by decomposing intermediate representations of each modality, aligning across modalities, predicting future representations, and implementing sparse comfort supervision, this method enables the model to focus on learning states relevant to thermal comfort prediction and air conditioning control, rather than simply reconstructing all original inputs. Furthermore, through a distillation learning mechanism between the teacher and student models, it utilizes all information from the training phase to improve the teacher model's representation quality, while ensuring that the student model only relies on available modal data from the vehicle during deployment. In addition, this method can combine historical user adjustment behaviors, limited feedback, and user embeddings to achieve individualized comfort prediction and air conditioning control for different users.

[0053] In another exemplary embodiment of this application, such as Figure 3 As shown, a vehicle air conditioning control system is provided, including: The data acquisition module 310 is used to acquire multimodal data, which is determined based on the target vehicle and the user using the vehicle air conditioner in the target vehicle. The encoding module 320 is used to encode multimodal data to obtain the corresponding word units of the multimodal data, and to determine the intermediate representation of the multimodal data based on the corresponding word units of the multimodal data. The model building module 330 is used to form an action condition world model based on intermediate representations. The action condition world model is used to predict at least one of the following within a future preset time range: cabin state, thermal comfort state, energy consumption state, user intervention probability, and air conditioning action. The vehicle air conditioning control module 340 is used to determine the air conditioning control action based on the action condition world model and preset constraints, and to control the vehicle air conditioning according to the determined air conditioning control action.

[0054] It is understood that the vehicle air conditioning control system provided in the above embodiments and the vehicle air conditioning control method provided in the above embodiments belong to the same concept. The specific way in which the vehicle air conditioning control method performs operations has been described in detail in the above method embodiments and will not be repeated here. In practical applications, the vehicle air conditioning control system provided in the above embodiments can allocate the above functions to different functional modules as needed. That is, the internal structure of the vehicle air conditioning control system is divided into different functional modules, and then all or part of the functions of the corresponding functional modules are implemented by the vehicle air conditioning control method described in the above embodiments. For example, all or part of the functions of the data acquisition module 310 can be implemented through the relevant execution process of step S110, all or part of the functions of the encoding module 320 can be implemented through the relevant execution process of step S120, all or part of the functions of the model building module 330 can be implemented through the relevant execution process of step S130, and all or part of the functions of the vehicle air conditioning control module 340 can be implemented through the relevant execution process of step S140. No specific limitations are imposed here.

[0055] Therefore, this application proposes an in-vehicle air conditioning control system. It acquires and encodes multimodal data to obtain corresponding terms, and determines intermediate representations of the multimodal data based on these terms. Then, it forms an action conditional world model based on these intermediate representations. This model predicts at least one of the following within a preset time range: cabin state, thermal comfort state, energy consumption state, user intervention probability, and air conditioning action. Finally, it determines the air conditioning control action based on the action conditional world model and preset constraints, and controls the in-vehicle air conditioning according to the determined action. This system, by utilizing multimodal data for in-vehicle air conditioning control, can more completely represent the vehicle cabin thermal environment and user thermal comfort state, supporting individualized thermal comfort control. Simultaneously, the system employs a constrained model predictive control mechanism, which can simultaneously consider constraints such as comfort, energy consumption, action smoothness, user takeover probability, and defogging safety. This explicitly satisfies vehicle air conditioning control safety constraints, improving the feasibility of deploying the action conditional world model in vehicles. Furthermore, this system does not directly treat user-initiated adjustments as the optimal control label. Instead, it predicts future outcomes under different candidate actions using an action-conditional world model, thus making more reasonable air conditioning action decisions. Simultaneously, this system addresses issues such as inconsistent sampling frequencies across different modalities, data gaps, and time jitter by unifying the time axis. Moreover, by decomposing intermediate representations of each modality, aligning them across modalities, predicting future representations, and implementing sparse comfort supervision, the system ensures that the model focuses on learning states relevant to thermal comfort prediction and air conditioning control, rather than simply reconstructing all original inputs. Furthermore, through a distillation learning mechanism between the teacher and student models, the system utilizes all information from the training phase to improve the teacher model's representation quality, while ensuring that the student model only relies on available modal data from the vehicle during deployment. In addition, this system can combine historical user adjustment behaviors, limited feedback, and user embeddings to achieve individualized comfort prediction and air conditioning control for different users.

[0056] In an exemplary embodiment of this application, a computer device is also provided. The computer device may include a memory, a processor, and a computer program stored in the memory. The processor can execute the computer program to cause the computer device to perform actions such as... Figure 1 The steps of the vehicle air conditioning control method are shown. Figure 4 A schematic diagram of the structure of a computer device 1000 is shown. (See attached diagram.) Figure 4 As shown, the computer device 1000 includes: a processor 1010, a memory 1020, a power supply 1030, a display unit 1040, and an input unit 1060.

[0057] The processor 1010 is the control center of the computer device 1000. It connects various components via interfaces and lines, and performs various functions of the computer device 1000 by running or executing computer programs / instructions stored in the memory 1020, thereby providing overall monitoring of the computer device 1000. In some embodiments, when the processor 1010 calls a computer program stored in the memory 1020, it can execute, for example... Figure 1 The steps of the vehicle air conditioning control method are shown. Optionally, the processor 1010 may include one or more processing units; preferably, the processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. In some embodiments, the processor 1010 and the memory 1020 may be implemented on a single chip; in other embodiments, they may be implemented on separate chips.

[0058] The memory 1020 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, various applications, etc.; the data storage area can store instruction data created according to the use of the computer device 1000. In addition, the memory 1020 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0059] The computer device 1000 also includes a power supply 1030 (such as a battery) that supplies power to various components. The power supply can be logically connected to the processor 1010 through a power management system, thereby enabling the management of functions such as charging, discharging, and power consumption through the power management system.

[0060] The display unit 1040 can be used to display information input by the user or information provided to the user, and can also be used to display various menus of the computer device 1000, etc. In this embodiment, it is mainly used to display the display interface of each application in the computer device 1000, as well as text, pictures, and other objects displayed in the display interface. The display unit 1040 may include a display panel 1050. The display panel 1050 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0061] The input unit 1060 can be used to receive information such as numbers or characters input by the user. The input unit 1060 may include a touch panel 1070 and other input devices 1080. The touch panel 1070 can also be referred to as a touch screen, and the touch panel 1070 can collect touch operations on or near the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1070).

[0062] Specifically, the touch panel 1070 can detect user touch operations and the signals generated by these operations, convert these signals into touch point coordinates and send them to the processor 1010, and receive and execute commands transmitted by the processor 1010. Furthermore, the touch panel 1070 can employ various input methods such as resistive, capacitive, infrared, and surface acoustic waves to achieve interaction. Other input devices 1080 include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, and joystick.

[0063] Of course, the touch panel 1070 can also cover the display panel 1050. When the touch panel 1070 detects a touch operation on or near it, it can transmit the information to the processor 1010 to determine the type of touch event. Subsequently, the processor 1010 provides corresponding visual output on the display panel 1050 based on the type of touch event. Although in Figure 4 In this embodiment, the touch panel 1070 and the display panel 1050 are two separate components to realize the input and output functions of the computer device 1000. However, in some embodiments, the touch panel 1070 and the display panel 1050 can be integrated to realize the input and output functions of the computer device 1000.

[0064] The computer device 1000 may also include one or more sensors, such as pressure sensors, gravity acceleration sensors, proximity sensors, etc. Of course, depending on the specific application scenario, the computer device 1000 may also include other components such as cameras.

[0065] In an exemplary embodiment of this application, a computer-readable storage medium is also provided, which stores a computer program / instructions. When executed by a processor, the computer program / instructions enable the aforementioned computer device to perform the functions described in this application. Figure 1 The steps of the vehicle air conditioning control method are shown.

[0066] It will be understood by those skilled in the art that Figure 4This is merely an example of a computer device and does not constitute a limitation on the device. The device may include more or fewer components than illustrated, or a combination of certain components, or different components. For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components. For example, as some examples, the aforementioned computer device may be a vehicle, an in-vehicle system, etc.

[0067] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for controlling a vehicle air conditioner, characterized in that, The method includes the following steps: Acquire multimodal data, which is determined based on the target vehicle and the user using the vehicle's air conditioning. The multimodal data is encoded to obtain the corresponding lexical units, and the intermediate representation of the multimodal data is determined based on the corresponding lexical units. Based on the intermediate representation, an action condition world model is formed. The action condition world model is used to predict at least one of the following within a future preset time range: cabin state, thermal comfort state, energy consumption state, user intervention probability, and air conditioning action. The air conditioning control action is determined based on the action condition world model and preset constraints, and the vehicle air conditioning is controlled according to the determined air conditioning control action.

2. The vehicle air conditioning control method according to claim 1, characterized in that, The process of encoding the multimodal data to obtain the corresponding lexical units, and determining the intermediate representation of the multimodal data based on the corresponding lexical units, includes: The multimodal data is structured and encoded according to its data type to generate image modal lexical units, structured temporal lexical units, human static lexical units, thermal field lexical units, and action lexical units. Image modal lexical units, structured temporal lexical units, human static lexical units, thermal field lexical units, and action lexical units are encoded to obtain intermediate representations of the image modal lexical units, the structured temporal lexical units, the human static lexical units, the thermal field lexical units, and the action lexical units, which are then used as intermediate representations of the multimodal data.

3. The vehicle air conditioning control method according to claim 1 or 2, characterized in that, The process of forming an action-conditional world model based on the intermediate representations includes: The intermediate representation of the multimodal data is decomposed into shared state representation and private state representation, and the shared state representation and the private state representation are optimized by a target loss; wherein, the shared state representation is used to perform cross-modal representation of the multimodal data, and the private state representation is used to retain the unique information of each modality; wherein, the target loss is obtained by weighting multiple losses; Based on optimized shared state representations and private state representations, the action condition world model and the comfort prediction head are formed; wherein, the comfort prediction head is used to perform regression output on the prediction results of the action condition world model.

4. The vehicle air conditioning control method according to claim 3, characterized in that, The process of forming an action-conditional world model based on the intermediate representations also includes: The action condition world model is constructed by taking the current state representation, historical air conditioning actions, all air conditioning action sequences, external environment state, weather forecast state, current vehicle state, and air conditioning state as inputs, and the state representation within a future preset time range, the whole body thermal comfort score within the future preset time range, the local thermal comfort score within the future preset time range, the air conditioning energy consumption estimate, the probability of user manual takeover, the probability of user readjustment, the defogging rate, the temperature change rate, and the adjustment action amplitude as outputs. The state representation includes shared state representation and / or private state representation. The cabin state within a future preset time range is represented by the state representation within the future preset time range. The thermal comfort state within the future preset time range is represented by the local thermal comfort score within the future preset time range. The energy consumption state is represented by the air conditioning energy consumption estimate. The probability of user intervention is represented by the probability of user manual takeover and the probability of user readjustment. The air conditioning action is represented by the defogging rate, the temperature change rate, and the adjustment action amplitude.

5. The vehicle air conditioning control method according to claim 4, characterized in that, The method further includes: The action-conditional world model trained with all modal data is denoted as the first model, and the action-conditional world model trained with some modal data is denoted as the second model. The distillation objective of the first model is learned using the second model, and the state representation at the current moment is determined using the learned second model; wherein, the distillation objective includes at least one of: shared state representation, current thermal comfort prediction distribution, future thermal comfort prediction distribution, future state representation, latent state, and uncertainty estimation.

6. The vehicle air conditioning control method according to claim 1, 2, or 4, characterized in that, The process of determining the air conditioning control action based on the aforementioned action condition world model and preset constraints includes: Obtain candidate air conditioner action sequences, which are predicted by the action condition world model and represent air conditioner actions within a preset time range in the future; The optimization objective is determined based on the preset constraints, and the air conditioning control action is determined from the candidate air conditioning action sequence based on the optimization objective; wherein, the preset constraints include at least one of the following: upper and lower limits of temperature, upper and lower limits of air volume, range of air outlet direction selection, action change rate limit, defogging rule, temperature change rate, air conditioning actuator constraint, user explicit constraint, user explicit preference constraint, and uncertainty backoff strategy.

7. The vehicle air conditioning control method according to claim 1, characterized in that, The multimodal data includes vehicle interior image data, vehicle controller local area network bus data, vehicle interior environment data, vehicle exterior environment data, vehicle exterior weather data, user static human body attribute data, cabin temperature sensor deployment point temperature, cabin thermal field data, user thermal comfort score data, and user active air conditioning adjustment action data. The vehicle controller local area network bus data includes at least one of the following: vehicle speed, air conditioning set temperature, actual air outlet temperature, blower speed, air conditioning internal and external circulation status, compressor status, outside temperature, sunlight intensity, door status, and seat status. The user thermal comfort score data includes at least one of the following: whole body thermal comfort score, head thermal comfort score, back thermal comfort score, and foot thermal comfort score.

8. A vehicle air conditioning control system, characterized in that, The system includes: A data acquisition module is used to acquire multimodal data, which is determined based on the target vehicle and the user using the vehicle's air conditioning. An encoding module is used to encode the multimodal data to obtain the lexical units corresponding to the multimodal data, and to determine the intermediate representation of the multimodal data based on the lexical units corresponding to the multimodal data. The model building module is used to form an action condition world model based on the intermediate representation. The action condition world model is used to predict at least one of the following within a future preset time range: cabin state, thermal comfort state, energy consumption state, user intervention probability, and air conditioning action. The vehicle air conditioning control module is used to determine the air conditioning control action based on the action condition world model and preset constraints, and to control the vehicle air conditioning according to the determined air conditioning control action.

9. A computer device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the vehicle air conditioning control method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the vehicle air conditioning control method according to any one of claims 1 to 7.

Images