Control method and device of vehicle cabin, vehicle and storage medium

Abstract

This application relates to the field of intelligent cockpit technology, and particularly to a control method, device, vehicle, and storage medium for a vehicle cockpit. The method includes: upon visual perception and location identification that the vehicle's external environment belongs to a preset natural aesthetic scene, extracting features such as color, texture, target objects, and emotional semantics to generate an initial aesthetic feature vector; converting this vector into stylized configuration parameters conforming to a preset theme specification according to a theme mapping rule library; and then generating multimodal linkage control commands to control in-cabin equipment (such as lights, audio, air conditioning, screens, etc.) to operate according to the theme configuration, achieving intelligent coordination and immersive experience between the vehicle's internal and external environments. This solves the problem that related technologies cannot perceive and understand the aesthetic features of the external environment and struggle to cope with infinitely rich real-world scenes, achieving deep linkage and emotional integration between the vehicle cockpit and the external natural environment, thus enhancing the user experience.

Description

Technical Field

[0001] This application relates to the field of smart cockpit technology, and in particular to a control method, device, vehicle, and storage medium for a vehicle cockpit. Background Technology

[0002] With the rapid development of intelligent cockpit technology, users' demands for in-vehicle experience have evolved from simply fulfilling functional needs to becoming more emotional, personalized, and immersive. Vehicles are no longer seen as mere transportation tools, but rather as offering a more intelligent, comfortable, and emotionally resonant cockpit experience based on the external environment and the user's state. Under this trend, achieving deep interaction between vehicles and the external natural environment has become an important research direction in the field of intelligent cockpits.

[0003] Among related technologies, some intelligent cockpit systems already possess a certain degree of environmental adaptability. For example, they can automatically switch ambient light colors, adjust air conditioning modes, or change the theme of the human-computer interaction interface based on time, geographical location, or weather information.

[0004] However, this method can only respond based on coarse-grained information such as geographical location or time, and cannot perceive or understand the aesthetic features of the external environment. It is difficult to cope with infinitely rich real-world scenarios and urgently needs to be solved. Summary of the Invention

[0005] This application provides a vehicle cockpit control method, device, vehicle, and storage medium to solve the problem that related technologies cannot perceive and understand the aesthetic features of the external environment and are difficult to cope with infinitely rich real-world scenarios, thereby achieving deep linkage and emotional integration between the vehicle cockpit and the external natural environment and improving the user experience.

[0006] To achieve the above objectives, the first aspect of this application proposes a vehicle cockpit control method, comprising the following steps: The system acquires visual and geographic information of the vehicle's external environment and determines whether the vehicle's external environment belongs to a preset natural aesthetic scene category based on the visual and geographic information. When the vehicle's external environment belongs to the preset natural aesthetic scene category, color features, texture features, target object features, and emotional semantic features are extracted from the visual information, and an initial aesthetic feature vector is generated based on the color features, texture features, target object features, and emotional semantic features. According to the preset theme mapping rule library, the initial aesthetic feature vector is mapped to stylized configuration parameters that conform to the preset theme specifications, and the target theme configuration set is determined according to the stylized configuration parameters; Based on the target theme configuration set, multimodal linkage control instructions are generated, and at least one target device in the vehicle cabin is controlled to operate according to the parameters in the target theme configuration set according to the multimodal linkage control instructions.

[0007] According to one embodiment of this application, determining whether the vehicle's external environment belongs to a preset natural aesthetic scene category based on the visual information and the geographical location information includes: The geographic location information is matched with a preset scene geographic tag library to obtain a first confidence level; The visual information is input into the scene classification model to obtain the second confidence level; If the first confidence level is greater than the first preset threshold and the second confidence level is greater than the second preset threshold, the vehicle's external environment is determined to belong to the preset natural aesthetic scene category.

[0008] According to one embodiment of this application, the extraction of color features, texture features, target object features, and emotional semantic features from the visual information includes: The visual information is segmented using a primary color extraction algorithm to obtain a preset number of key color clusters and the color value and weight of each key color cluster, and the key color clusters and the color value and weight of each key color cluster are used as the color features. The visual information is input into a texture recognition model to identify the texture category in the visual information, and the texture category is used as the texture feature. The visual information is input into the target detection model to identify the target object in the visual information; The visual information is input into the sentiment classification model to obtain sentiment type labels and their confidence scores. The sentiment type labels and their confidence scores are then used as the sentiment semantic features.

[0009] According to one embodiment of this application, generating an initial aesthetic feature vector based on the color features, the texture features, the target object, and the emotional semantic features includes: The color features are converted into color feature sub-vectors, which include color value components and weight components of at least one key color cluster. The texture features are converted into texture feature sub-vectors, the texture feature sub-vectors including the encoded representation of the texture category; The target object is converted into an object feature vector, which contains an encoded representation of the object category; The emotional semantic features are converted into emotional feature sub-vectors, which include the encoded representation of the emotional type label and the confidence component of the emotional type label; The initial aesthetic feature vector is generated by concatenating the color feature sub-vector, the texture feature sub-vector, the object feature sub-vector, and the emotion feature sub-vector.

[0010] According to one embodiment of this application, the preset theme mapping rule library includes color mapping rules, shape mapping rules, and linkage mapping rules. Mapping the initial aesthetic feature vector to stylized configuration parameters conforming to the preset theme specifications includes: According to the color mapping rules, the color features in the initial aesthetic feature vector are mapped to the target color values ​​in the preset color wheel; According to the morphological mapping rule, the texture features and target objects in the initial aesthetic feature vector are mapped to preset design elements; According to the linkage mapping rule, the emotional semantic features in the initial aesthetic feature vector are mapped to the operating parameters of multiple target devices.

[0011] According to the vehicle cockpit control method proposed in this application, when the external environment of the vehicle belongs to a preset natural aesthetic scene (such as natural scenery) through visual perception and positioning identification, features such as color, texture, target objects, and emotional semantics are extracted to generate an initial aesthetic feature vector. This vector is then converted into stylized configuration parameters conforming to the preset theme specifications according to a theme mapping rule library. This generates multimodal linkage control commands to control the operation of in-cabin equipment (such as lights, audio, air conditioning, screens, etc.) according to the theme configuration, achieving intelligent coordination and immersive experience between the vehicle's internal and external environments. This solves the problem that related technologies cannot perceive and understand the aesthetic features of the external environment and struggle to cope with infinitely rich real-world scenes, achieving deep linkage and emotional integration between the vehicle cockpit and the external natural environment, thus enhancing the user experience.

[0012] To achieve the above objectives, a second aspect of this application provides a vehicle cockpit control device, comprising: The judgment module is used to acquire visual information and geographical location information of the vehicle's external environment, and based on the visual information and geographical location information, to determine whether the vehicle's external environment belongs to a preset natural aesthetic scene category; The generation module is used to extract color features, texture features, target object and emotional semantic features from the visual information when the vehicle's external environment belongs to the preset natural aesthetic scene category, and generate an initial aesthetic feature vector based on the color features, texture features, target object and emotional semantic features; The determination module is used to map the initial aesthetic feature vector into stylized configuration parameters that conform to the preset theme specifications according to the preset theme mapping rule library, and to determine the target theme configuration set according to the stylized configuration parameters; The control module is used to generate multimodal linkage control commands based on the target theme configuration set, and control at least one target device in the vehicle cabin to operate according to the parameters in the target theme configuration set according to the multimodal linkage control commands.

[0013] According to one embodiment of this application, the determining module is specifically used for: The geographic location information is matched with a preset scene geographic tag library to obtain a first confidence level; The visual information is input into the scene classification model to obtain the second confidence level; If the first confidence level is greater than the first preset threshold and the second confidence level is greater than the second preset threshold, the vehicle's external environment is determined to belong to the preset natural aesthetic scene category.

[0014] According to one embodiment of this application, the step of extracting color features, texture features, target object features, and emotional semantic features from the visual information, wherein the generation module is specifically used for: The visual information is segmented using a primary color extraction algorithm to obtain a preset number of key color clusters and the color value and weight of each key color cluster, and the key color clusters and the color value and weight of each key color cluster are used as the color features. The visual information is input into a texture recognition model to identify the texture category in the visual information, and the texture category is used as the texture feature. The visual information is input into the target detection model to identify the target object in the visual information; The visual information is input into the sentiment classification model to obtain sentiment type labels and their confidence scores. The sentiment type labels and their confidence scores are then used as the sentiment semantic features.

[0015] According to one embodiment of this application, the generation module, which generates an initial aesthetic feature vector based on the color features, the texture features, the target object, and the emotional semantic features, is specifically used for: The color features are converted into color feature sub-vectors, which include color value components and weight components of at least one key color cluster. The texture features are converted into texture feature sub-vectors, the texture feature sub-vectors including the encoded representation of the texture category; The target object is converted into an object feature vector, which contains an encoded representation of the object category; The emotional semantic features are converted into emotional feature sub-vectors, which include the encoded representation of the emotional type label and the confidence component of the emotional type label; The initial aesthetic feature vector is generated by concatenating the color feature sub-vector, the texture feature sub-vector, the object feature sub-vector, and the emotion feature sub-vector.

[0016] According to one embodiment of this application, the determining module is specifically used for: According to the color mapping rules, the color features in the initial aesthetic feature vector are mapped to the target color values ​​in the preset color wheel; According to the morphological mapping rule, the texture features and target objects in the initial aesthetic feature vector are mapped to preset design elements; According to the linkage mapping rule, the emotional semantic features in the initial aesthetic feature vector are mapped to the operating parameters of multiple target devices.

[0017] According to the vehicle cockpit control device proposed in this application, when the external environment of the vehicle belongs to a preset natural aesthetic scene (such as natural scenery) through visual perception and positioning recognition, it extracts features such as color, texture, target objects, and emotional semantics to generate an initial aesthetic feature vector. This vector is then converted into stylized configuration parameters conforming to the preset theme specifications according to a theme mapping rule library. This generates multimodal linkage control commands to control the operation of in-cabin equipment (such as lights, audio, air conditioning, screens, etc.) according to the theme configuration, achieving intelligent coordination and immersive experience between the interior and exterior environments. This solves the problem that related technologies cannot perceive and understand the aesthetic features of the external environment and struggle to cope with infinitely rich real-world scenes, achieving deep linkage and emotional integration between the vehicle cockpit and the external natural environment, thus enhancing the user experience.

[0018] To achieve the above objectives, a third aspect of this application provides a vehicle comprising: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the vehicle cockpit control method as described in the above embodiments.

[0019] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the vehicle cockpit control method as described in the above embodiments.

[0020] To achieve the above objectives, a fifth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, is used to implement the vehicle cockpit control method as described in the above embodiments.

[0021] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0022] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a vehicle cockpit control method according to an embodiment of this application; Figure 2 This is a flowchart of another vehicle cockpit control method provided according to an embodiment of this application; Figure 3 This is a block diagram of a vehicle cockpit control device provided according to an embodiment of this application; Figure 4 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0023] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0024] The following description, with reference to the accompanying drawings, describes a vehicle cockpit control method, apparatus, vehicle, and storage medium according to embodiments of this application. First, the vehicle cockpit control method according to embodiments of this application will be described with reference to the accompanying drawings.

[0025] Figure 1 This is a flowchart of a vehicle cockpit control method according to an embodiment of this application.

[0026] For example, such as Figure 1 As shown, the control method for the vehicle's cockpit includes the following steps: In step S101, visual information and geographic location information of the vehicle's external environment are obtained, and based on the visual information and geographic location information, it is determined whether the vehicle's external environment belongs to a preset natural aesthetic scene category.

[0027] It is understood that, in this embodiment, visual information refers to image or video data of the vehicle's external environment collected by in-vehicle cameras (such as front-facing cameras and surround-view cameras), used to identify visual elements such as color, texture, and objects in the scene. Geographical location information refers to the latitude and longitude coordinates of the vehicle's current location obtained through positioning devices such as GPS (Global Positioning System), used to assist in determining the geographical attributes of the scene. Preset natural aesthetic scene categories refer to natural scene types with high aesthetic value predefined by the system, such as cherry blossom forests, bamboo forests, seasides, maple leaf avenues, and snowfields, which can trigger immersive experience linkages in the cockpit.

[0028] In other words, the system can first obtain visual information about the vehicle's external environment through the vehicle's onboard camera, and at the same time obtain the vehicle's current geographical location information through the positioning device. Then, it can make a fusion judgment based on these two types of information to identify whether the vehicle has entered a natural scene with aesthetic value.

[0029] As one possible implementation, in some embodiments, determining whether the vehicle's external environment belongs to a preset natural aesthetic scene category based on visual information and geographic location information includes: matching geographic location information with a preset scene geographic tag library to obtain a first confidence level; inputting visual information into a scene classification model to obtain a second confidence level; and determining that the vehicle's external environment belongs to a preset natural aesthetic scene category if the first confidence level is greater than a first preset threshold and the second confidence level is greater than a second preset threshold.

[0030] It is understood that, in this embodiment, the preset scene geographic tag library refers to a geographic information database pre-established by the system, which stores geographic location area tags corresponding to various natural aesthetic scenes, such as GPS coordinate ranges or geofence data for scenes like cherry blossom forests, bamboo forests, and seaside, used to match the vehicle's current geographic location. The first confidence level is a quantitative value representing the degree of matching between the geographic location information and the preset scene geographic tag library, reflecting the credibility of the vehicle's current location belonging to a certain type of natural aesthetic scene; its value range can be 0 to 1. The scene classification model refers to a trained deep learning neural network model used to identify scene categories in images captured by the vehicle-mounted camera, outputting the scene type of the image and its confidence level, for example, identifying scene categories such as "cherry blossom forest," "seaside," and "bamboo forest." The second confidence level is the confidence score of the scene classification model for the image recognition result, reflecting the reliability of the model's judgment; its value range can be 0 to 1. Both the first and second preset thresholds can be pre-set by those skilled in the art, obtained through a limited number of experiments, or obtained through a limited number of computer simulations; no specific limitations are made here.

[0031] To determine whether a vehicle's external environment belongs to a preset natural aesthetic scene category based on visual and geographic information, a dual-determination mechanism using cross-validation can be employed. Specifically, the system first matches the vehicle's current geographic location with a preset scene geographic label library. For example, when the vehicle travels to a cherry blossom grove area, if the GPS coordinates fall within the preset geofence of that grove, the system calculates a first confidence level of 0.95. Simultaneously, the system inputs visual information collected by the vehicle's camera into a pre-trained scene classification model. This model analyzes and identifies the image content. If a large number of cherry blossom objects and typical features such as pink and white tones are identified in the image, the model outputs the scene category as "cherry blossom grove" and gives a second confidence level of 0.88. The system then compares the first confidence level with a first preset threshold (e.g., 0.8) and the second confidence level with a second preset threshold (e.g., 0.7). If the first confidence level (0.95) is greater than the first preset threshold (0.8), and the second confidence level (0.88) is greater than the second preset threshold (0.7), the system can determine that the current external environment of the vehicle belongs to the preset natural aesthetic scene category of "cherry blossom grove." This dual verification mechanism fully leverages the complementarity of geographic location information and visual information. It avoids misjudgments caused by positioning drift that may occur when relying solely on GPS signals, and compensates for recognition deviations that may occur due to factors such as light and occlusion when relying solely on visual recognition. This significantly improves the accuracy and robustness of scene recognition and provides a reliable triggering premise for subsequent immersive cabin experience linkages.

[0032] It should be noted that the scene classification model is used to identify whether the visual information captured by the vehicle-mounted camera belongs to a preset natural aesthetic scene category. In this embodiment, the model is a trained deep learning neural network model, and its specific settings are as follows: The input of the scene classification model can be an image of the vehicle's external environment captured by the vehicle-mounted camera. The image size is preprocessed to a preset resolution (e.g., 224×224 pixels) and normalized to scale the pixel values ​​to a preset range. The output of the scene classification model is the scene category label and its corresponding confidence score. The scene category label includes preset natural aesthetic scene categories, such as cherry blossom forest, bamboo forest, seaside, maple leaf avenue, snowfield, etc. Each input image corresponds to an output scene category label and its confidence score. The confidence score ranges from 0 to 1, indicating the credibility of the scene classification model's judgment. The scene classification model can adopt a convolutional neural network or a visual Transformer architecture, specifically ResNet (Residual Network), EfficientNet (High-Efficiency Neural Network), ViT (Vision Transformer), and other mainstream classification network structures. The model improves the accuracy of recognizing natural aesthetic scenes by pre-training on a large-scale dataset of natural scene images and fine-tuning on a labeled dataset for preset scene categories. The training process for the scene classification model includes: collecting image samples containing various natural aesthetic scenes, manually labeling the scene categories to construct a training dataset; inputting images from the training dataset into the model and calculating the loss value between the model output and the labeled labels; iteratively updating the model parameters using the backpropagation algorithm until the loss value converges. After training, the model is capable of outputting the scene category and confidence level for input images.

[0033] This application applies a scene classification model to the field of vehicle cockpit control. The model's input comes directly from real-time images captured by the vehicle's camera, and the output is used for cross-validation with the matching results of geographic location information to jointly determine whether to trigger the immersive cockpit experience. In this way, the scene classification model and the vehicle cockpit control scene are deeply integrated.

[0034] In step S102, if the vehicle's external environment belongs to a preset natural aesthetic scene category, color features, texture features, target object features, and emotional semantic features are extracted from visual information, and an initial aesthetic feature vector is generated based on the color features, texture features, target object features, and emotional semantic features.

[0035] It is understood that, in this embodiment, color features refer to color attributes extracted from visual information, including the primary colors in the scene and their weights. For example, in a cherry blossom forest, pinkish-white is the primary color, brown branches are the secondary color, and blue sky is the accent color. Key color clusters and their color values ​​and weights are obtained through a primary color extraction algorithm. Texture features refer to surface structure attributes extracted from visual information, reflecting the visual tactile feel or arrangement patterns of object surfaces in the scene. For example, the granular texture of snowflakes, the vertical lines of bamboo forests, and the flowing curves of ocean waves. Target objects refer to specific natural objects identified from visual information, such as cherry blossoms, maple leaves, bamboo, ocean waves, and snowflakes, which can be identified and located using a target detection model. Emotional semantic features refer to emotional attributes extracted from visual information, reflecting the emotional experience evoked by the scene, including emotional type labels (such as romantic, tranquil, spring, and vibrant) and their corresponding confidence scores.

[0036] Specifically, once the system confirms that the vehicle's external environment belongs to a preset natural aesthetic scene category—for example, confirming that it is currently in a "cherry blossom forest" scene—the system immediately extracts multi-dimensional aesthetic features from the visual information captured by the vehicle's cameras, including color features, texture features, target object features, and emotional semantic features. These extracted multi-dimensional aesthetic features are then concatenated to generate a unified multi-dimensional initial aesthetic feature vector. This vector quantitatively and completely characterizes the aesthetic attributes of the current natural scene, providing a structured input foundation for subsequent style mapping and multimodal interaction.

[0037] As one possible implementation, in some embodiments, extracting color features, texture features, target objects, and emotional semantic features from visual information includes: segmenting the visual information using a primary color extraction algorithm to obtain a preset number of key color clusters and the color value and weight of each key color cluster, and using the key color clusters and the color value and weight of each key color cluster as color features; inputting the visual information into a texture recognition model to identify the texture category in the visual information, and using the texture category as texture features; inputting the visual information into a target detection model to identify the target object in the visual information; and inputting the visual information into an emotion classification model to obtain emotion type labels and the confidence scores of the emotion type labels, and using the emotion type labels and the confidence scores of the emotion type labels as emotional semantic features.

[0038] It is understood that, in the embodiments of this application, the dominant color extraction algorithm refers to an image processing algorithm used to extract the dominant color from an image. It can identify several colors with the highest frequency or visual weight in an image through color quantization, cluster analysis, etc., and output key color clusters and their color values ​​and weights. A key color cluster refers to a representative set of colors identified from an image by the dominant color extraction algorithm. Each color cluster contains a dominant hue and its weight ratio in the image. For example, in a cherry blossom forest image, pinkish-white is used as the dominant color cluster, and its weight can be 60%. The texture recognition model is a trained deep learning neural network model used to identify texture categories in an image, and can output texture type labels such as "grainy," "vertical lines," and "flowing curves." The object detection model is also a trained deep learning neural network model used to identify specific objects in an image and locate their positions, and can output object category labels such as "cherry blossom," "maple leaf," and "bamboo." The sentiment classification model is also a trained deep learning neural network model used to analyze image content and output the category of emotional experience it evokes. It can output sentiment type labels (such as "romantic", "tranquil", "spring") and their confidence scores.

[0039] Specifically, extracting color features, texture features, target object features, and emotional semantic features from visual information can employ a multi-model collaborative extraction mechanism. Specifically, in color feature extraction, the system can utilize a primary color extraction algorithm to perform image segmentation and color clustering analysis on images captured by the vehicle-mounted camera. For example, processing an image of a cherry blossom grove, the algorithm can output three key color clusters: the primary cluster is pinkish-white (RGB values ​​255, 200, 220, weight 60%), the secondary cluster is brown (RGB values ​​160, 120, 80, weight 25%), and the accent cluster is light blue (RGB values ​​200, 220, 255, weight 15%). These key color clusters and their color values ​​and weights are used as color features. In texture feature extraction, the system can input visual information into a pre-trained texture recognition model. The model analyzes the surface structure patterns in the image and outputs a texture category of "fine graininess" as a texture feature. In terms of object recognition, the system can input visual information into an object detection model. The model identifies numerous "cherry blossom" objects in the image and may also detect related objects such as "tree branches" and "sky," which are then used as target objects. For sentiment semantic feature extraction, the system can input visual information into a sentiment classification model. The model outputs sentiment type labels "romantic" (confidence 0.92) and "tranquil" (confidence 0.85), which are used as sentiment semantic features. Through these methods, the system extracts structured, multi-dimensional aesthetic features from visual information, providing a complete data foundation for the subsequent generation of initial aesthetic feature vectors.

[0040] It should be noted that the texture recognition model is used to identify texture categories in the visual information captured by the vehicle-mounted camera, providing texture feature basis for subsequent morphological mapping. This model is a trained deep learning neural network model, and its specific settings are as follows: The input to the texture recognition model is an image of the vehicle's external environment captured by the vehicle-mounted camera. The image size is preprocessed to a preset resolution (e.g., 224×224 pixels) and normalized, scaling the pixel values ​​to a preset range. The output of the texture recognition model is a texture category label. The texture category label includes various preset texture types, such as graininess, vertical lines, flowing curves, mesh textures, and wavy textures. Each input image corresponds to one output texture category label, representing the dominant texture feature type in the image. Texture recognition models can employ convolutional neural network architectures, specifically ResNet, VGG (Visual Geometry Group), and MobileNet (Lightweight Convolutional Neural Network), or networks specifically designed for texture recognition such as DeepTen (Texture Encoding Network) and FV-CNN (Fisher Vector Convolutional Neural Networks). The model is pre-trained on large-scale texture image datasets (such as FMD (Flickr Material Database) and DTD (Describable Textures Dataset)) and fine-tuned on labeled datasets for predefined texture categories to improve accuracy. The training process includes: collecting image samples containing various texture features, manually labeling texture categories to construct a training dataset; inputting images from the training dataset into the model, calculating the loss between the model output and the labeled values; iteratively updating the model parameters using backpropagation until the loss converges. After training, the model is capable of outputting texture category labels for input images.

[0041] This application applies a texture recognition model to the field of vehicle cockpit control. The model's input comes directly from real-time images captured by an onboard camera, and the output is used in shape mapping rules to convert natural texture features into preset design elements, such as mapping vertical straight line textures to separators or striped backgrounds in the interface, and mapping curved textures to smooth animation curve parameters.

[0042] The object detection model is used to identify target objects in the visual information captured by an in-vehicle camera, providing object category information for subsequent shape mapping. This model is a trained deep learning neural network model, and its specific settings are as follows: The input to the object detection model is an image of the vehicle's external environment captured by the in-vehicle camera. The image size is preprocessed to a preset resolution (e.g., 640×640 pixels) and normalized, scaling the pixel values ​​to a preset range. The output of the object detection model is the category label of the detected target object and its corresponding location information (e.g., bounding box coordinates). The target object category labels include preset natural object categories, such as cherry blossoms, maple leaves, bamboo, ocean waves, snowflakes, and trees. Each input image can output multiple target objects along with their category labels and location information. Object detection models can employ mainstream object detection network architectures, such as the YOLO (You Only Look Once) series (YOLOv5, YOLOv8, etc.), Faster R-CNN (Faster Region-based Convolutional Neural Networks), or Transformer-based detection networks (such as DETR (Detection Transformer). The model is pre-trained on publicly available object detection datasets (such as COCO (Common Objects in Context)) and fine-tuned on labeled datasets for predefined natural object categories to improve its ability to detect objects in natural environments. The training process includes: collecting image samples containing various types of natural objects, manually labeling object categories and bounding box locations to construct a training dataset; inputting images from the training dataset into the model, calculating the loss values ​​between the model's output category and location predictions and the labeled values; iteratively updating the model parameters using the backpropagation algorithm until the loss values ​​converge. After training, the model is capable of outputting the object category and location for the input image.

[0043] This application applies the target detection model to the field of vehicle cockpit control. The model's input comes directly from real-time images captured by the vehicle camera, and the output is used in the shape mapping rules to map the identified natural objects to corresponding icons or decorative elements, such as mapping cherry blossom objects to cherry blossom icons in the interface, and mapping maple leaf objects to maple leaf decorative patterns.

[0044] The sentiment classification model is used to identify the emotional semantic features in visual information captured by an in-vehicle camera, outputting sentiment type labels and their confidence scores to provide emotional basis for subsequent linkage mapping. This model is a trained deep learning neural network model, and its specific settings are as follows: The input to the sentiment classification model is an image of the vehicle's external environment captured by the in-vehicle camera. The image size is preprocessed to a preset resolution (e.g., 224×224 pixels) and normalized, scaling the pixel values ​​to a preset range. The output of the sentiment classification model is a sentiment type label and its corresponding confidence score. Sentiment type labels include preset sentiment categories, such as romantic, tranquil, spring, vibrant, warm, fresh, and solemn. Each input image corresponds to one or more sentiment type labels and their confidence scores, with confidence scores ranging from 0 to 1, indicating the reliability of the model's judgment. The sentiment classification model can adopt a convolutional neural network architecture, such as ResNet, DenseNet (Densely Connected Convolutional Networks), EfficientNet, and other classification network structures. The model can be pre-trained on sentiment image datasets (such as ArtPhoto Dataset, FI (Flickr and Instagram), and EmotionROI (Emotion Region of Interest) datasets) and fine-tuned on labeled datasets for preset sentiment categories to improve its ability to recognize image sentiment semantics. The training process includes: collecting image samples containing various natural aesthetic scenes, manually labeling them with sentiment types to construct a training dataset; inputting images from the training dataset into the model, calculating the loss value between the model output and the labeled labels; and iteratively updating the model parameters using the backpropagation algorithm until the loss value converges. After training, the model is capable of outputting sentiment type labels and confidence levels for input images.

[0045] This application applies an emotion classification model to the field of vehicle cockpit control. The model's input comes directly from real-time images captured by the vehicle's camera, and the output is used in the linkage mapping rules to convert emotional semantic features into the operating parameters of multiple target devices. For example, romantic emotions are mapped to the breathing pattern of ambient lights, the release parameters of specific fragrances, and the style of soothing music.

[0046] As one possible implementation, in some embodiments, an initial aesthetic feature vector is generated based on color features, texture features, target object features, and emotional semantic features. This includes: converting color features into color feature sub-vectors, each including a color value component and a weight component of at least one key color cluster; converting texture features into texture feature sub-vectors, each including an encoded representation of the texture category; converting target object features into object feature sub-vectors, each including an encoded representation of the object category; converting emotional semantic features into emotional feature sub-vectors, each including an encoded representation of the emotional type label and a confidence component of the emotional type label; and concatenating the color feature sub-vectors, texture feature sub-vectors, object feature sub-vectors, and emotional feature sub-vectors to generate the initial aesthetic feature vector.

[0047] It is understood that, in the embodiments of this application, the color feature sub-vector refers to the result after converting color features into a numerical vector form, which includes the color value components of key color clusters (such as RGB (Red, Green, Blue) or HSV (Hue, Saturation, Brightness) values) and weight components, used to quantify and describe the color composition of the scene. The encoding representation of texture categories refers to converting the texture categories (such as "fine graininess," "vertical lines," "flowing curves") output by the texture recognition model into a computer-processable numerical form, which can be achieved using methods such as ordinal encoding, one-hot encoding, or embedding vectors. The encoding representation of object categories refers to converting the object categories (such as "cherry blossoms," "maple leaves," "bamboo") output by the object detection model into a computer-processable numerical form, which can also be achieved using methods such as ordinal encoding, one-hot encoding, or embedding vectors. The encoding representation of sentiment type labels refers to converting the sentiment type labels (such as "romantic," "tranquil," "spring") output by the sentiment classification model into a computer-processable numerical form, which can be achieved using methods such as ordinal encoding or one-hot encoding. The confidence score is the confidence score corresponding to the sentiment type label output by the sentiment classification model. It reflects the reliability of the model's judgment on the sentiment and can range from 0 to 1.

[0048] Specifically, the system first converts various features into corresponding sub-vectors. For color feature processing, the system converts the extracted color features into color feature sub-vectors, each containing a color value component and a weight component for at least one key color cluster. The color value component quantifies the specific color value of the color cluster, while the weight component represents the visual proportion of the color cluster in the entire scene. Multiple key color clusters are arranged in a preset order to collectively form the color feature sub-vector. For texture feature processing, the system converts the extracted texture features into texture feature sub-vectors, each containing an encoded representation of the texture category. For target object processing, the system converts the identified target objects into object feature sub-vectors, each containing an encoded representation of the object category. For sentiment semantic feature processing, the system converts the extracted sentiment semantic features into sentiment feature sub-vectors, each containing an encoded representation of the sentiment type label and a confidence component for the sentiment type label. The encoded representation of the sentiment type label identifies the sentiment category, and the confidence component quantifies the reliability of the sentiment judgment; both together constitute the sentiment feature sub-vector. After completing the above transformation, the system can concatenate the color feature sub-vector, texture feature sub-vector, object feature sub-vector, and emotion feature sub-vector in a preset order, that is, connect multiple sub-vectors end to end and merge them into a higher-dimensional unified vector, which serves as the initial aesthetic feature vector. This vector fully encapsulates the multi-dimensional aesthetic attributes of the natural scene in a structured numerical form, providing a unified and computable input data foundation for subsequent style mapping.

[0049] For example, in color feature vectorization, the system can convert color features extracted from a cherry blossom forest image into color feature sub-vectors. These sub-vectors contain the color value components and weight components of three key color clusters, such as pinkish-white (RGB values ​​255, 200, 220, weight 0.6), brown (RGB values ​​160, 120, 80, weight 0.25), and light blue (RGB values ​​200, 220, 255, weight 0.15). These values ​​are arranged sequentially to form a multi-dimensional vector. In texture feature vectorization, the system can encode the "fine grainy" texture category output by the texture recognition model, for example, by using ordinal encoding to convert it into the value "1", as a texture feature sub-vector. In target object vectorization, the system can encode the "cherry blossom" object identified by the target detection model, for example, by using ordinal encoding to convert it into the value "2" (if the cherry blossom is numbered 2 in the preset object category), as an object feature sub-vector. In terms of emotional semantic feature vectorization, the system can encode the "romantic" (confidence 0.92) and "tranquil" (confidence 0.85) output by the emotional classification model. For example, "romantic" can be converted to "1" and "tranquil" to "2" using ordinal encoding, and then combined with their respective confidence scores to form an emotional feature sub-vector [1, 0.92, 2, 0.85]. After completing the above conversion, the system concatenates the color feature sub-vector, texture feature sub-vector, object feature sub-vector, and emotional feature sub-vector in a preset order to generate a unified multi-dimensional initial aesthetic feature vector, for example, [pink RGB value, weight, brown RGB value, weight, light blue RGB value, weight, texture encoding, object encoding, emotional 1 encoding, confidence, emotional 2 encoding, confidence].

[0050] In step S103, the initial aesthetic feature vector is mapped to stylized configuration parameters that conform to the preset theme specifications according to the preset theme mapping rule library, and the target theme configuration set is determined according to the stylized configuration parameters.

[0051] It can be understood that in the embodiments of the present application, the preset theme mapping rule library refers to a set of rules pre-constructed by the system, which is used to convert various aesthetic features in the initial aesthetic feature vector into stylized parameters that conform to the overall vehicle design language. The preset theme specification refers to the unified design standard predefined by the vehicle cockpit system, covering style constraints for multi-modal outputs such as vision, hearing, smell, and somatosensation (such as line style (round / sharp), icon style (linear / planar), dynamic transition curve (gentle / rapid), iconic pattern), ensuring the consistency and coordination of all outputs in terms of color, form, interaction logic, etc. The stylized configuration parameters refer to the set of structured parameters generated after being mapped by the preset theme mapping rule library and conforming to the preset theme specification, which are used to describe the specific style features that each subsystem of the cockpit should present. For example, the main color value, auxiliary color value, dynamic effect curve, atmosphere light mode, fragrance type, etc. The target theme configuration set is the complete set of parameters determined according to the stylized configuration parameters and used to drive the linkage of multi-modal devices, including the specific operation parameters of each subsystem such as the human-computer interaction interface, atmosphere light, fragrance, air conditioner, audio, etc., and is the direct basis for driving the linkage of multi-modal devices.

[0052] Specifically, the preset theme mapping rule library is a predefined and configurable set of rules, which is used to convert the quantified aesthetic features (such as color value, texture category, object category, emotion label and its confidence) in the initial aesthetic feature vector into stylized expressions that conform to the unified design standard of the vehicle cockpit. This mapping process can include conversions in multiple dimensions: in the color dimension, the preset theme mapping rule library can screen and adjust the original color values extracted from natural scenes to make them coordinated with the preset color palette system of the vehicle; in the form dimension, the preset theme mapping rule library can convert the identified texture features and object features into visual elements or dynamic effect parameters that match the vehicle design language; in the linkage dimension, the preset theme mapping rule library can convert the emotional semantic features into the respective operation mode parameters of multi-modal devices (such as atmosphere light, fragrance, air conditioner, audio, etc.). Through the above mapping, the system converts the unconstrained aesthetic feature vector originally from natural scenes into a set of structured stylized configuration parameters that conform to the preset theme specification.

[0053] Subsequently, the system determines the target theme configuration set based on the stylized configuration parameters. The target theme configuration set is a complete data set containing the operation parameters of multiple subsystems in the cockpit, and the parameters of each subsystem are logically consistent and stylistically coordinated with the stylized configuration parameters. This configuration set serves as the direct basis for subsequent multi-modal linkage control, ensuring that devices such as the human-computer interaction interface, atmosphere light, fragrance release device, air conditioning system, and audio system in the cockpit can operate in coordination around the unified theme specification when performing outputs, so as to achieve the unified expression of multi-dimensional experiences such as vision, hearing, smell, and somatosensation.

[0054] As one possible implementation, in some embodiments, the preset theme mapping rule library includes color mapping rules, shape mapping rules, and linkage mapping rules, which map the initial aesthetic feature vector to stylized configuration parameters that conform to the preset theme specifications. This includes: mapping the color features in the initial aesthetic feature vector to target color values ​​in a preset color wheel according to the color mapping rules; mapping the texture features and target objects in the initial aesthetic feature vector to preset design elements according to the shape mapping rules; and mapping the emotional semantic features in the initial aesthetic feature vector to the respective operating parameters of multiple target devices according to the linkage mapping rules.

[0055] It is understood that, in the embodiments of this application, the preset color palette refers to a predefined set of colors, including the primary color, secondary color, accent color, and their color value ranges allowed by vehicle design specifications, used to constrain the output results of color mapping. The preset design elements refer to various predefined visual components, motion parameters, or interaction modes that are consistent with the vehicle design language, such as separators, striped backgrounds, icons, animation curves, etc.

[0056] Specifically, in terms of color mapping, the system can map color features (such as the color values ​​and weights of key color clusters) in the initial aesthetic feature vector to target color values ​​in a preset color wheel according to color mapping rules. This process involves a color gamut adsorption mechanism, which adjusts natural color values ​​to the closest values ​​in the preset color wheel that conform to design specifications, and automatically derives auxiliary and accent colors based on color matching logic. In terms of form mapping, the system can map texture features and target objects in the initial aesthetic feature vector to preset design elements according to form mapping rules. For example, it maps vertical line textures to vertical separators or striped backgrounds in the interface, curve textures to smooth animation curve parameters, and identifies natural objects (such as cherry blossoms and maple leaves) to corresponding icons or decorative patterns. In terms of linkage mapping, the system can map emotional semantic features (such as emotional type labels and their confidence levels) in the initial aesthetic feature vector to the operating parameters of multiple target devices according to linkage mapping rules. For example, based on emotional labels such as "romantic" or "tranquil," it determines the dynamic mode of ambient lighting, the release parameters of fragrance, the air conditioning's wind mode, and the audio parameters of speakers. Through the synergistic effect of the above three types of rules, the system can transform the multi-dimensional aesthetic features in the initial aesthetic feature vector into a complete set of stylized configuration parameters that conform to the preset theme specifications, laying the foundation for the subsequent determination of the target theme configuration set.

[0057] For example, taking the scene of a vehicle entering a cherry blossom forest as an example, the initial aesthetic feature vector generated by the system includes color features such as pinkish white (RGB values ​​255, 200, 220, weight 0.6), brown (RGB values ​​160, 120, 80, weight 0.25), and light blue (RGB values ​​200, 220, 255, weight 0.15), as well as emotional semantic features such as "fine grain texture", "cherry blossom" target object, "romantic" (confidence 0.92) and "tranquil" (confidence 0.85). The system inputs these features into a preset theme mapping rule library for processing: In terms of color mapping, the color gamut adsorption mechanism in the preset theme mapping rule library can map potentially overly vibrant pink and white in nature to a more sophisticated "matte cherry blossom pink" (RGB values ​​245, 190, 200) in the vehicle's preset color palette, and automatically derive auxiliary and accent colors based on vehicle visual specifications; in terms of shape mapping, the preset theme mapping rule library can map "fine grainy" texture features to particle animation parameters for falling petals in the UI (User Interface); in terms of linkage mapping, the preset theme mapping rule library can map the emotional tags of "romance" and "tranquility" to the "soothing breathing" mode of ambient lighting, the exclusive "Kyoto early cherry blossom" fragrance of the air conditioner, the "gentle breeze" mode of the air conditioner, and the tranquil pure music parameters of the "Springtime" audio system. After the above mapping, the system generates a set of stylized configuration parameters that conform to the preset theme specifications, and determines the target theme configuration set accordingly. This configuration set is a structured and complete set of parameters, including visual parameters of the human-computer interaction interface (such as primary color, secondary color, background pattern, and motion curve), light effect parameters of ambient lighting (such as color, brightness, and dynamic mode), release parameters of fragrance devices (such as fragrance type and concentration), wind parameters of the air conditioning system (such as wind speed, wind direction, and mode), and audio parameters of the audio system (such as sound source, volume, and playback logic), providing a unified and coordinated control basis for subsequent multimodal device linkage execution.

[0058] In step S104, multimodal linkage control instructions are generated based on the target theme configuration set, and at least one target device in the vehicle cabin is controlled to operate according to the parameters in the target theme configuration set according to the multimodal linkage control instructions.

[0059] It is understood that, in the embodiments of this application, multimodal linkage control commands refer to specific control signals generated based on the target theme configuration set for synchronous or coordinated control of multiple devices with different modes. Each command corresponds to a specific execution action of one or more devices. The target devices are various execution devices in the cockpit that can receive control commands and output them, including but not limited to human-machine interfaces (central control screen, instrument panel), ambient lighting, fragrance release devices, air conditioning systems, and audio systems.

[0060] Specifically, the target theme configuration set is a structured and complete set of parameters, which includes the operating parameters corresponding to each subsystem in the cockpit. For example, the visual parameters of the human-machine interaction interface (main color value, auxiliary color value, background pattern, font style, animation curve), the light effect parameters of the ambient light (color, brightness, dynamic mode, change speed), the release parameters of the fragrance device (fragrance type, release concentration, release duration), the wind feeling parameters of the air conditioning system (wind speed, wind direction, operation mode), and the audio parameters of the audio system (sound source selection, volume, playback mode, fade-in and fade-out effect), etc. The system can convert these parameters into multi-modal linkage control instructions that can be parsed and executed by each device. For example, it can convert the visual parameters into screen display instructions, the light effect parameters into lighting controller instructions, the odor parameters into fragrance device instructions, the wind feeling parameters into air conditioning system instructions, and the audio parameters into audio system instructions. At the instruction execution level, the system sends these control instructions to the corresponding target devices, driving each device to operate collaboratively according to the parameters assigned to it in the target theme configuration set. This collaborative mechanism can ensure that all devices work around a unified theme specification during output. For example, the color of the human-machine interaction interface is coordinated with the color of the ambient light, the sound effect of the audio is matched with the emotional semantic features, and the wind feeling of the air conditioning is integrated with the scene atmosphere, so as to achieve the unified expression and immersive experience of multi-dimensional outputs such as vision, hearing, smell, and body sensation.

[0061] To facilitate those skilled in the art to further understand the control method of the vehicle cockpit proposed in the embodiments of the present application, the following will be further elaborated with specific embodiments.

[0062] Such as Figure 2As shown, assume that the user drives the vehicle into a cherry blossom forest. The system cross-verifies the on-vehicle GPS positioning information with the visual information collected by the front camera, and confirms that the external environment of the vehicle belongs to the "cherry blossom forest" scene in the preset natural aesthetics scene category. Subsequently, the system analyzes the visual information collected by the camera, identifies a large number of "cherry blossom" target objects, and extracts the main environmental color as "pink and white". These original "pink and white" color values and "cherry blossom" semantic labels are sent to the style mapping module. The style mapping module maps the slightly bright pink and white in nature to a "matte cherry pink" with a fine-tuned saturation in the preset color palette as the target color value, and automatically generates a set of coordinated auxiliary colors and accent colors according to the preset theme specifications. At the same time, the "cherry blossom" label triggers the preset "Quiet Realm of Spring Cherry" theme narrative, which covers a unified visual, animation, fragrance, and sound expression logic. The generative scene theme engine calls the corresponding full set of pre-designed parameters from the preset theme configuration database according to the translated "Quiet Realm of Spring Cherry" theme identifier as the target theme configuration set. These parameters are not randomly generated, but are pre-defined jointly by designers and engineers in advance, ensuring the ultimate quality and consistency of the experience. The linkage execution module synchronously controls each target device in the cockpit within seconds: the human-machine interaction interface switches to a special cherry blossom dynamic theme, and the dynamic curves of the icons, fonts, and falling cherry petals strictly follow the preset theme specifications; the ambient lights of the whole vehicle gently turn into "matte cherry pink" color and adopt the preset "slow breathing" mode; the fragrance release device releases the specially formulated "Early Cherry in Kyoto" exclusive fragrance; the air conditioner fan speed is automatically reduced, and the wind direction is adjusted to a dispersed mode to simulate the natural breeze under the cherry blossom forest; the sound system fades in and plays the carefully selected quiet pure music "Spring Time". All outputs operate in coordination around the unified target theme configuration set, creating an immersive experience in the cockpit that echoes the cherry blossom forest outside the vehicle.

[0063] When the system determines based on visual information and geographical location information that the external environment of the vehicle no longer belongs to the preset natural aesthetics scene category, the system does not immediately close the theme, but maintains the output of the current target theme configuration set for a preset continuation duration according to the preset experience design principles, allowing the emotional experience to continue naturally. Subsequently, the system stops or switches the outputs of each target device in the preset fade-out order: the sound system fades out first, the ambient lights slowly return to the default state, and the human-machine interaction interface finally switches to the default theme, and the whole process is smooth and imperceptible. If the system detects that the vehicle drives into a new natural aesthetics scene, such as the seaside, it immediately starts a new round of recognition and mapping processes, generates a new target theme configuration set, and executes the corresponding multi-modal linkage control.

[0064] According to the vehicle cockpit control method proposed in this application, when the external environment of the vehicle belongs to a preset natural aesthetic scene (such as natural scenery) through visual perception and positioning identification, features such as color, texture, target objects, and emotional semantics are extracted to generate an initial aesthetic feature vector. This vector is then converted into stylized configuration parameters conforming to the preset theme specifications according to a theme mapping rule library. This generates multimodal linkage control commands to control the operation of in-cabin equipment (such as lights, audio, air conditioning, screens, etc.) according to the theme configuration, achieving intelligent coordination and immersive experience between the vehicle's internal and external environments. This solves the problem that related technologies cannot perceive and understand the aesthetic features of the external environment and struggle to cope with infinitely rich real-world scenes, achieving deep linkage and emotional integration between the vehicle cockpit and the external natural environment, thus enhancing the user experience.

[0065] Next, the control device for the vehicle cabin according to an embodiment of this application is described with reference to the accompanying drawings.

[0066] Figure 3 This is a block diagram of a vehicle cockpit control device according to an embodiment of this application.

[0067] like Figure 3 As shown, the control device 10 of the vehicle cockpit includes: a judgment module 100, a generation module 200, a determination module 300 and a control module 400.

[0068] The judgment module 100 is used to acquire visual information and geographical location information of the vehicle's external environment, and based on the visual information and geographical location information, to determine whether the vehicle's external environment belongs to a preset natural aesthetic scene category. The generation module 200 is used to extract color features, texture features, target object and emotional semantic features from visual information when the vehicle's external environment belongs to a preset natural aesthetic scene category, and to generate an initial aesthetic feature vector based on the color features, texture features, target object and emotional semantic features. The determination module 300 is used to map the initial aesthetic feature vector to stylized configuration parameters that conform to the preset theme specifications according to the preset theme mapping rule library, and to determine the target theme configuration set according to the stylized configuration parameters; The control module 400 is used to generate multimodal linkage control commands based on the target theme configuration set, and to control at least one target device in the vehicle cabin to operate according to the parameters in the target theme configuration set according to the multimodal linkage control commands.

[0069] Optionally, in some embodiments, the determination module 100 is specifically used for: The geographic location information is matched with a preset scene geographic tag library to obtain the first confidence level; Visual information is input into a scene classification model to obtain the second confidence level; If the first confidence level is greater than the first preset threshold and the second confidence level is greater than the second preset threshold, the vehicle's external environment is determined to belong to the preset natural aesthetic scene category.

[0070] Optionally, in some embodiments, color features, texture features, target object features, and emotional semantic features are extracted from visual information to generate module 200, which is specifically used for: The main color extraction algorithm is used to segment the visual information to obtain a preset number of key color clusters and the color value and weight of each key color cluster. The key color clusters and the color value and weight of each key color cluster are used as color features. Visual information is input into the texture recognition model to identify the texture category in the visual information, and the texture category is used as the texture feature. Input visual information into the object detection model to identify target objects in the visual information; Visual information is input into the sentiment classification model to obtain sentiment type labels and their confidence scores. The sentiment type labels and their confidence scores are then used as sentiment semantic features.

[0071] Optionally, in some embodiments, an initial aesthetic feature vector is generated based on color features, texture features, target object features, and emotional semantic features. The generation module 200 is specifically used for: The color features are converted into color feature subvectors, which include the color value component and weight component of at least one key color cluster. The texture features are converted into texture feature subvectors, which include the encoded representation of the texture category; The target object is converted into an object feature vector, which contains an encoded representation of the object category; The sentiment semantic features are converted into sentiment feature sub-vectors, which include the encoded representation of sentiment type labels and the confidence component of sentiment type labels; The color feature vector, texture feature vector, object feature vector, and emotion feature vector are combined to generate the initial aesthetic feature vector.

[0072] Optionally, in some embodiments, the determining module 300 is specifically used for: According to the color mapping rules, the color features in the initial aesthetic feature vector are mapped to the target color values ​​in the preset color wheel; According to the shape mapping rules, the texture features and target objects in the initial aesthetic feature vector are mapped to preset design elements; According to the linkage mapping rule, the emotional semantic features in the initial aesthetic feature vector are mapped to the operating parameters of multiple target devices.

[0073] It should be noted that the foregoing explanation of the vehicle cockpit control method embodiment also applies to the vehicle cockpit control device of this embodiment, and will not be repeated here.

[0074] According to the vehicle cockpit control device proposed in this application, when the external environment of the vehicle belongs to a preset natural aesthetic scene (such as natural scenery) through visual perception and positioning recognition, it extracts features such as color, texture, target objects, and emotional semantics to generate an initial aesthetic feature vector. This vector is then converted into stylized configuration parameters conforming to the preset theme specifications according to a theme mapping rule library. This generates multimodal linkage control commands to control the operation of in-cabin equipment (such as lights, audio, air conditioning, screens, etc.) according to the theme configuration, achieving intelligent coordination and immersive experience between the interior and exterior environments. This solves the problem that related technologies cannot perceive and understand the aesthetic features of the external environment and struggle to cope with infinitely rich real-world scenes, achieving deep linkage and emotional integration between the vehicle cockpit and the external natural environment, thus enhancing the user experience.

[0075] Figure 4 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include: The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0076] When the processor 402 executes the program, it implements the vehicle cockpit control method provided in the above embodiments.

[0077] Furthermore, the vehicle also includes: Communication interface 403 is used for communication between memory 401 and processor 402.

[0078] The memory 401 is used to store computer programs that can run on the processor 402.

[0079] The memory 401 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0080] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0081] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0082] Processor 402 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0083] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described vehicle cockpit control method.

[0084] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described vehicle cockpit control method.

[0085] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0086] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0087] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for controlling a vehicle cockpit, characterized in that, Includes the following steps: The system acquires visual and geographic information of the vehicle's external environment and determines whether the vehicle's external environment belongs to a preset natural aesthetic scene category based on the visual and geographic information. When the vehicle's external environment belongs to the preset natural aesthetic scene category, color features, texture features, target object features, and emotional semantic features are extracted from the visual information, and an initial aesthetic feature vector is generated based on the color features, texture features, target object features, and emotional semantic features. According to the preset theme mapping rule library, the initial aesthetic feature vector is mapped to stylized configuration parameters that conform to the preset theme specifications, and the target theme configuration set is determined according to the stylized configuration parameters; Based on the target theme configuration set, multimodal linkage control instructions are generated, and at least one target device in the vehicle cabin is controlled to operate according to the parameters in the target theme configuration set according to the multimodal linkage control instructions.

2. The method according to claim 1, characterized in that, The step of determining whether the vehicle's external environment belongs to a preset natural aesthetic scene category based on the visual information and the geographical location information includes: The geographic location information is matched with a preset scene geographic tag library to obtain a first confidence level; The visual information is input into the scene classification model to obtain the second confidence level; If the first confidence level is greater than the first preset threshold and the second confidence level is greater than the second preset threshold, the vehicle's external environment is determined to belong to the preset natural aesthetic scene category.

3. The method according to claim 1, characterized in that, The extraction of color features, texture features, target object features, and emotional semantic features from the visual information includes: The visual information is segmented using a primary color extraction algorithm to obtain a preset number of key color clusters and the color value and weight of each key color cluster, and the key color clusters and the color value and weight of each key color cluster are used as the color features. The visual information is input into a texture recognition model to identify the texture category in the visual information, and the texture category is used as the texture feature. The visual information is input into the target detection model to identify the target object in the visual information; The visual information is input into the sentiment classification model to obtain sentiment type labels and their confidence scores. The sentiment type labels and their confidence scores are then used as the sentiment semantic features.

4. The method according to claim 3, characterized in that, The process of generating an initial aesthetic feature vector based on the color features, texture features, target object, and emotional semantic features includes: The color features are converted into color feature sub-vectors, which include color value components and weight components of at least one key color cluster. The texture features are converted into texture feature sub-vectors, the texture feature sub-vectors including the encoded representation of the texture category; The target object is converted into an object feature vector, which contains an encoded representation of the object category; The emotional semantic features are converted into emotional feature sub-vectors, which include the encoded representation of the emotional type label and the confidence component of the emotional type label; The initial aesthetic feature vector is generated by concatenating the color feature sub-vector, the texture feature sub-vector, the object feature sub-vector, and the emotion feature sub-vector.

5. The method according to claim 1, characterized in that, The preset theme mapping rule base includes color mapping rules, shape mapping rules, and linkage mapping rules. Mapping the initial aesthetic feature vector to stylized configuration parameters conforming to the preset theme specifications includes: According to the color mapping rules, the color features in the initial aesthetic feature vector are mapped to the target color values ​​in the preset color wheel; According to the morphological mapping rule, the texture features and target objects in the initial aesthetic feature vector are mapped to preset design elements; According to the linkage mapping rule, the emotional semantic features in the initial aesthetic feature vector are mapped to the operating parameters of multiple target devices.

6. A control device for a vehicle cockpit, characterized in that, include: The judgment module is used to acquire visual information and geographical location information of the vehicle's external environment, and based on the visual information and geographical location information, to determine whether the vehicle's external environment belongs to a preset natural aesthetic scene category; The generation module is used to extract color features, texture features, target object and emotional semantic features from the visual information when the vehicle's external environment belongs to the preset natural aesthetic scene category, and generate an initial aesthetic feature vector based on the color features, texture features, target object and emotional semantic features; The determination module is used to map the initial aesthetic feature vector into stylized configuration parameters that conform to the preset theme specifications according to the preset theme mapping rule library, and to determine the target theme configuration set according to the stylized configuration parameters; The control module is used to generate multimodal linkage control commands based on the target theme configuration set, and control at least one target device in the vehicle cabin to operate according to the parameters in the target theme configuration set according to the multimodal linkage control commands.

7. The apparatus according to claim 6, characterized in that, The judgment module is specifically used for: The geographic location information is matched with a preset scene geographic tag library to obtain a first confidence level; The visual information is input into the scene classification model to obtain the second confidence level; If the first confidence level is greater than the first preset threshold and the second confidence level is greater than the second preset threshold, the vehicle's external environment is determined to belong to the preset natural aesthetic scene category.

8. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, the processor executing the program to implement the vehicle cockpit control method as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the vehicle cockpit control method as described in any one of claims 1-5.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, is used to implement the vehicle cockpit control method according to any one of claims 1-5.

Images