Automobile cabin intelligent control method and system based on image acquisition

Abstract

The invention discloses an automobile cabin intelligent control method and system based on image acquisition, which is used in the technical field of intelligent automobiles, and comprises the following steps: acquiring original image data in a cabin, and preprocessing the original image data based on a PID control algorithm to obtain standard image data; using a multi-task loss function to train an SSD model, and analyzing the standard image data to obtain a target detection result; constructing a decision model based on the target detection result and the environmental data, and optimizing the decision model in combination with user preference to obtain a reinforcement learning decision model; and obtaining a real-time decision result according to the real-time standard image data in combination with the reinforcement learning decision model and the safety guarantee rule, and adjusting the cabin according to the real-time decision result. According to the method, the state in the cabin and passenger behaviors can be accurately recognized, more accurate environment adjustment and personalized setting are provided, and the user experience is improved by combining the reinforcement learning model, the adaptive changing environment and user habits.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent vehicles, and more particularly, to a method and system for intelligent control of a vehicle cockpit based on image acquisition. Background Art

[0002] An intelligent cockpit refers to the interior space of a vehicle that integrates a variety of advanced sensors, computing platforms, human-machine interfaces, and intelligent algorithms to achieve dynamic perception and intelligent control of the vehicle interior environment. It aims to provide a safer, more comfortable, more convenient, and personalized driving experience. An intelligent cockpit generally includes, but is not limited to, the following aspects: driver monitoring systems (such as fatigue detection), passenger status recognition, automatic temperature adjustment, personalized seat and audio settings, augmented reality navigation, etc.

[0003] In recent years, deep learning algorithms, especially convolutional neural networks (CNNs), have made breakthroughs in object detection, face recognition, etc., making high-precision image analysis possible. At the same time, the cost of high-definition cameras and other types of sensors (such as infrared sensors, ultrasonic sensors) has been reduced and their performance has been greatly improved, providing strong hardware support for realizing comprehensive monitoring of the vehicle interior environment.

[0004] Modern consumers increasingly focus on the personalization and differentiation of products. For vehicles, in addition to the basic transportation function, they also expect to obtain a richer and more comfortable driving experience. This trend has promoted the development of the intelligent cockpit concept. However, the current market cannot provide a safer, more comfortable, and more personalized cockpit experience.

[0005] Regarding the problems in the related art, no effective solutions have been proposed yet. Summary of the Invention

[0006] In order to overcome the above problems, the present invention aims to propose a method and system for intelligent control of a vehicle cockpit based on image acquisition, aiming to solve the problem of inability to provide a more personalized cockpit experience.

[0007] To this end, the specific technical solutions adopted by the present invention are as follows: According to one aspect of the present invention, there is provided a method for intelligent control of a vehicle cockpit based on image acquisition, the method including the following steps: S1. Collect the original image data in the cockpit, preprocess the original image data based on the PID control algorithm to obtain standard image data; S2. Train the SSD model using a multi-task loss function, analyze the standard image data to obtain the target detection result; S3. Based on the object detection results and environmental data, construct a decision-making model, and optimize the decision-making model in combination with user preferences to obtain a reinforcement learning decision-making model; S4. According to the real-time standard image data, combine the reinforcement learning decision-making model and safety guarantee rules to obtain real-time decision results, and adjust the cockpit according to the real-time decision results.

[0008] Optionally, collect the original image data in the cockpit, and preprocess the original image data based on the PID control algorithm to obtain standard image data, including the following steps: S11. According to the light intensity of the current environment, automatically adjust the camera parameters based on the PID control algorithm, and collect the original image data in the cockpit in real time; S12. Process the original image data based on dynamic contrast enhancement and real-time noise reduction to obtain enhanced image data; S13. Adjust the enhanced image data to a unified size and convert it to a unified format to obtain standard image data.

[0009] Optionally, the expression formula of the PID control algorithm is: u(t)=P+I+D; In the formula, u(t) represents the control output; P represents the proportional term; I represents the integral term; D represents the derivative term.

[0010] Optionally, initialize the SSD model using the multi-task loss function, and analyze the standard image data to obtain object detection results, including the following steps: S21. Label the object detection results of each image in the collected image data through manual annotation to obtain the training set and test set of the object detection result annotation; S22. Initialize the SSD model parameters using the multi-task loss function, and train and test the SSD model based on the training set and test set of the object detection result annotation; S23. Input the standard image data into the SSD model after training and testing, and output through the SSD model to obtain the object detection results.

[0011] Optionally, initializing the SSD model parameters using the multi-task loss function includes the following steps: S221. The SSD model uses the pre-trained CNN as the backbone network, loads the pre-trained weights, and initializes the feature extraction part; S222. Add convolutional layers on the basis of the backbone network to generate feature maps of different scales; S223. Set prior boxes at each position of each feature map, and calculate the offset between the predicted box and the real box; S224. Calculate and update the SSD model parameters according to the multi-task loss function, and iteratively optimize the SSD model.

[0012] Optionally, the expression formula of the multi-task loss function is: ; In the formula, x represents the prior box matching indicator; c represents the class prediction confidence; l represents the offset of the predicted box; g represents the coordinates of the ground truth box; N represents the number of matched prior boxes; α represents the localization loss weight; L(x, c, l, g) represents the multi-task loss function; L conf represents the classification loss function; L loc represents the localization loss function.

[0013] Optionally, based on the object detection results and environmental data, construct a decision-making model, and optimize the decision-making model in combination with user preferences to obtain a reinforcement learning decision-making model, including the following steps: S31. Integrate the object detection results and environmental data, perform feature extraction, construct a unified feature vector, and obtain the state space; S32. Based on the pre-configured action space, combine the reward function and the state space to construct a decision-making model; S33. Collect the historical behavior data of the user's cockpit environment adjustment, construct a user preference model based on the information gain algorithm, and train the decision-making model in combination with the user preference model to obtain a reinforcement learning decision-making model.

[0014] Optionally, record the historical behavior data of the user's cockpit environment adjustment, construct a user preference model based on the information gain algorithm, and train the decision-making model in combination with the user preference model to obtain a reinforcement learning decision-making model, including the following steps: S331. Collect the behavior data of the user's cockpit adjustment and the context information during the adjustment to obtain the historical behavior data; S332. Clean and preprocess the collected historical data, convert it into a unified format, perform feature extraction, and construct a user preference model based on the information gain algorithm; S333. Optimize the reward function in the decision-making model according to the user preference model to obtain a reinforcement learning decision-making model.

[0015] Optionally, the expression formula of the information gain algorithm is: ; In the formula, Gain(D,A) represents the information gain function, D represents the current node dataset; A represents the feature to be partitioned; V represents the number of values of feature A; D v represents the subset of feature A taking the v-th value; H(D) represents the entropy of dataset D; The expression formula of the reward function is as follows: ; wherein, R(s,a) represents the reward obtained by taking action a in state s; P pref (s,a) represents the preference reward score calculated based on the user preference model; Q base (s,a) represents the basic reward score calculated based on other factors; w1 represents the weight of the preference reward; w2 represents the weight of the basic reward.

[0016] According to another aspect of the present invention, there is also provided an intelligent control system for an automotive cockpit based on image acquisition, and the system includes: an image acquisition module, an image analysis module, a decision-making module, and a control module; The image acquisition module is used to acquire the original image data inside the cockpit, and preprocess the original image data based on the PID control algorithm to obtain standard image data; The image analysis module is used to train the SSD model using a multi-task loss function, and analyze the standard image data to obtain a target detection result; The decision-making module is used to construct a decision-making model based on the target detection result and environmental data, and optimize the decision-making model in combination with user preferences to obtain a reinforcement learning decision-making model; The control module is used to obtain a real-time decision result according to the real-time standard image data, in combination with the reinforcement learning decision-making model and safety guarantee rules, and adjust the cockpit according to the real-time decision result.

[0017] Compared with the prior art, the present application has the following beneficial effects: 1. Through real-time image acquisition and analysis, the present invention can accurately identify the state inside the cockpit and the behavior of passengers, and provide more precise environmental adjustment and personalized settings. Compared with the traditional data acquisition method based on fixed sensors, image data can provide a richer information dimension, improving the response speed and decision-making accuracy of the system.

[0018] 2. The present invention combines a reinforcement learning model, and this method can adapt to the changing environment and user habits, continuously learn and optimize the decision-making strategy to provide a control scheme that better meets the user's needs, and can intelligently adjust the cockpit settings in different situations, improving the user experience.

[0019] 3. By learning the historical behavior and preferences of users, the present invention can provide personalized cockpit adjustment suggestions, meet the personalized needs of different users, improve user satisfaction, and enable users to feel more convenience and comfort during use. Description of the Drawings

[0020] With the following description of the embodiments, the above characteristics, features, and advantages of the present invention, as well as the implementation manners and methods thereof, become more understandable. The embodiments are elaborated in detail in conjunction with the accompanying drawings. It is illustrated schematically as follows: Figure 1 is a flowchart of a method for intelligent control of an automotive cockpit based on image acquisition according to an embodiment of the present invention; Figure 2 is a schematic block diagram of a system for intelligent control of an automotive cockpit based on image acquisition according to an embodiment of the present invention.

[0021] In the figure: 1. Image acquisition module; 2. Image analysis module; 3. Decision-making module; 4. Control module. Specific embodiments

[0022] In order to enable those skilled in the art of the present technology to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0023] According to an embodiment of the present invention, a method and a system for intelligent control of an automotive cockpit based on image acquisition are provided.

[0024] Now, the present invention will be further described in conjunction with the accompanying drawings and specific embodiments. As Figure 1 shown, according to an embodiment of the present invention, a method for intelligent control of an automotive cockpit based on image acquisition is provided. The method includes the following steps: S1. Collect the original image data in the cockpit, and preprocess the original image data based on the PID control algorithm to obtain standard image data.

[0025] Preferably, collecting the original image data in the cockpit and preprocessing the original image data based on the PID control algorithm to obtain standard image data includes the following steps: S11. Automatically adjust the camera parameters based on the PID control algorithm according to the light intensity of the current environment, and collect the original image data in the cockpit in real time; S12. Process the original image data based on dynamic contrast enhancement and real-time noise reduction to obtain enhanced image data; S13. Adjust the enhanced image data to a unified size and convert it to a unified format to obtain standard image data.

[0026] Preferably, the expression formula of the PID control algorithm is: u(t) = P + I + D; Wherein, u(t) represents the control output; P represents the proportional term; I represents the integral term; D represents the derivative term.

[0027] It should be explained that a light sensor is installed in the cockpit to monitor the ambient light intensity in real time, and the initial image collected by the camera is analyzed to evaluate the current lighting conditions (such as too bright, too dark or normal).

[0028] Adjust the exposure time of the camera according to the light intensity to make the average image brightness μ approach the target brightness μ target (usually set to 128 / 255), and automatically adjust the exposure time of the camera based on the PID control algorithm; when the exposure time reaches the limit and still cannot meet the brightness requirement, automatically enable the gain assist control based on the PID control algorithm; in order to make the R / G / B channel means of the gray object in the image equal (μ R = μ G = μ B ), automatically adjust the camera white balance based on the PID control algorithm.

[0029] The expression formula of the PID control algorithm is: ; Wherein, u(t) represents the control output; e(t) represents the difference between the target value and the actual value; K p represents the proportional gain coefficient; K i represents the integral gain coefficient; K d represents the derivative gain coefficient.

[0030] The specific steps for automatically adjusting the camera parameters based on the PID control algorithm are as follows: 1. Calculate the error.

[0031] The expression formula of the error formula is: ; Wherein, e(t) represents the difference between the target value and the actual value; μ target represents the expected value; μ(t) represents the actual value.

[0032] 2. Implement the PID controller.

[0033] 1) Calculate the proportional term (directly proportional to the current error, quickly responding to the error).

[0034] The calculation formula of the proportional term is: ; Wherein, P represents the proportional term; K p represents the proportional gain coefficient; e( t ) represents the difference between the target value and the actual value.

[0035] 2) Calculate the integral term (used to eliminate the steady-state error).

[0036] The calculation formula for the integral term is: ; In the formula, I represents the integral term; K i represents the integral gain coefficient; e(τ) represents the error value that changes with time; τ represents the time dummy variable in the integral operation; t represents the current moment.

[0037] 3) Calculate the derivative term (used to predict the error trend and provide damping).

[0038] The calculation formula for the derivative term is: ; In the formula, D represents the integral term; K d represents the derivative gain coefficient; e(t) represents the difference between the target value and the actual value; represents the rate of change of the error e(t) with time.

[0039] 3. Update the camera parameters.

[0040] The expression formula of the control algorithm is: u(t)=P + I + D; In the formula, u(t) represents the control output; P represents the proportional term; I represents the proportional term; D represents the proportional term.

[0041] Adjust the exposure time, gain, or white balance parameters of the camera according to the control output. Through PID control, precise adjustment of the camera parameters can be achieved, thereby maintaining the stability of the image quality under different lighting conditions. Adjusting the PID parameters requires optimization according to the specific application scenario to achieve the best control effect.

[0042] According to the above operations, the original image data in the cockpit is collected in real time. To adaptively adjust the brightness and contrast of the original image data, the adaptive histogram equalization (CLAHE) technique can be used to enhance the details of the image. If it is detected that the noise of the original image data is relatively high, an adaptive filtering technique (such as adaptive median filtering) is applied to remove the noise while retaining the edge details of the original image data. Adjust the enhanced image data to a unified size for subsequent processing. Bilinear interpolation can be used for scaling, and the enhanced image data is converted into the standard format required by the model, such as converting the image from the RGB format to the grayscale image or other specific color spaces (such as YUV), and finally obtaining the standard image data.

[0043] S2. Train the SSD model using the multi - task loss function, analyze the standard image data, and obtain the object detection results.

[0044] Preferably, initializing the SSD model using the multi - task loss function, analyzing the standard image data, and obtaining the object detection results include the following steps: S21. Label the object detection results for each image of the collected image data through manual annotation to obtain the training set and test set of the object detection result annotations. S22. Initialize the SSD model parameters using the multi - task loss function, and train and test the SSD model based on the training set and test set of the object detection result annotations. S23. Input the standard image data into the SSD model after training and testing are completed, and obtain the object detection results through the output of the SSD model.

[0045] Preferably, initializing the SSD model parameters using the multi - task loss function includes the following steps: S221. The SSD model uses a pre - trained CNN as the backbone network, loads the pre - trained weights, and initializes the feature extraction part. S222. Add convolutional layers on the basis of the backbone network to generate feature maps of different scales. S223. At each position of each feature map, set the prior boxes and calculate the offsets between the predicted boxes and the ground - truth boxes. S224. Calculate and update the SSD model parameters according to the multi - task loss function, and iteratively optimize the SSD model.

[0046] Preferably, the expression formula of the multi - task loss function is: ; In the formula, x represents the prior box matching indicator; c represents the class prediction confidence; l represents the offset of the predicted box; g represents the coordinates of the ground - truth box; N represents the number of matching prior boxes; α represents the localization loss weight; L(x, c, l, g) represents the multi - task loss function; L conf represents the classification loss function; L loc represents the localization loss function.

[0047] It should be noted that the specific steps of training the SSD model using the multi - task loss function, analyzing the standard image data, and obtaining the object detection results are as follows: Step 1. Label the object detection results for each image of the collected image data through manual annotation to obtain the training set and test set of the object detection result annotations.

[0048] Use publicly available PASCAL VOC or COCO datasets, which contain a large number of images and annotation information in natural scenes. Adopt annotation tools such as LabelImg or CVAT to perform bounding box annotation and class annotation on the targets in the images. Save the annotation results in XML or JSON format, including information such as image path, target class, and bounding box coordinates. Divide the annotated dataset into a training set and a test set, with a ratio usually of 8:2 or 7:3.

[0049] Step 2: Initialize the SSD model parameters using a multi-task loss function, and train and test the SSD model based on the annotated training set and test set.

[0050] Select a pre-trained CNN (such as VGG16, etc.) as the backbone network of the SSD model, load the VGG16 weights pre-trained on the ImageNet dataset, and initialize the feature extraction part of the SSD model.

[0051] Based on VGG16, add multiple convolutional layers (such as 3×3 convolutional kernels) to generate multiple feature maps of different scales (such as 38×38, 19×19, 10×10, etc.) to capture targets of different sizes (such as large targets at close range and small targets at long range).

[0052] Preset a set of prior boxes (Anchor Boxes) at each position of each feature map. These boxes have different width-to-height ratios (such as 1:1, 1:2, 2:1, etc.), and the number and ratio of prior boxes are adjusted according to the actual task requirements. For each prior box, calculate the offset between it and the ground truth box (including the offset of the center point and the scaling of width and height).

[0053] The expression formula of the multi-task loss function is: ; In the formula, x represents the prior box matching indicator; c represents the class prediction confidence; l represents the offset of the predicted box; g represents the coordinates of the ground truth box; N represents the number of matching prior boxes; α represents the localization loss weight; L(x, c, l, g) represents the multi-task loss function; L conf represents the classification loss function; L loc represents the localization loss function.

[0054] The expression formula of the classification loss function is: ; In the formula, x represents the prior box matching indicator. If the prior box matches a certain ground truth target box, then x = 1; otherwise, x = 0. c represents the class prediction confidence, that is, the probability distribution of each class predicted by the model.

[0055] The expression formula of the localization loss function is as follows: ; ; In the formula, x represents the prior box matching indicator; l represents the offset of the predicted box, which is the adjustment value of the center point coordinates and the width and height dimensions of the predicted box relative to the prior box; g represents the coordinates of the ground truth box, which is the position information of the actual bounding box of the target object; i represents the i-th prior box or target; smooth L1 represents the Smooth L1 loss function; z represents the difference between the offset l of the predicted box and the offset g of the ground truth box.

[0056] Step 3: Input the standard image data into the SSD model after training and testing are completed, and obtain the target detection result through the output of the SSD model.

[0057] Use the training set to train the SSD model, and the initial learning rate is 0.001. Calculate the multi-task loss function value for each iteration and update the model parameters through backpropagation. Input the standard image data in the test set into the trained SSD model, and the model outputs the class prediction confidence and bounding box coordinates of each target. Use the non-maximum suppression (NMS) algorithm to remove redundant detection boxes, retain the box with the highest confidence, and output the final target detection result, including the class label and bounding box coordinates.

[0058] For example, input an image, and after detection by the trained SSD model, a driver (class: "driver", confidence: 0.95, bounding box coordinates: 120, 80, 200, 150) and a passenger (class: "passenger", confidence: 0.90, bounding box coordinates: 300, 100, 380, 170) are obtained.

[0059] S3. Based on the target detection result and environmental data, construct a decision-making model, and optimize the decision-making model in combination with user preferences to obtain a reinforcement learning decision-making model.

[0060] Preferably, based on the target detection result and environmental data, constructing a decision-making model, and optimizing the decision-making model in combination with user preferences to obtain a reinforcement learning decision-making model includes the following steps: S31. Integrate the target detection result and environmental data, perform feature extraction, construct a unified feature vector, and obtain the state space; S32. Based on the pre-configured action space, combine the reward function and the state space to construct a decision-making model; S33. Collect the historical behavior data of the user's cockpit environment adjustment, construct a user preference model based on the information gain algorithm, and train the decision-making model in combination with the user preference model to obtain a reinforcement learning decision-making model.

[0061] Preferably, record the historical behavior data of the user's adjustment of the cockpit environment, construct a user preference model based on the information gain algorithm, and train a decision model in combination with the user preference model to obtain a reinforcement learning decision model, including the following steps: S331. Collect the behavior data of the user's adjustment of the cockpit and the context information during the adjustment to obtain historical behavior data; S332. Clean and preprocess the collected historical data, convert it into a unified format, and perform feature extraction. Based on the information gain algorithm, construct a user preference model; S333. Optimize the reward function in the decision model according to the user preference model to obtain a reinforcement learning decision model.

[0062] Preferably, the expression formula of the information gain algorithm is: ; In the formula, Gain(D,A) represents the information gain function, D represents the current node data set; A represents the feature to be partitioned; V represents the number of values of feature A; D v represents the subset of feature A taking the v-th value; H(D) represents the entropy of the data set D; The expression formula of the reward function is: ; In the formula, R(s,a) represents the reward obtained by taking action a in state s; P pref (s,a) represents the preference reward score calculated based on the user preference model; Q base (s,a) represents the basic reward score calculated based on other factors; w1 represents the weight of the preference reward; w2 represents the weight of the basic reward.

[0063] It should be explained that, based on the object detection result and environmental data, construct a decision model, and optimize the decision model in combination with the user preference to obtain a reinforcement learning decision model, including the following steps: Step 1. Integrate the object detection result and environmental data, perform feature extraction, construct a unified feature vector, and obtain a state space.

[0064] The object detection results obtained from the SSD model include: the emotional state of the driver (such as "fatigue", "concentration", "relaxation"), the number and position of passengers in the cockpit (such as "someone in the driver's seat", "no one in the co-pilot seat").

[0065] The data provided by the environmental sensor includes: the current cockpit temperature (such as 25°C), humidity (such as 60%), and light intensity (such as 500 lux).

[0066] Extract the key features of the object detection result and environmental data and perform normalization processing, for example: The emotional state is encoded as a numerical value (e.g., "fatigue" = 0.1, "focus" = 0.5, "relaxation" = 0.9); The temperature, humidity, and light intensity are normalized to the range [0, 1].

[0067] Combine the above features into a unified feature vector, for example: S = [emotional state, number of passengers, temperature, humidity, light intensity]; Example state vector: S = [0.5, 1, 0.7, 0.6, 0.8].

[0068] Step 2: Based on the pre-configured action space, combine the reward function and the state space to build a decision-making model.

[0069] Define the set of executable actions, including: adjusting the air conditioning temperature (±2°C); adjusting the seat angle (±10°); changing the brightness of the in-vehicle lights (±20%), etc.

[0070] The expression formula of the reward function is: ; In the formula, R(s, a) represents the reward obtained by taking action a in state s; P pref (s, a) represents the preference reward score calculated based on the user preference model; Q base (s, a) represents the basic reward score calculated based on other factors; w1 represents the weight of the preference reward; w2 represents the weight of the basic reward.

[0071] Based on the above action space, combine the reward function and the state space to build a decision-making model.

[0072] Step 3: Collect the historical behavior data of the user's cockpit environment adjustment, build a user preference model based on the information gain algorithm, and combine the user preference model to train the decision-making model to obtain a reinforcement learning decision-making model.

[0073] The system records the user's manual adjustment behaviors in different scenarios, including: when the temperature is 28°C, the user lowers the air conditioning to 24°C. When the light intensity is 800 lux, the user lowers the light brightness to 30%. At the same time, record the context information (such as time, weather, number of people in the vehicle, etc.).

[0074] Remove abnormal data (such as duplicate records or obviously incorrect adjustments) from the historical behavior data of the user's cockpit environment adjustment. Standardize all data into a unified format, for example, normalize temperature, humidity, etc. to the range of [0, 1]. Extract features that can reflect user preferences, such as: action type (adjusting temperature, lighting, seat, etc.); action amplitude (such as temperature change range, lighting brightness change range); context information (such as time, weather conditions).

[0075] Calculate the information gain to evaluate the importance of each feature. The expression formula of the information gain algorithm is: ; ; In the formula, Gain(D, A) represents the information gain function, D represents the current node data set; A represents the feature to be partitioned; V represents the number of values of feature A; D v represents the subset of the v-th value of feature A; K represents the number of categories; p k represents the proportion of the k-th type of samples in the data; H(D) represents the entropy of the data set D.

[0076] If the user is more inclined to lower the temperature in a high-temperature environment, the information gain of the "temperature" feature is higher. According to the information gain ranking, select the most important features to build a user preference model.

[0077] Adjust the preference reward P pref (s, a) according to the user preference model. The expression formula of the reward function is: ; In the formula, R(s, a) represents the reward obtained by taking action a in state s; P pref (s, a) represents the preference reward score calculated based on the user preference model; Q base (s, a) represents the basic reward score calculated based on other factors; w1 represents the weight of the preference reward; w2 represents the weight of the basic reward.

[0078] If the user prefers a lower temperature, increase the reward for lowering the temperature in the relevant state. The weight w1 of the updated reward function is increased to 0.8 to further emphasize the user preference and ensure that it can better adapt to the personalized needs of the user.

[0079] S4. According to the real-time standard image data, combine the reinforcement learning decision model and the safety guarantee rules to obtain the real-time decision result, and adjust the cockpit according to the real-time decision result.

[0080] It should be noted that the safety guarantee rules include: safety priority rule, temperature control rule, seat adjustment rule, volume control rule, screen display rule and operation confirmation rule; The safety priority rule is used to ensure that all cockpit operations do not pose a threat to the safety of passengers or the vehicle. For example, during vehicle driving, setting adjustments that may distract the driver are prohibited.

[0081] The temperature control rule reasonably adjusts the cockpit temperature according to environmental data and user preferences, avoiding the impact on passengers' health caused by too high or too low temperature, and preventing excessive energy consumption of the equipment at the same time.

[0082] The seat adjustment rule, when adjusting the seat position or angle, ensures that passengers will not be squeezed or have other safety hazards, and avoids the normal use of other in-vehicle devices being affected by seat movement.

[0083] The volume control rule limits the maximum volume of the audio system, preventing hearing damage to passengers caused by too high volume, and ensuring that volume adjustment does not interfere with the driver's perception of surrounding environmental sounds.

[0084] The screen display rule standardizes the brightness, content display and operation logic of the in-vehicle screen, avoiding distraction of the driver caused by too high screen brightness or too complex display content.

[0085] The operation confirmation rule sends a confirmation prompt to the user before performing key operations (such as window opening / closing, large-scale seat adjustment, etc.), preventing unnecessary risks caused by misoperations.

[0086] When multiple rules are triggered simultaneously, the priority of the safety priority rule > the priority of the operation confirmation rule > the priority of other rules; if a collision risk is detected during seat adjustment, the adjustment is immediately aborted and the safety priority rule is executed; except for the safety priority rule, other rules can be forcibly cancelled through physical knobs / emergency switches.

[0087] For example, when a rear passenger attempts to adjust the seat significantly forward, the system analyzes through real-time standard image data and a reinforcement learning decision model and finds that although there are no obvious obstacles ahead, the seat adjustment may be close to the in-vehicle refrigerator. According to the seat adjustment rule, to ensure that the normal opening of the in-vehicle refrigerator door is not affected, the system allows the seat to be adjusted forward by a certain distance and synchronously displays a reminder message on the screen: "The seat has been adjusted. Please note to avoid affecting the normal use of the in-vehicle refrigerator." According to another embodiment of the present invention, as Figure 2 shown, there is also provided an intelligent control system for an automotive cockpit based on image acquisition, and the system includes: an image acquisition module 1, an image analysis module 2, a decision module 3 and a control module 4; The image acquisition module 1 is used to collect the original image data inside the cockpit, preprocess the original image data based on the PID control algorithm, and obtain the standard image data; The image analysis module 2 is used to train the SSD model using the multi-task loss function, analyze the standard image data, and obtain the target detection result; The decision-making module 3 is used to construct a decision-making model based on the target detection result and environmental data, and optimize the decision-making model in combination with user preferences to obtain a reinforcement learning decision-making model; The control module 4 is used to obtain the real-time decision result according to the real-time standard image data, in combination with the reinforcement learning decision-making model and safety guarantee rules, and adjust the cockpit according to the real-time decision result. In summary, by means of the above technical solutions of the present invention, through real-time image acquisition and analysis, the present invention can accurately identify the state inside the cockpit and the behavior of passengers, and provide more accurate environmental adjustment and personalized settings. Compared with the traditional data acquisition method based on fixed sensors, image data can provide a richer information dimension, improve the response speed and decision-making accuracy of the system; the present invention combines a reinforcement learning model, and this method can adapt to changing environments and user habits, continuously learn and optimize decision-making strategies to provide a control solution that better meets user needs, and can intelligently adjust the cockpit settings in different situations to enhance the user experience; the present invention learns the historical behavior and preferences of users, can provide personalized cockpit adjustment suggestions, meet the personalized needs of different users, improve user satisfaction, and enable users to feel more convenience and comfort during use.

[0088] Although the present invention has been disclosed above with preferred embodiments, the embodiments are only for the purpose of illustration and exemplification, and are not intended to limit the present invention. Those skilled in the art can make several modifications and refinements without departing from the spirit and scope of the present invention. The scope of protection claimed by the present invention shall be subject to what is described in the claims.

Claims

1. An intelligent control method for a vehicle cockpit based on image acquisition, characterized in that, The intelligent control method for vehicle cockpits based on image acquisition includes the following steps: S1. Collect the original image data inside the cockpit, and preprocess the original image data based on the PID control algorithm to obtain standard image data; S2. Train the SSD model using the multi-task loss function, analyze the standard image data, and obtain the object detection result; S3. Based on the object detection result and environmental data, construct a decision model, and optimize the decision model in combination with user preferences to obtain a reinforcement learning decision model; S4. According to the real-time standard image data, combine the reinforcement learning decision model and safety guarantee rules to obtain a real-time decision result, and adjust the cockpit according to the real-time decision result.

2. The intelligent control method for a vehicle cockpit based on image acquisition according to claim 1, characterized in that The step of collecting the original image data inside the cockpit and preprocessing the original image data based on the PID control algorithm to obtain standard image data includes the following steps: S11. Automatically adjust the camera parameters based on the PID control algorithm according to the illumination intensity of the current environment, and collect the original image data inside the cockpit in real time; S12. Process the original image data based on dynamic contrast enhancement and real-time noise reduction to obtain enhanced image data; S13. Adjust the enhanced image data to a unified size and convert it to a unified format to obtain standard image data.

3. The intelligent control method for a vehicle cockpit based on image acquisition according to claim 2, characterized in that The expression formula of the PID control algorithm is: u(t)=P+I+D; In the formula, u(t) represents the control output; P represents the proportional term; I represents the integral term; D represents the derivative term.

4. The intelligent control method for a vehicle cockpit based on image acquisition according to claim 1, wherein The step of initializing the SSD model using the multi-task loss function, analyzing the standard image data, and obtaining the object detection result includes the following steps: S21. Mark the object detection result of each image in the collected image data through manual marking to obtain the training set and test set of the object detection result marking; S22. Initialize the SSD model parameters using the multi-task loss function, and train and test the SSD model based on the training set and test set of the object detection result marking; S23. Input the standard image data into the SSD model after training and testing, and output through the SSD model to obtain the object detection result.

5. The intelligent control method for a vehicle cockpit based on image acquisition according to claim 4, characterized in that The step of initializing the SSD model parameters using the multi-task loss function includes the following steps: S221. The SSD model uses the pre-trained CNN as the backbone network, loads the pre-trained weights, and initializes the feature extraction part; S222. Add convolutional layers on the basis of the backbone network to generate feature maps of different scales; S223. Set prior boxes at each position of each feature map, and calculate the offset between the predicted box and the real box; S224. Calculate and update the SSD model parameters according to the multi-task loss function, and iteratively optimize the SSD model.

6. The intelligent control method for an automotive cockpit based on image acquisition according to claim 5, wherein, The expression formula of the multi-task loss function is: ； Wherein, x represents the prior box matching indicator; c represents the class prediction confidence; l represents the offset of the predicted box; g represents the coordinates of the ground truth box; N represents the number of matched prior boxes; α represents the localization loss weight; L(x, c, l, g) represents the multi-task loss function; L conf represents the classification loss function; L loc represents the localization loss function.

7. A method for intelligent control of an automotive cockpit based on image acquisition according to claim 1, characterized in that, The step of constructing a decision model based on the object detection result and environmental data, and optimizing the decision model in combination with user preferences to obtain a reinforcement learning decision model includes the following steps: S31. Integrate the object detection result and environmental data, and perform feature extraction to construct a unified feature vector to obtain the state space; S32. Based on the pre-configured action space, combine the reward function and the state space to construct a decision model; S33. Collect the historical behavior data of the user's cockpit environment adjustment, construct a user preference model based on the information gain algorithm, and train a decision-making model in combination with the user preference model to obtain a reinforcement learning decision-making model.

8. The intelligent control method for an automotive cockpit based on image acquisition according to claim 7, wherein, The steps of recording the historical behavior data of the user's cockpit environment adjustment, constructing a user preference model based on the information gain algorithm, and training a decision-making model in combination with the user preference model to obtain a reinforcement learning decision-making model include the following steps: S331. Collect the behavior data of the user's cockpit adjustment and the context information during the adjustment to obtain historical behavior data; S332. Clean and preprocess the collected historical data, convert it into a unified format, and perform feature extraction. Based on the information gain algorithm, construct a user preference model; S333. Optimize the reward function in the decision-making model according to the user preference model to obtain a reinforcement learning decision-making model.

9. A method for intelligent control of an automotive cockpit based on image acquisition according to claim 8, characterized in that, The expression formula of the information gain algorithm is: ； In the formula, Gain(D,A) represents the information gain function, D represents the current node data set; A represents the feature to be divided; V represents the number of values of feature A; Dv represents the subset of feature A taking the v-th value; H(D) represents the entropy of the data set D; The expression formula of the reward function is: ； Wherein, R(s,a) represents the reward obtained by taking action a in state s; P pref (s,a) represents the preference reward score calculated based on the user preference model; Q base (s,a) represents the basic reward score calculated based on other factors; w1 represents the weight of the preference reward; w2 represents the weight of the basic reward.

10. An intelligent control system for a vehicle cockpit based on image acquisition, which is used to implement an intelligent control method for a vehicle cockpit based on image acquisition according to any one of claims 1-9, characterized in that, The system includes: an image acquisition module, an image analysis module, a decision-making module, and a control module; The image acquisition module is used to collect the original image data in the cockpit, preprocess the original image data based on the PID control algorithm to obtain standard image data; The image analysis module is used to train the SSD model using the multi-task loss function, analyze the standard image data, and obtain the target detection result; The decision-making module is used to construct a decision-making model based on the target detection result and the environmental data, and optimize the decision-making model in combination with the user preference to obtain a reinforcement learning decision-making model; The control module is used to obtain the real-time decision result according to the real-time standard image data, in combination with the reinforcement learning decision-making model and the safety guarantee rules, and adjust the cockpit according to the real-time decision result.

Images