A vehicle air conditioning control method based on visual perception

Abstract

The application discloses a vehicle air conditioner control method based on visual perception, and the method comprises the following steps: collecting image data before and after passengers get on the vehicle; evaluating the wet degree of the passengers based on the image data before and after the passengers get on the vehicle; and controlling the vehicle air conditioner according to the wet degree of the passengers. Through the method, the inconvenience and discomfort of the passengers after being wet in the rain are solved, and the vehicle can intelligently perceive the wet degree of the passengers, so that the passengers can have a more comfortable experience of using the vehicle air conditioner.

Description

Technical Field

[0001] This invention relates to the field of vehicle air conditioning control, and more particularly to a vehicle air conditioning control method based on visual perception. Background Technology

[0002] Nowadays, vehicle air conditioning systems have become an essential component of vehicles, and their performance is an important factor in vehicle selection. Especially in rainy weather, passengers often get soaked while rushing to board a vehicle, and wet clothes, skin, or hair can be very uncomfortable and even affect health. Manually turning on the vehicle's air conditioning, adjusting the vents, and controlling the temperature are cumbersome and difficult to adapt to the increasingly advanced environmental and passenger-friendly perception capabilities of automobiles. Summary of the Invention

[0003] The purpose of this invention is to provide a vehicle air conditioning control method based on visual perception, which is applicable to the control of vehicle air conditioning in rainy weather. It can assess the degree of wetness of passengers based on image data before and after passengers get in the vehicle, and perform corresponding vehicle air conditioning control to achieve quick and comfortable drying of passengers' wet areas, providing passengers with a more convenient and comfortable vehicle air conditioning experience.

[0004] According to a first aspect of the present invention, a vision-based vehicle air conditioning control method is provided, the control method being applicable to rainy weather control of vehicle air conditioning, the method comprising: Collect image data of passengers before and after boarding; The degree of wetness of the passenger is assessed based on the video data of the passenger before and after boarding the vehicle; The vehicle's air conditioning should be controlled accordingly, at least based on the degree of wetness of the passengers.

[0005] Optionally, the collection of image data before and after passengers board the vehicle includes: The vehicle's interior image data is collected by an image acquisition device deployed inside the vehicle. Based on the interior image data, a face detection algorithm is used to detect and record the facial feature data of the passenger's first appearance and record the corresponding timestamp. The image data after the timestamp in the interior image data is used as the image data of the passenger after boarding the vehicle. External image data of the vehicle is collected by an image acquisition device deployed on the vehicle body, and image data within a specified time period back from the timestamp is extracted from the external image data as the image data before the passenger gets on the vehicle.

[0006] Optionally, assessing the passenger's degree of wetness based on image data before and after boarding includes: Based on the image data of the passenger before boarding the vehicle, rain gear in the image is identified, and the first confidence level of getting wet, C1, is calculated using the accuracy of rain gear identification; Based on the image data of the passenger after boarding the vehicle, detect whether the passenger's hair and / or skin are wet, and calculate the second wetness confidence C2; The overall wetness confidence level C is calculated by combining the first wetness confidence level C1 and the second wetness confidence level C2, and the degree of wetness of the passenger is evaluated using the overall wetness confidence level. Optionally, the step of identifying rain gear in the image based on the image data of the passenger before boarding, and calculating the first confidence level C1 of getting wet using the accuracy of the rain gear identification, includes: The rain gear is identified using an object recognition model based on the image data of the passenger before boarding the vehicle. The recognition accuracy p of the rain gear is recorded, and the first confidence level of getting wet is calculated based on the recognition accuracy p. The formula for calculating the first confidence level of wetness is C1 = 1 - Where n is the total number of all rain gear identified in the image data, n>=0, and is a natural number. This represents the accuracy of identifying the i-th rain gear. .

[0007] Optionally, the step of detecting whether the passenger's hair and / or skin is wet based on the image data after the passenger boards the vehicle, and calculating the second wetness confidence level C2, includes: Image data of the passenger's hair and / or skin were segmented from the image data of the passenger after boarding the vehicle using an object segmentation model; The confidence level C2 of the wetness of the image data is evaluated by comparing it with stored image data of the passenger's dry hair and / or skin.

[0008] Optionally, the step of calculating the overall wetness confidence C based on the first wetness confidence C1 and the second wetness confidence C2 includes: obtaining the passenger's overall wetness confidence C by linearly combining the first wetness confidence C1 and the second wetness confidence C2; The overall wetness confidence level C = k1*C1 + k2*C2, where k1 + k2 = 1.

[0009] Optionally, assessing the passenger's degree of wetness using the overall wetness confidence level C includes: The degree of wetness is determined by the wetness value L= To determine, among which and The minimum and maximum values ​​of the overall wetness confidence level C are calculated under laboratory conditions to simulate the wetness process of passengers before and after boarding the vehicle. According to the wetness degree value of the passenger, the wetness degree of the passenger is divided into slight, moderate and severe: When a first preset threshold g1 < L ≤ a second preset threshold g2, the wetness degree is determined as slight wetness; When the second preset threshold g2 < L ≤ a third preset threshold g3, the wetness degree is determined as moderate wetness; When L > the third preset threshold g3, the wetness degree is determined as severe wetness; Different wetness degrees correspond to different air-conditioning controls.

[0010] Optionally, evaluating the wetness degree of the passenger by using the overall wetness confidence further comprises: if the wetness degree value of the passenger is lower than the first preset threshold g1, requesting the passenger to confirm his / her own wetness degree by means of voice information or text information.

[0011] Optionally, the step of requesting the passenger to confirm his / her own wetness degree by means of voice information or text information if the wetness degree value L of the passenger is lower than the first preset threshold g1 comprises: when the wetness degree value of the passenger is lower than the first preset threshold g1, interacting with the passenger by means of voice information and / or text information to determine whether the current wetness degree of the passenger is accurate, and receiving the wetness degree fed back by the passenger if not.

[0012] Optionally, correspondingly controlling the vehicle air conditioner at least according to the wetness degree of the passenger further comprises: correspondingly controlling the vehicle air conditioner by comprehensively combining the internal temperature, the external temperature of the vehicle and the wetness degree of the passenger.

[0013] Optionally, the step of correspondingly controlling the vehicle air conditioner by comprehensively combining the internal temperature, the external temperature of the vehicle and the wetness degree of the passenger comprises: when it is determined that the wetness degree of the passenger is one of slight, moderate and severe, further determining the temperature difference between the internal temperature and the external temperature of the vehicle and performing corresponding air conditioner control, wherein the air conditioner control comprises one or more of the following: the working mode of the air conditioner, opening and closing of an air door of an air outlet, air outlet duration, sweeping direction of the air outlet and air outlet temperature.

[0014] According to a second aspect of the present invention, there is provided an electronic device for a vehicle air conditioner control method based on visual perception, comprising: at least one processor and a memory; the memory stores computer execution instructions; and the at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the method according to any one of the first aspect.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The vehicle air conditioning control method and electronic device based on visual perception provided by this invention solve the problems of existing technology where the vehicle air conditioning cannot automatically sense the wetness of passengers in rainy weather, and the process of manually turning on the vehicle air conditioning and adjusting the air outlets and temperature is cumbersome.

[0016] This system solves the problem of intelligent sensing when passengers enter a vehicle wet in rainy weather. Based on visual perception technology and a wetness level model, it automatically assesses the degree of wetness of passengers and controls the vehicle's air conditioning accordingly. This provides passengers with a convenient and comfortable drying experience, greatly facilitating the vehicle's air conditioning to dry wet passengers and ensuring their health. Attached Figure Description

[0017] The above and other objects, features, and advantages of the present invention will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings. The drawings described below are merely some embodiments disclosed in the present invention; those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0018] Figure 1 A flowchart illustrating an embodiment of the present invention is shown; Figure 2 The network structure diagram of YOLOv3 according to an embodiment of the present invention is shown; Figure 3 A diagram of the U-Net network structure according to an embodiment of the present invention is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. “Multiple” generally includes at least two, but does not exclude the inclusion of at least one.

[0021] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0022] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0023] To further illustrate the technical solutions in this invention, the following specific embodiments are provided.

[0024] Example This invention provides a vehicle air conditioning control method based on vision perception, such as... Figure 1 As shown, the vehicle air conditioning control method based on vision perception according to this embodiment of the invention may include at least the following three steps: S1, S2, and S3. S1 collects image data of passengers before and after boarding the vehicle.

[0025] S2 assesses the degree of wetness of passengers based on video data before and after they board the vehicle.

[0026] S3, at least according to the degree of wetness of the passengers, controls the vehicle's air conditioning accordingly.

[0027] in: Step S1: Collect image data of passengers before and after boarding the vehicle.

[0028] Cameras inside and outside the vehicle collect and save data in real time until a face is detected in the data collected inside the vehicle. Using this time as a timestamp, image data before and after the passenger boards the vehicle is extracted. This process includes the following two steps, S11 and S12: Step S11: One or more cameras deployed at different locations on the vehicle body collect and save external image data in real time; one or more cameras deployed at different locations inside the vehicle collect and save internal image data in real time. Based on the image data collected in real time by the in-vehicle cameras, a face detection algorithm is used to detect the time when a passenger's face is first detected, and this time is saved as a timestamp. Image data after this timestamp is extracted from the internal image data and saved as the image data after the passenger boards the vehicle.

[0029] Step S12: Extract image data from the saved external image data that traces back a specified time length from the recorded timestamp and save it as image data before the passenger boards the vehicle.

[0030] For example, if a passenger enters the vehicle at 13:00 and is detected by the in-vehicle camera's face, then 13:00 is recorded as a timestamp. The video data continuously recorded by the in-vehicle camera after this timestamp is saved, forming the video data after the passenger boarded the vehicle. Since the external cameras are also continuously collecting external video data, the frame at this moment is located in the saved external video using the 13:00 timestamp, and then a 10-minute video file is extracted up to 12:50. This video file is used as the video data before the passenger boarded the vehicle.

[0031] Step S2: Assess the degree of wetness of passengers based on video data before and after they board the vehicle.

[0032] Based on the obtained image data of passengers before and after boarding, a visual algorithm is used to characterize the degree of wetness of the passengers. This is divided into three steps: S21, S22, and S23. S21, based on the image data of the passenger before boarding the vehicle, identify the rain gear in the image, and calculate the first confidence level of getting wet using the accuracy of the rain gear identification.

[0033] S22, Detect whether the passenger's hair and / or skin are wet based on the image data after the passenger boards the vehicle, and calculate the second wetness confidence level.

[0034] S23, calculate the overall wetness confidence based on the first wetness confidence and the second wetness confidence, and use the overall wetness confidence to assess the degree of wetness of the passenger.

[0035] in: Step S21: Identify rain gear in the image based on the image data before the passenger boards the vehicle, and calculate the first confidence level of getting wet using the accuracy of rain gear identification.

[0036] Using each frame of image data from the video data before passengers board the vehicle, an object detection algorithm is used to detect rain gear. Preferably, rain gear includes umbrellas, raincoats, rain boots, rain shoes, tents, waterproof pants, and nano raincoats / umbrellas. The object detection model uses the YOLOv3 object detection model and is trained using a public rain gear dataset and a custom-made rain gear dataset as training sets to obtain a YOLOv3-based rain gear detection model.

[0037] like Figure 2 The YOLOv3 network model shown is composed of three parts: S211, S212, and S213, which are Darknet53, FPN, and YOLO Head, respectively.

[0038] S211 and Darknet53 can be considered the backbone feature extraction network of YOLOv3. The input image is first processed for feature extraction within Darknet53; the extracted features are called feature layers, which are the feature sets of the input image. In the backbone, three feature layers are used for the next step of network construction; these three feature layers are called effective feature layers.

[0039] S212,FPN can be called an enhanced feature extraction network for YOLOv3. The three effective feature layers obtained in the backbone are fused in this part, and the purpose of feature fusion is to combine feature information at different scales. In the FPN part, the already obtained effective feature layers are used to continue extracting features.

[0040] S213, the Yolo Head, is the classifier and regressor of YoloV3. Through Darknet53 and FPN, it has obtained three enhanced effective feature layers with shapes of (52, 52, 128), (26, 26, 256), and (13, 13, 512), respectively. Each feature layer has width, height, and number of channels. The feature map can then be viewed as a collection of feature points, each with a number of channels. The Yolo Head's function is to determine whether each feature point corresponds to an object.

[0041] Using the trained YOLOv3-based rain gear detection model, all rain gears appearing in each frame of the image data before passengers board the vehicle are detected. The recognition accuracy p of each detected rain gear is calculated, which represents the feature similarity with the trained rain gear model and ranges from (0, 1). The larger the p of the currently detected rain gear, the higher the probability that the currently detected object is a rain gear and the more accurate the recognition effect.

[0042] Here, the p-value is used to reflect the first wetness confidence level C1, i.e., C1 = 1 - Where n is the total number of all rain gear identified in the image data, n>=0, and is a natural number. This represents the accuracy of identifying the i-th rain gear. The probability of getting wet is indicated by the degree of certainty in the detection of rain gear. For example, if three rain gears are detected in the footage before passenger A boards the vehicle, the accuracy of identifying the first rain gear is... The accuracy of identifying the second rain gear is The accuracy of identifying the third rain gear was... ,so Then the confidence level for passenger A's first wetness is C1 = 1 - =1-0.21=0.79; Similarly, if five rain pelts are detected in the footage before passenger B boards the vehicle, the accuracy of identifying the first rain pelt is: The accuracy of identifying the second rain gear is The accuracy of identifying the third rain gear was... The accuracy of identifying the fourth rain gear was... The accuracy of identifying the fifth rain gear was... ,so Then the confidence level for passenger B's first wetness is C1 = 1 - =1-0.36288=0.63712; The comparison between passengers A and B above shows that passenger A had fewer rain gears detected, and the accuracy of rain gear identification was relatively low. This indicates that passenger A used less rain gear and was more likely to get wet in the rain. On the other hand, passenger B had more rain gears detected, and the average accuracy of rain gear identification was higher than that of passenger A. This indicates that passenger B used more rain gear, provided more protection from the rain, and was more likely to not get wet. Therefore, passenger B's first confidence level of getting wet is lower, meaning the probability of getting wet is lower and the likelihood is less.

[0043] The above examples illustrate the relationship between the first wetness confidence level C1 and the number of rain gears and the accuracy of rain gear identification.

[0044] Step S22: Detect whether the passenger's hair and / or skin is wet based on the image data after the passenger boards the vehicle, and calculate the second wetness confidence level.

[0045] Based on image data of passengers after boarding, hair and / or skin images are segmented. The second wetness confidence score is calculated by comparing the image brightness of the segmented hair and / or skin images with that of dry hair and skin. This process can be divided into two steps, S221 and S222: Step S221: Segment hair and / or skin images based on image data after the passenger boards the vehicle.

[0046] Based on each frame of the image data after the passenger boards the vehicle, the U-Net object segmentation model is used to segment the passenger's hair and skin data. The model is trained using publicly available human hair and skin datasets and custom-made hair and skin datasets to obtain the U-Net-based object segmentation model.

[0047] like Figure 3 The U-Net network structure shown mainly consists of convolutional layers, max pooling layers (downsampling), deconvolutional layers (upsampling), and the ReLU nonlinear activation function.

[0048] The entire network process is as follows: Max pooling layer (downsampling) process: Convert the input image to a 572*572 grayscale image.

[0049] First, it undergoes two convolution operations with 3*3*64 kernels to become a size of 568*568*64.

[0050] Then a 2x2 max pooling operation is performed, resulting in a 248x248x64 convolution. Each 3x3 convolution is followed by a ReLU non-linear transformation.

[0051] Repeat the above process 4 times, that is, perform (3*3 convolution + 2*2 pooling) 4 times. In the first 3*3 convolution operation after each pooling operation, the number of 3*3 convolution kernels increases exponentially.

[0052] When the image reaches the lowest level, after the fourth max pooling operation, it becomes 32*32*512 in size. Then, two more 3*3*1024 convolution operations are performed, resulting in a final size of 28*28*1024.

[0053] Deconvolutional layer (upsampling) process: The image size is 28*28*1024. First, a 2*2 deconvolution operation is performed to change the image size to 56*56*512.

[0054] Then, the image before the corresponding max pooling layer is copied and cropped, and concatenated with the image obtained from deconvolution to obtain an image of size 56*56*1024, and then a 3*3*512 convolution operation is performed.

[0055] Repeat the above process 4 times, that is, perform (2*2 deconvolution + 3*3 convolution)*4 times. In the first 3*3 convolution operation after each concatenation, the number of 3*3 convolution kernels is reduced by a factor of two.

[0056] When the top layer is reached, that is, after the 4th deconvolution, the image becomes 392*392*64 in size. It is then copied, cropped, and stitched together to get a size of 392*392*128. Then, two more 3*3*64 convolution operations are performed.

[0057] The final image size is 388*388*64, and then a 1*1*2 convolution operation is performed.

[0058] Using a U-Net-based hair and / or skin segmentation network, and taking image data based on passengers boarding the vehicle as input, segmented hair and / or skin images can be obtained.

[0059] Step S222: Calculate the second wetness confidence score by comparing the segmented hair and / or skin images with pre-stored dry hair and skin images taken inside the vehicle to determine image similarity. Using the mean squared error as a similarity evaluation parameter, the mean squared error e is calculated based on the segmented hair and / or skin images and pre-stored dry hair and skin images taken inside the vehicle. The mean squared error e is calculated for all segmented hair and / or skin images, and the maximum mean squared error is counted. and minimum value The second confidence level of wetting is expressed as C2= Here, n represents the total number of images of hair and skin that have been segmented. From the relationship between the second wetness confidence level and the mean squared error of the images, we can see that when the image similarity is high, meaning the passenger's skin or hair detected inside the vehicle is similar in dryness to when the skin or hair is dry, it indicates that the passenger's hair or skin is relatively dry, and the probability of being wet by rain is lower, meaning the C2 value is smaller.

[0060] Step S23: Calculate the overall wetness confidence based on the first wetness confidence and the second wetness confidence, and use the overall wetness confidence to assess the degree of wetness of the passenger.

[0061] The degree of wetness of a passenger can be assessed using the first and second wetness confidence levels. This process requires the following three steps: S231, S232, and S233. S231, calculate the overall confidence level of passengers getting wet.

[0062] S232, construct the degree of wetting based on the overall wetting confidence level.

[0063] S233, the degree of wetness is classified by the wetness value.

[0064] in: Step S231: Calculate the overall confidence level of the passenger getting wet.

[0065] The first confidence level C1 and the second confidence level C2 for getting wet were calculated using image data before and after boarding the vehicle. The overall confidence level C for getting wet, which reflects the likelihood of a passenger getting wet, is obtained by linearly combining the first confidence level C1 and the second confidence level C2. That is, C = k1*C1 + k2*C2, where k1 + k2 = 1, preferably k1 = 0.4 and k2 = 0.6.

[0066] Step S232: Construct a wetness degree value based on the overall wetness confidence level.

[0067] Since the overall wetness confidence score is not intuitive for evaluating the degree of wetness, a wetness degree value is constructed as an evaluation parameter for assessing the degree of wetness, i.e., the wetness degree value L= ,in and These are the minimum and maximum values ​​of the overall wetness confidence level calculated under laboratory conditions to simulate the wetness process of passengers before and after boarding the vehicle.

[0068] Step S233: Classify the degree of wetness based on the wetness value.

[0069] Based on the degree of wetness, the degree of wetness is divided into three levels: slight, moderate, and severe, for easy assessment and control of automotive air conditioning. The grading rules are as follows: when At 30, the degree of wetting is slight wetting.

[0070] when At 80 degrees, the degree of wetting is considered normal wetting.

[0071] when At that time, the degree of wetting was described as severe wetting.

[0072] Step S3: Control the vehicle's air conditioning according to at least the degree of wetness of the passengers.

[0073] The system controls the vehicle's air conditioning system's operating mode, vent opening and closing, airflow duration, vent sweeping direction, and airflow temperature by taking into account the vehicle's internal and external temperatures, as well as the passengers' level of humidity.

[0074] For example: When passengers are slightly wet, and the current season is summer, if the interior temperature of the vehicle is 5 degrees Celsius higher than the exterior temperature, the air conditioning will be set to cooling mode, the air vents will be opened, and the airflow will continue for 5 minutes. The airflow direction will be towards the passengers' seats, and the air temperature will be 25 degrees Celsius. If the interior temperature of the vehicle is 5 degrees Celsius lower than the exterior temperature, the air conditioning will be set to heating mode, the air vents will be opened, and the airflow will continue for 5 minutes. The airflow direction will be towards the passengers' seats, and the air temperature will be 27 degrees Celsius.

[0075] When the passenger is moderately wet, and the current season is spring, if the vehicle's interior temperature is 5 degrees Celsius higher than the exterior temperature, the air conditioning will be set to heating mode, the air vents will be opened, and the airflow will continue for 10 minutes, with the airflow direction pointing towards the passenger's seat and the air temperature set to 28 degrees Celsius. If the vehicle's interior temperature is 5 degrees Celsius lower than the exterior temperature, the air conditioning will be set to heating mode, the air vents will be opened, and the airflow will continue for 15 minutes, with the airflow direction pointing towards the passenger's seat and the air temperature set to 30 degrees Celsius. When the passenger is severely wet, and the current season is autumn, if the vehicle's interior temperature is higher than the exterior temperature, the air conditioning will be set to heating mode, the air vents will be opened, and the airflow will continue for 20 minutes. For the first 10 minutes, the airflow direction will be towards the passenger's seat, and for the next 10 minutes, it will be towards the seat. The first 15 minutes will have a strong airflow, and the next 5 minutes a weak airflow. If the vehicle's interior temperature is lower than the exterior temperature, the air conditioning will be set to heating mode, and the airflow will continue for 30 minutes. For the first 20 minutes, the airflow direction will be towards the passenger's seat, and for the next 10 minutes, it will be towards the seat. The first 15 minutes will have a strong airflow, and the next 15 minutes a weak airflow. The airflow temperature for the first 15 minutes will be 30 degrees Celsius, and for the next 15 minutes, it will be 25 degrees Celsius.

[0076] According to one or more embodiments of the present invention, the present invention also provides a non-transitory computer-readable storage medium having program instructions stored thereon. When the program instructions are executed by one or more processors, the one or more processors are used to implement the methods or processes shown in the various embodiments of the present invention above. According to one embodiment of the present invention, the method of the vehicle visual perception of passenger wetness level using the model of the present invention is stored as a program in a readable storage medium. In addition to storing the program that implements each function in a computer, it can also be stored in a recording medium such as a USB flash drive, portable hard drive, optical disc, or hard disk.

[0077] According to one or more embodiments of the present invention, the present invention also provides an electronic device for a vehicle air conditioning control method based on vision perception, which includes one or more processors and a non-transitory computer-readable storage medium storing program instructions. When the one or more processors execute the program instructions, the one or more processors are used to implement the methods or processes shown in the various embodiments of the present invention above.

[0078] According to one or more embodiments of the present invention, the visual perception method for passenger wetness can implement the processing of the control method described above using encoded instructions (e.g., computer and / or machine-readable instructions) stored on a non-transitory computer and / or machine-readable medium (e.g., hard disk drive, flash memory, read-only memory, optical disk, digital multifunction disk, cache, random access memory, and / or any other storage device or storage disk). Information for any time period (e.g., extended time periods, permanent, transient instances, temporary caches, and / or information caches) is stored in the non-transitory computer and / or machine-readable medium. As used herein, the term "non-transitory computer-readable medium" is explicitly defined to include any type of computer-readable storage device and / or storage disk, excluding propagated signals and transmission media.

[0079] According to one or more embodiments of the present invention, the vehicle-mounted system or control module of an automobile may include one or more processors and may also internally include a non-transitory computer-readable medium. Specifically, in the vehicle data acquisition device (vehicle control system or control module) of the present invention, a microcontroller (MCU) may be included, which is arranged in the vehicle for various operations of acquiring data from the air conditioner and implementing various functions. The processor of the air conditioner with data acquisition function may be, such as, but not limited to, one or more single-core or multi-core processors. The processor (one or more) may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, etc.). The processor may be coupled thereto and / or may include a memory / storage device, and may be configured to execute instructions stored in the memory / storage device to implement various applications and / or operating systems running on the controller of the present invention.

[0080] The accompanying drawings and detailed description of the invention, cited above as examples, serve to explain the invention but do not limit its meaning or scope as described in the claims. Therefore, those skilled in the art can readily make modifications from the above description. Furthermore, those skilled in the art can remove some of the components described herein without degrading performance, or add other components to improve performance. Additionally, those skilled in the art can change the order of steps in the method described herein depending on the process or equipment environment. Therefore, the scope of the invention should not be determined by the embodiments described above, but rather by the claims and their equivalents.

[0081] Although the invention has been described in conjunction with embodiments now considered to be achievable, it should be understood that the invention is not limited to the disclosed embodiments, but rather is intended to cover various modifications and equivalent configurations included within the spirit and scope of the appended claims.

Claims

1. A visual perception-based vehicle air conditioning control method, the control method being applied to rain control of a vehicle air conditioner, characterized by, The method includes: Collect image data of passengers before and after boarding; The degree of wetness of the passenger is assessed based on the video data of the passenger before and after boarding the vehicle; The vehicle's air conditioning should be controlled accordingly, at least based on the degree of wetness of the passengers. The assessment of the passenger's degree of wetness based on the image data before and after the passenger boarded the vehicle includes: Based on the image data of the passenger before boarding the vehicle, rain gear in the image is identified, and the first confidence level of getting wet, C1, is calculated using the accuracy of rain gear identification; Based on the image data of the passenger after boarding the vehicle, detect whether the passenger's hair and / or skin are wet, and calculate the second wetness confidence C2; The overall wetness confidence level C is calculated by combining the first wetness confidence level C1 and the second wetness confidence level C2, and the degree of wetness of the passenger is evaluated using the overall wetness confidence level.

2. The method of claim 1, wherein, The collected image data of passengers before and after boarding includes: The vehicle's interior image data is collected by an image acquisition device deployed inside the vehicle. Based on the interior image data, a face detection algorithm is used to detect and record the facial feature data of the passenger's first appearance and record the corresponding timestamp. The image data after the timestamp in the interior image data is used as the image data of the passenger after boarding the vehicle. External image data of the vehicle is collected by an image acquisition device deployed on the vehicle body, and image data within a specified time period back from the timestamp is extracted from the external image data as the image data before the passenger gets on the vehicle.

3. The method according to claim 1, characterized in that, The step of detecting whether the passenger's hair and / or skin is wet based on the image data after the passenger boards the vehicle, and calculating the second wetness confidence level C2, includes: Image data of the passenger's hair and / or skin were segmented from the image data of the passenger after boarding the vehicle using an object segmentation model; The confidence level C2 of the wetness of the image data is evaluated by comparing it with stored image data of the passenger's dry hair and / or skin.

4. The method according to claim 1, characterized in that, The step of calculating the overall wetness confidence C based on the first wetness confidence C1 and the second wetness confidence C2 includes: obtaining the overall wetness confidence C of the passenger by linearly combining the first wetness confidence C1 and the second wetness confidence C2; The overall wetness confidence level C = k1*C1 + k2*C2, where k1 + k2 = 1.

5. The method according to claim 1, characterized in that, The assessment of the passenger's degree of wetness using the overall wetness confidence level C includes: The degree of wetness is determined by the wetness value L= To determine, among which and The minimum and maximum values ​​of the overall wetness confidence level C are calculated under laboratory conditions to simulate the wetness process of passengers before and after boarding the vehicle. Based on the passenger's level of wetness, the degree of wetness is categorized as slightly wet, moderately wet, or severely wet. When the first preset threshold g1 < When the second preset threshold g2 is reached, the degree of wetting is determined to be slight wetting. when When the third preset threshold g3 is reached, the degree of wetting is determined to be normal wetting. when At that time, the degree of wetting was determined to be severe wetting; Different levels of wetness correspond to different air conditioning controls.

6. The method according to claim 5, characterized in that, The method of assessing the passenger's degree of wetness using the overall wetness confidence score also includes: If the passenger's degree of wetness is lower than the first preset threshold g1, the system will request the passenger to confirm their degree of wetness via voice or text message.

7. The method according to claim 6, characterized in that, If the passenger's degree of wetness is... If the temperature is below the first preset threshold g1, the system will request the passenger to confirm their level of wetness via voice or text message, including: When the passenger's wetness level is lower than the first preset threshold g1, the system interacts with the passenger via voice and / or text messages to determine whether the passenger's current wetness level is accurate. If not, the system receives the passenger's feedback on the wetness level.

8. The method according to claim 1, characterized in that, The method of controlling the vehicle air conditioning according to at least the degree of wetness of the passenger also includes: The vehicle's air conditioning is controlled accordingly, taking into account the vehicle's internal temperature, external temperature, and the degree of wetness of the passengers.

9. The method according to claim 8, characterized in that, The method of controlling the vehicle's air conditioning based on the vehicle's internal temperature, external temperature, and the passengers' level of wetness includes: When the degree of wetness of the passenger is determined to be slightly, moderately, or severely, the temperature difference between the interior and exterior temperatures of the vehicle is further determined and the air conditioning is controlled accordingly. The air conditioning control includes one or more of the following: the working mode of the air conditioning, the opening and closing of the air outlet damper, the duration of airflow, the direction of airflow from the air outlet, and the air outlet temperature.

10. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1-9.

Images