Cockpit sunshade control method, system and device based on machine vision and medium
By recognizing the driver's eye features using machine vision and automatically adjusting the light transmittance of the sunshade dimming film using a CNN-LSTM network, the safety hazards and user experience issues of manually adjusting the sunshade under sunlight are solved. This achieves real-time automatic control of the cockpit sunshade, improving driving safety and experience.
Patent Information
- Application Number
- CN202511057425.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-30
- Publication Date
- 2025-11-11
AI Technical Summary
In existing technologies, drivers need to manually adjust the sun visor when exposed to sunlight, which leads to safety hazards and a poor driving experience, especially in cloudy weather with frequent light changes, affecting driving safety.
By acquiring images of the driver's eye area through machine vision, extracting temporal features of light and dark, muscle, and pupil, and using a pre-trained CNN-LSTM hybrid neural network to identify light transmission adjustment needs, the light transmittance of the sunshade and dimming film on the upper part of the windshield is automatically adjusted.
It enables real-time automatic control of the cockpit sunshade, improving the efficiency and accuracy of sunshade control, and enhancing the driver's driving experience and driving safety.
Smart Images
Figure CN120921883A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of vehicle control technology, and in particular to a cockpit sunshade control method, system, device and medium based on machine vision. Background Technology
[0002] On sunny days, sunlight may shine through the windshield and into the driver's eyes while the car is in motion, affecting their vision. In such cases, the driver needs to manually adjust the sun visor to block the sunlight. However, at high speeds, manually adjusting the sun visor may pose a safety hazard. In cloudy weather, with frequent changes in light and darkness, the driver may need to repeatedly adjust the sun visor, impacting both the driving experience and safety. Summary of the Invention
[0003] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.
[0004] Therefore, one objective of this invention is to provide a machine vision-based cockpit sunshade control method. This method enables real-time and automatic cockpit sunshade control without requiring manual operation by the driver, thereby improving the efficiency and accuracy of cockpit sunshade control, as well as enhancing the driver's driving experience and driving safety.
[0005] Another objective of this invention is to provide a cockpit sunshade control system based on machine vision.
[0006] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include:
[0007] In a first aspect, embodiments of the present invention provide a cockpit sunshade control method based on machine vision, comprising the following steps:
[0008] Obtain an image of the target driver's eye region, and extract the temporal features of the brightness and darkness of the eye region, the temporal features of the eye muscles, and the temporal features of the pupil based on the image of the eye region.
[0009] The temporal features of light and dark in the eye region, the temporal features of the eye muscles, and the temporal features of the pupil are input into a pre-built light transmittance adjustment requirement recognition model to obtain the target light transmittance adjustment value.
[0010] Adjust the light transmittance of the sunshade and dimming film located in the upper area of the windshield according to the target light transmittance adjustment value.
[0011] Furthermore, in one embodiment of the present invention, the step of acquiring an image of the target driver's eye region and extracting temporal features of eye region brightness and darkness, temporal features of eye muscles, and temporal features of pupil size from the image of the eye region specifically includes:
[0012] The facial image information of the target driver is acquired by a camera device installed in the cockpit. Edge detection and texture analysis are performed on the facial image information to obtain the eye contour. The eye region image is extracted from the facial image information based on the eye contour.
[0013] The brightness value distribution matrix of each pixel in the eye region image is determined, and the brightness value distribution matrix corresponding to multiple consecutive frames of the eye region image is processed temporally to obtain the brightness and darkness temporal features of the eye region.
[0014] Based on the eye region image, extract the periocular muscle texture features and determine the eyelid opening and closing degree. Perform temporal processing on the periocular muscle texture features and eyelid opening and closing degree corresponding to multiple consecutive frames of the eye region image to obtain the temporal features of the eye muscles.
[0015] The eye region image is binarized and the pupil contour is detected. The pupil position and pupil diameter are determined based on the pupil contour. The pupil position and pupil diameter corresponding to multiple consecutive frames of the eye region image are temporally processed to obtain the temporal features of the eyeball pupil.
[0016] Furthermore, in one embodiment of the present invention, the light transmittance adjustment requirement recognition model is trained through the following steps:
[0017] Obtain the initial light transmittance of the sunshade and dimming film on the test vehicle;
[0018] Obtain eye region image samples of test personnel inside the test vehicle, and extract eye region light and dark temporal feature samples, eye muscle temporal feature samples, and eyeball pupil temporal feature samples based on the eye region image samples.
[0019] Obtain the optimal light transmittance after the test personnel adjust the sunshade and dimming film of the test vehicle to the most comfortable state;
[0020] Training samples are generated based on the temporal feature samples of light and dark in the eye region, the temporal feature samples of the eye muscles, and the temporal feature samples of the pupil. A transmittance adjustment value label is determined based on the difference between the initial transmittance and the optimal transmittance. A training dataset is then constructed based on the training samples and the corresponding transmittance adjustment value label.
[0021] The training dataset is input into a pre-built CNN-LSTM hybrid neural network for training to obtain the trained light transmission adjustment requirement recognition model.
[0022] Furthermore, in one embodiment of the present invention, the CNN-LSTM hybrid neural network includes an input layer, a CNN convolutional layer, a feature fusion layer, an LSTM layer, an attention layer, and an output layer. The input layer is used to input the training samples. The CNN convolutional layer is used to extract features from the training samples to obtain local temporal features. The feature fusion layer is used to fuse the local temporal features to obtain fused temporal features. The LSTM layer is used to generate a hidden state sequence based on the fused temporal features. The attention layer is used to dynamically assign weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism. The output layer is used to map the dynamically weighted hidden state sequence into a transmittance adjustment value recognition result.
[0023] Furthermore, in one embodiment of the present invention, the step of inputting the training dataset into a pre-constructed CNN-LSTM hybrid neural network for training to obtain the trained light transmittance adjustment requirement recognition model specifically includes:
[0024] The training samples are input through the input layer;
[0025] The CNN convolutional layer extracts features from the temporal features of the light and dark areas of the eye region, the temporal features of the eye muscles, and the temporal features of the pupil, respectively, to obtain local features of the light and dark areas of the eye region, local features of the eye muscles, and local features of the pupil.
[0026] The feature fusion layer performs feature fusion on the local features of light and dark areas in the eye region, the local features of the eye muscles, and the local features of the pupil to obtain fused temporal features;
[0027] The LSTM layer generates a hidden state sequence based on the fused temporal features;
[0028] The attention layer dynamically assigns weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism.
[0029] The output layer maps the hidden state sequence after dynamic weight allocation into a transmittance adjustment value recognition result.
[0030] The loss value is determined based on the transmittance adjustment value identification result and the corresponding transmittance adjustment value label;
[0031] The parameters of the CNN-LSTM hybrid neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained light transmission adjustment requirement recognition model.
[0032] Furthermore, in one embodiment of the present invention, adjusting the light transmittance of the sunshade and dimming film located in the upper region of the windshield according to the target light transmittance adjustment value specifically includes:
[0033] Obtain the current transmittance of the sunshade and light-regulating film, and determine the target transmittance based on the current transmittance and the target transmittance adjustment value;
[0034] Obtain the voltage-transmittance mapping table of the shading and dimming film, and determine the target operating voltage of the shading and dimming film based on the target transmittance and the voltage-transmittance mapping table;
[0035] Adjust the operating voltage of the sunshade and dimming film to the target operating voltage.
[0036] Furthermore, in one embodiment of the present invention, the shading dimming film is an EC dimming film or an SPD dimming film.
[0037] Secondly, embodiments of the present invention provide a cockpit sunshade control system based on machine vision, comprising:
[0038] The image processing module is used to acquire an image of the target driver's eye region, and extract the temporal features of the brightness and darkness of the eye region, the temporal features of the eye muscles, and the temporal features of the pupil from the image of the eye region.
[0039] The transmittance adjustment value determination module is used to input the temporal characteristics of the brightness and darkness of the eye region, the temporal characteristics of the eye muscles, and the temporal characteristics of the pupil into a pre-constructed transmittance adjustment demand recognition model to obtain the target transmittance adjustment value.
[0040] The sunshade and dimming film adjustment module is used to adjust the light transmittance of the sunshade and dimming film located in the upper area of the windshield according to the target light transmittance adjustment value.
[0041] Thirdly, embodiments of the present invention provide a cockpit sunshade control device based on machine vision, comprising:
[0042] At least one processor;
[0043] At least one memory for storing at least one program;
[0044] When the at least one program is executed by the at least one processor, the at least one processor implements the above-described machine vision-based cockpit sunshade control method.
[0045] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the aforementioned machine vision-based cockpit sunshade control method.
[0046] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention:
[0047] This invention acquires an image of the driver's eye region, extracts temporal features of eye region brightness and darkness, eye muscle temporal features, and pupil temporal features from the image, and inputs these features into a pre-built light transmittance adjustment requirement recognition model to obtain a target light transmittance adjustment value. The light transmittance of the sunshade film located in the upper region of the windshield is then adjusted according to this target light transmittance adjustment value. This invention extracts temporal features of eye region brightness and darkness, eye muscle temporal features, and pupil temporal features from the driver's eye region image, and automatically analyzes whether the driver has a light transmittance adjustment requirement due to eye discomfort caused by external light exposure based on a pre-trained light transmittance adjustment requirement recognition model. It then outputs the corresponding light transmittance adjustment value, thereby automatically adjusting the light transmittance of the sunshade film in the upper region of the windshield. This achieves real-time, automatic cockpit sun shading control without manual operation by the driver, improving the efficiency and accuracy of cockpit sun shading control, and enhancing the driver's driving experience and safety. Attached Figure Description
[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0049] Figure 1 A flowchart illustrating the steps of a cockpit sunshade control method based on machine vision, provided in an embodiment of the present invention.
[0050] Figure 2 This is a schematic diagram illustrating the layout of the sunshade and light-dimming film provided in an embodiment of the present invention.
[0051] Figure 3 This is a schematic diagram of the structure of the CNN-LSTM hybrid neural network provided in an embodiment of the present invention;
[0052] Figure 4A structural block diagram of a cockpit sunshade control system based on machine vision provided in an embodiment of the present invention;
[0053] Figure 5 This is a structural block diagram of a cockpit sunshade control device based on machine vision, provided in an embodiment of the present invention. Detailed Implementation
[0054] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.
[0055] In the description of this invention, "multiple" means two or more. The use of "first" and "second" is for distinguishing technical features only and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or the order of the indicated technical features. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.
[0056] Reference Figure 1 This invention provides a machine vision-based cockpit sunshade control method, which specifically includes the following steps:
[0057] S101. Obtain an image of the target driver's eye region, and extract the temporal features of the brightness and darkness of the eye region, the temporal features of the eye muscles, and the temporal features of the pupil based on the image of the eye region.
[0058] S102. Input the temporal features of light and dark in the eye region, the temporal features of eye muscles, and the temporal features of pupils into the pre-built light transmittance adjustment requirement recognition model to obtain the target light transmittance adjustment value.
[0059] S103. Adjust the light transmittance of the sunshade and dimming film located in the upper area of the windshield according to the target light transmittance adjustment value.
[0060] Specifically, when a driver's eyes are exposed to direct sunlight from outside the vehicle, phenomena such as slight eyelid closure, eyeball deflection, and changes in pupil diameter may occur. In addition, the brightness of the driver's eye area image will also be higher. These features can be used to identify whether the driver has a need for eye shading. This invention extracts the temporal features of brightness and darkness in the eye area, the temporal features of eye muscles, and the temporal features of pupils from the driver's eye area image. Based on a pre-trained light transmission adjustment need recognition model, it automatically analyzes whether the driver has a need for light transmission adjustment due to eye discomfort caused by external light exposure, and outputs the corresponding light transmission adjustment value. This enables automatic adjustment of the light transmission of the sunshade and dimming film in the upper area of the windshield, realizing real-time and automatic cockpit sun shading control without manual operation by the driver. This improves the efficiency and accuracy of cockpit sun shading control, as well as the driver's driving experience and driving safety.
[0061] like Figure 2 The diagram shown is a schematic diagram of the layout of the sunshade and dimming film provided in an embodiment of the present invention. As can be seen, the sunshade and dimming film in this embodiment of the present invention is only set in the upper area of the windshield. While blocking the sunlight that shines directly into the driver's eyes, it does not affect the driver's view of the road conditions in front of the vehicle, thus ensuring driving safety.
[0062] As a further optional implementation, an image of the target driver's eye region is acquired, and temporal features of eye region brightness and darkness, temporal features of eye muscles, and temporal features of the pupil are extracted from the eye region image. Specifically, this includes:
[0063] S1011. Acquire the facial image information of the target driver through the camera device installed in the cockpit, perform edge detection and texture analysis on the facial image information to obtain the eye contour, and extract the eye region image from the facial image information based on the eye contour.
[0064] S1012. Determine the brightness value distribution matrix of each pixel in the eye region image, and perform temporal processing on the brightness value distribution matrix corresponding to multiple consecutive frames of eye region images to obtain the temporal characteristics of brightness and darkness in the eye region.
[0065] S1013. Extract the periocular muscle texture features based on the eye region image and determine the eyelid opening and closing degree. Perform temporal processing on the periocular muscle texture features and eyelid opening and closing degree corresponding to multiple consecutive frames of eye region images to obtain the temporal features of the eye muscles.
[0066] S1014. Binarize the eye region image and detect the pupil contour. Determine the pupil position and pupil diameter based on the pupil contour. Perform temporal processing on the pupil position and pupil diameter corresponding to multiple consecutive frames of eye region images to obtain the temporal features of the eyeball pupil.
[0067] Specifically, real-time facial images of the driver are captured using in-vehicle cameras (such as those in front of the steering wheel or on the dashboard). This requires ensuring adaptability to varying lighting conditions and angular coverage. Multiple receiving units (such as millimeter-wave radar / Bluetooth sensors) are used to locate the driver's position, and the camera angle is dynamically adjusted to ensure complete coverage of the eye area. Edge detection and texture analysis are performed on the original images to segment candidate eye regions. A sliding window iterative calculation of the "nearest neighbor fit index" is used to merge highly similar regions, generating precise eye position coordinates. The resulting cropped eye region image (including structures such as the eyelids, iris, and pupil) is then output.
[0068] The feature extraction process is as follows:
[0069] 1) Temporal characteristics of light and dark in the eye area:
[0070] The brightness distribution of pixels in the eye region is statistically analyzed to obtain a brightness value distribution matrix. The brightness value distribution matrix corresponding to multiple consecutive frames of eye region images is then processed temporally to obtain the temporal characteristics of brightness and darkness in the eye region.
[0071] 2) Temporal characteristics of eye muscles:
[0072] Based on the images of the eye region, the texture features of the muscles around the eyes (such as changes in crow's feet wrinkles) are extracted and the opening and closing of the eyelids are analyzed. The texture features of the muscles around the eyes and the opening and closing of the eyelids corresponding to multiple consecutive frames of images of the eye region are processed temporally to obtain the temporal features of the eye muscles.
[0073] 3) Temporal characteristics of pupil:
[0074] The eye region is binarized, and the pupil contour is detected using the findContours function of OpenCV. The convex hull of the contour is calculated, and the pupil center coordinates and pupil diameter are determined by the minimum bounding rectangle (minAreaRect). The changes in pupil center coordinates and pupil diameter in consecutive frames are recorded to obtain the temporal features of the pupil.
[0075] As an optional implementation, the light transmittance adjustment requirement recognition model is trained through the following steps:
[0076] S201. Obtain the initial light transmittance of the sunshade and dimming film of the test vehicle;
[0077] S202. Obtain image samples of the eye region of the test personnel inside the test vehicle, and extract the temporal feature samples of the brightness and darkness of the eye region, the temporal feature samples of the eye muscles, and the temporal feature samples of the pupil based on the image samples of the eye region.
[0078] S203. Obtain the optimal light transmittance after the test personnel adjust the sunshade and light-dimming film of the test vehicle to the most comfortable state;
[0079] S204. Generate training samples based on the temporal feature samples of light and dark areas of the eye region, the temporal feature samples of eye muscles, and the temporal feature samples of pupils. Determine the transmittance adjustment value label based on the difference between the initial transmittance and the optimal transmittance. Then, construct a training dataset based on the training samples and the corresponding transmittance adjustment value label.
[0080] S205. Input the training dataset into the pre-built CNN-LSTM hybrid neural network for training to obtain a trained light transmission adjustment requirement recognition model.
[0081] Specifically, firstly, the initial transmittance of the sunshade film of the test vehicle is pre-acquired in the test scenario; then, image samples of the eye area of the test personnel inside the test vehicle are acquired, and temporal feature samples of eye area brightness and darkness, eye muscle, and pupil are extracted from the eye area image samples; the test personnel manually adjust the sunshade film of the test vehicle until it reaches the most comfortable state for their eyes, and the optimal transmittance of the sunshade film at this time is obtained; the temporal feature samples of eye area brightness and darkness, eye muscle, and pupil are used as training samples, and the transmittance adjustment value label is determined according to the difference between the initial transmittance and the optimal transmittance, thus obtaining a set of training data; multiple tests are conducted under different test personnel, different initial transmittance, and different external light conditions to obtain a sufficient amount of training data to construct a training dataset; the training dataset is input into a pre-constructed CNN-LSTM hybrid neural network for training, thus obtaining a trained transmittance adjustment demand recognition model.
[0082] As an optional implementation, the CNN-LSTM hybrid neural network includes an input layer, a CNN convolutional layer, a feature fusion layer, an LSTM layer, an attention layer, and an output layer. The input layer is used to input training samples, the CNN convolutional layer is used to extract features from the training samples to obtain local temporal features, the feature fusion layer is used to fuse the local temporal features to obtain fused temporal features, the LSTM layer is used to generate a hidden state sequence based on the fused temporal features, the attention layer is used to dynamically assign weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism, and the output layer is used to map the dynamically weighted hidden state sequence into a transmittance adjustment value recognition result.
[0083] like Figure 3The diagram shows the structure of the CNN-LSTM hybrid neural network provided in this embodiment of the invention. As can be seen, this embodiment of the invention sets up three CNN convolutional layers to extract local temporal features from the light and dark temporal feature samples of the eye region, the eye muscle temporal feature samples, and the pupil temporal feature samples, respectively. The obtained local features of light and dark in the eye region, the eye muscle, and the pupil will enter the feature fusion layer for feature fusion. Finally, the fused temporal features will be input into the LSTM layer for subsequent recognition.
[0084] As a further optional implementation, the training dataset is input into a pre-built CNN-LSTM hybrid neural network for training to obtain a trained light transmission adjustment requirement recognition model, which specifically includes:
[0085] S2051. Input training samples through the input layer;
[0086] S2052. By using CNN convolutional layers to extract features from the temporal feature samples of light and dark areas in the eye region, the temporal feature samples of eye muscles, and the temporal feature samples of pupils, the local features of light and dark areas in the eye region, the local features of eye muscles, and the local features of pupils are obtained.
[0087] S2053. By using a feature fusion layer, the local features of light and dark areas in the eye region, the local features of eye muscles, and the local features of the pupil are fused to obtain fused temporal features.
[0088] S2054. Generate hidden state sequences based on fused temporal features using LSTM layers;
[0089] S2055. Dynamic weight allocation is performed on each dimension of the hidden state sequence through the attention layer based on a multi-head self-attention mechanism.
[0090] S2056. The hidden state sequence after dynamic weight allocation is mapped to the transmittance adjustment value recognition result through the output layer.
[0091] S2057. Determine the loss value based on the transmittance adjustment value identification result and the corresponding transmittance adjustment value label;
[0092] S2058. Update the parameters of the CNN-LSTM hybrid neural network according to the loss value through the backpropagation algorithm to obtain the trained light transmission adjustment requirement recognition model.
[0093] Specifically, after inputting the fused temporal features into the LSTM layer, a corresponding hidden state sequence is generated. Based on the multi-head self-attention mechanism, dynamic weights are assigned to each dimension of the hidden state sequence, and then mapped to the transmittance adjustment value recognition result. The loss value is determined based on the difference between the transmittance adjustment value recognition result and the transmittance adjustment value label. The choice of loss function is not limited in this embodiment of the invention. The parameters of the CNN-LSTM hybrid neural network are updated based on the loss value through the backpropagation algorithm, and then the next round of iterative training is entered. When the preset convergence condition is reached (the number of iterations reaches a threshold, and the loss value is lower than the preset threshold), the training is stopped, and the trained transmittance adjustment requirement recognition model is obtained.
[0094] As a further optional implementation, the light transmittance of the sunshade and dimming film located in the upper region of the windshield is adjusted according to the target light transmittance adjustment value, specifically including:
[0095] S1031. Obtain the current transmittance of the sunshade and light-regulating film, and determine the target transmittance based on the current transmittance and the target transmittance adjustment value.
[0096] S1032. Obtain the voltage-transmittance mapping table of the shading and dimming film, and determine the target operating voltage of the shading and dimming film based on the target transmittance and the voltage-transmittance mapping table.
[0097] S1033. Adjust the working voltage of the sunshade and dimming film to the target working voltage.
[0098] Specifically, a spectrophotometer or transmittance meter (such as automotive film testing equipment) is used to measure the current transmittance of the sunshade film in real time. The target transmittance is determined based on the sum of the current transmittance and the target transmittance adjustment value (positive values indicate an increase, negative values indicate a decrease). The voltage-transmittance mapping table is provided by the sunshade film manufacturer and is usually non-linear. The nearest neighbor value is matched in the mapping table according to the target transmittance, and the precise target operating voltage is determined by interpolation (such as linear interpolation). The operating voltage of the sunshade film is adjusted to the target operating voltage by the electronic control device of the sunshade film, thereby adjusting the transmittance of the sunshade film to the target transmittance currently required by the driver. In addition, the sunshade film has a protection mechanism. If the voltage exceeds the limit or the transmittance is abnormal, a protection strategy (such as resetting to a safe voltage) is triggered.
[0099] As an optional implementation, the shading dimming film is an EC dimming film or an SPD dimming film.
[0100] The EC dimming film contains electrochromic materials that change color under different voltages, thereby altering the light transmittance. When a positive voltage is applied, the material changes from a transparent state to a dark state, reducing the light transmittance. When a reverse voltage is applied, the material returns to a transparent state, increasing the light transmittance.
[0101] The transmittance adjustment of SPD dimming films is based on suspended particle (SPD) technology. Its core principle is to dynamically adjust light transmittance by changing the electric field intensity to control the arrangement of internal suspended particles. The specific mechanism is as follows:
[0102] No voltage / low voltage state: The particles exhibit disordered Brownian motion and randomly scatter light. At this time, the light transmittance of the dimming film is at its lowest (1% to 60%, which may vary depending on the product), and it appears dark or opaque.
[0103] High voltage state: The electric field causes the particles to align in an orderly manner along the direction of the electric field, reducing light scattering and increasing the light transmittance to the maximum (up to 60% or 100%, depending on the product), resulting in a transparent state.
[0104] Therefore, SPD dimming film can also achieve continuous change in light transmittance by adjusting the voltage.
[0105] In this embodiment of the invention, EC dimming film or SPD dimming film is used as the sunshade dimming film, which can accurately adjust the light transmittance of the sunshade dimming film. Compared with the traditional technology that uses PDLC dimming film, which can only achieve binary state (transparent state, opaque fog state) adjustment, the accuracy of the control of the sunshade dimming film is greatly improved, thereby greatly improving the accuracy of the cockpit sunshade control.
[0106] The method steps of the embodiments of the present invention have been described above. It can be understood that the embodiments of the present invention extract the temporal features of brightness and darkness in the eye region, the temporal features of eye muscles, and the temporal features of the pupil based on the driver's eye region image. Based on a pre-trained light transmittance adjustment demand recognition model, it automatically analyzes whether the driver has a need for light transmittance adjustment due to eye discomfort caused by external light exposure, and outputs the corresponding light transmittance adjustment value. This enables automatic adjustment of the light transmittance of the sunshade and dimming film in the upper area of the windshield, achieving real-time, automatic cockpit sunshade control without manual operation by the driver. This improves the efficiency and accuracy of cockpit sunshade control, and also enhances the driver's driving experience and driving safety.
[0107] Reference Figure 4 This invention provides a machine vision-based cockpit sunshade control system, comprising:
[0108] The image processing module is used to acquire images of the target driver's eye region and extract the temporal features of brightness and darkness of the eye region, temporal features of eye muscles, and temporal features of pupil from the eye region images.
[0109] The transmittance adjustment value determination module is used to input the temporal features of brightness and darkness in the eye region, the temporal features of eye muscles, and the temporal features of pupils into a pre-built transmittance adjustment demand recognition model to obtain the target transmittance adjustment value.
[0110] The sunshade and dimming film adjustment module is used to adjust the light transmittance of the sunshade and dimming film located in the upper area of the windshield according to the target light transmittance adjustment value.
[0111] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.
[0112] Reference Figure 5 This invention provides a machine vision-based cockpit sunshade control device, comprising:
[0113] At least one processor;
[0114] At least one memory for storing at least one program;
[0115] When the above-mentioned at least one program is executed by the above-mentioned at least one processor, the above-mentioned at least one processor implements the above-mentioned machine vision-based cockpit sunshade control method.
[0116] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0117] This invention also provides a computer-readable storage medium storing a processor-executable program that, when executed by a processor, performs the aforementioned machine vision-based cockpit sunshade control method.
[0118] This invention provides a computer-readable storage medium that can execute a machine vision-based cockpit sunshade control method provided in the method embodiments of this invention. It can execute any combination of the implementation steps of the method embodiments and has the corresponding functions and beneficial effects of the method.
[0119] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform... Figure 1 The method shown.
[0120] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.
[0121] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.
[0122] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0123] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.
[0124] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or, if necessary, processing in other suitable ways, and then stored in computer memory.
[0125] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0126] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0127] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
[0128] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. A cockpit sunshade control method based on machine vision, characterized in that, Includes the following steps: Obtain an image of the target driver's eye region, and extract the temporal features of the brightness and darkness of the eye region, the temporal features of the eye muscles, and the temporal features of the pupil based on the image of the eye region. The temporal features of light and dark in the eye region, the temporal features of the eye muscles, and the temporal features of the pupil are input into a pre-built light transmittance adjustment requirement recognition model to obtain the target light transmittance adjustment value. Adjust the light transmittance of the sunshade and dimming film located in the upper area of the windshield according to the target light transmittance adjustment value.
2. The cockpit sunshade control method based on machine vision according to claim 1, characterized in that, The process of acquiring an image of the target driver's eye region, and extracting temporal features of eye region brightness and darkness, temporal features of eye muscles, and temporal features of pupil size from the eye region image, specifically includes: The facial image information of the target driver is acquired by a camera device installed in the cockpit. Edge detection and texture analysis are performed on the facial image information to obtain the eye contour. The eye region image is extracted from the facial image information based on the eye contour. The brightness value distribution matrix of each pixel in the eye region image is determined, and the brightness value distribution matrix corresponding to multiple consecutive frames of the eye region image is processed temporally to obtain the brightness and darkness temporal features of the eye region. Based on the eye region image, extract the periocular muscle texture features and determine the eyelid opening and closing degree. Perform temporal processing on the periocular muscle texture features and eyelid opening and closing degree corresponding to multiple consecutive frames of the eye region image to obtain the temporal features of the eye muscles. The eye region image is binarized and the pupil contour is detected. The pupil position and pupil diameter are determined based on the pupil contour. The pupil position and pupil diameter corresponding to multiple consecutive frames of the eye region image are temporally processed to obtain the temporal features of the eyeball pupil.
3. The cockpit sunshade control method based on machine vision according to claim 1, characterized in that, The light transmittance adjustment requirement identification model is trained through the following steps: Obtain the initial light transmittance of the sunshade and dimming film on the test vehicle; Obtain eye region image samples of test personnel inside the test vehicle, and extract eye region light and dark temporal feature samples, eye muscle temporal feature samples, and eyeball pupil temporal feature samples based on the eye region image samples. Obtain the optimal light transmittance after the test personnel adjust the sunshade and dimming film of the test vehicle to the most comfortable state; Training samples are generated based on the temporal feature samples of light and dark in the eye region, the temporal feature samples of the eye muscles, and the temporal feature samples of the pupil. A transmittance adjustment value label is determined based on the difference between the initial transmittance and the optimal transmittance. A training dataset is then constructed based on the training samples and the corresponding transmittance adjustment value label. The training dataset is input into a pre-built CNN-LSTM hybrid neural network for training to obtain the trained light transmission adjustment requirement recognition model.
4. The cockpit sunshade control method based on machine vision according to claim 3, characterized in that, The CNN-LSTM hybrid neural network includes an input layer, a CNN convolutional layer, a feature fusion layer, an LSTM layer, an attention layer, and an output layer. The input layer is used to input the training samples. The CNN convolutional layer is used to extract features from the training samples to obtain local temporal features. The feature fusion layer is used to fuse the local temporal features to obtain fused temporal features. The LSTM layer is used to generate a hidden state sequence based on the fused temporal features. The attention layer is used to dynamically assign weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism. The output layer is used to map the dynamically weighted hidden state sequence into a transmittance adjustment value recognition result.
5. The cockpit sunshade control method based on machine vision according to claim 4, characterized in that, The step of inputting the training dataset into a pre-constructed CNN-LSTM hybrid neural network for training to obtain the trained light transmittance adjustment requirement recognition model specifically includes: The training samples are input through the input layer; The CNN convolutional layer extracts features from the temporal features of the light and dark areas of the eye region, the temporal features of the eye muscles, and the temporal features of the pupil, respectively, to obtain local features of the light and dark areas of the eye region, local features of the eye muscles, and local features of the pupil. The feature fusion layer performs feature fusion on the local features of light and dark areas in the eye region, the local features of the eye muscles, and the local features of the pupil to obtain fused temporal features; The LSTM layer generates a hidden state sequence based on the fused temporal features; The attention layer dynamically assigns weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism. The output layer maps the hidden state sequence after dynamic weight allocation into a transmittance adjustment value recognition result. The loss value is determined based on the transmittance adjustment value identification result and the corresponding transmittance adjustment value label; The parameters of the CNN-LSTM hybrid neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained light transmission adjustment requirement recognition model.
6. The cockpit sunshade control method based on machine vision according to claim 1, characterized in that, The adjustment of the light transmittance of the sunshade and dimming film located in the upper region of the windshield according to the target light transmittance adjustment value specifically includes: Obtain the current transmittance of the sunshade and light-regulating film, and determine the target transmittance based on the current transmittance and the target transmittance adjustment value; Obtain the voltage-transmittance mapping table of the shading and dimming film, and determine the target operating voltage of the shading and dimming film based on the target transmittance and the voltage-transmittance mapping table; Adjust the operating voltage of the sunshade and dimming film to the target operating voltage.
7. A cockpit sunshade control method based on machine vision according to any one of claims 1 to 6, characterized in that: The shading and dimming film is either an EC dimming film or an SPD dimming film.
8. A cockpit sunshade control system based on machine vision, characterized in that, include: The image processing module is used to acquire an image of the target driver's eye region, and extract the temporal features of the brightness and darkness of the eye region, the temporal features of the eye muscles, and the temporal features of the pupil from the image of the eye region. The transmittance adjustment value determination module is used to input the temporal characteristics of the brightness and darkness of the eye region, the temporal characteristics of the eye muscles, and the temporal characteristics of the pupil into a pre-constructed transmittance adjustment demand recognition model to obtain the target transmittance adjustment value. The sunshade and dimming film adjustment module is used to adjust the light transmittance of the sunshade and dimming film located in the upper area of the windshield according to the target light transmittance adjustment value.
9. A cockpit sunshade control device based on machine vision, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a machine vision-based cockpit sunshade control method as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform a machine vision-based cockpit sunshade control method as described in any one of claims 1 to 7.