Dimming method, system, device, and medium based on high dynamic response image data

Abstract

The present application belongs to the technical field of image processing, and particularly relates to a dimming method, system, device and medium based on high dynamic response image data, which does not need to rely on the spatial coordinate positioning of a strong light source, fits the outside scene image of the driver's field of view, combines the driver's pupil positioning, extends the fitting dimming area to the dimming screen in the inverse light path, determines the dimming position, and effectively solves the problem of inaccurate positioning of the outside scene strong light in the existing active dimming technology. In addition, through transition processing and pre-judgment softening of the shielding area, the field of view light path fault tolerance is effectively improved, the fault tolerance correction of the possible shading deviation of bumps, shading omissions and the like in the high-speed driving process of the vehicle is achieved, and the driver's observation comfort is met.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a dimming method, system, device, and medium based on high dynamic response image data. Background Technology

[0002] Drivers of vehicles, ships, aircraft, and other transport vehicles are susceptible to glare from direct sunlight, strong light reflection, oncoming high beams, natural or artificial flashes, etc., which can cause eye discomfort, dizziness, or even temporary blindness, seriously endangering the driver's health and creating driving safety hazards.

[0003] Currently, anti-glare technologies for vehicle windshields are mainly divided into two categories: passive and active. Passive technologies have no electronic control or dynamic adjustment functions. They permanently reduce light reflection, filter strong light, and disperse glare through glass material optimization, coating treatment, interlayer design, and surface processing. However, their dimming effect is fixed and cannot adapt to complex and changing external lighting environments.

[0004] Active anti-glare technology typically consists of sensors, controllers, and electronically controlled dimming glass. By sensing the intensity of ambient light in real time, it dynamically adjusts the glass's light transmittance to achieve precise local or global anti-glare. More advanced methods combine ambient light sensors, cameras / radar, and eye-tracking technology to collect information such as light intensity, light source position, and the driver's line of sight, calculating and determining the dimming area before performing the anti-glare operation.

[0005] However, existing active dimming technologies generally rely on spatial coordinate positioning of strong light to determine the working position of the dimming screen through optical path calculations. Currently, the depth measurement of optical objects of uncertain size by monocular or binocular cameras is limited by a "1-pixel parallax," with a detection distance typically not exceeding 20 meters, and a theoretical limit of no more than 50 meters. For high-speed vehicles such as airplanes and automobiles, this detection distance is far from meeting practical needs, especially since strong light in the driver's main field of vision is often hundreds of meters away. Existing technologies lack sufficient positioning accuracy for this area, or even fail to locate it at all, leading to poor dimming effects and anti-glare failure. Summary of the Invention

[0006] To address the problems in the background technology, the present invention provides a dimming method, system, device, and medium based on high dynamic response image data.

[0007] The technical solution of the present invention is as follows: This invention provides a dimming method based on high dynamic response image data, comprising: S1: Initialization parameters, including: preset light intensity threshold, preset strong light range threshold, and dimming transition strategy parameters; S2: Acquire outdoor environment image data, compare light intensity pixel by pixel, and mark pixels whose light intensity exceeds the preset light intensity threshold, assigning values ​​according to the amount of light intensity exceeding the threshold; classify all adjacent marked pixels into a strong light range, retain marked pixels whose area in the strong light range is not less than the preset strong light range threshold, and form a pre-adjustment area; divide the area into several dimming sub-regions according to the values ​​assigned to the pixels in the pre-adjustment area. S3: Acquire the driver's facial image, locate the pupil spatial coordinates using a multi-view coordinate recognition method, and calculate the relative position parameters of the pupil with respect to the external shooting device and the dimming screen; S4: Merge the light-adjusting sub-regions corresponding to the same strong light source from different outdoor shooting devices to obtain the superimposed light-adjusting sub-region; Based on the coordinate ratio of the driver's pupil to different outdoor shooting devices, the superimposed dimming area is mapped to the corresponding plane coordinates to obtain the position of the superimposed dimming area; S5: Using the pupil as the origin, extend the superimposed dimming sub-area along the backlight path of the shooting angle, calculate the intersection position of the extended path and the spatial coordinates of the dimming screen, and determine the dimming area.

[0008] Based on the dimming method based on high dynamic response image data described above, step S4, which maps the superimposed dimming sub-region to the corresponding plane coordinates according to the coordinate ratio of the driver's pupil and different outdoor shooting devices, to obtain the position of the superimposed dimming sub-region, specifically: Based on the reference horizontal coordinates of the superimposed dimming areas of the same strong light source in different outdoor shooting devices, and the horizontal distance between the driver's single pupil and each outdoor shooting device, the horizontal coordinates are mapped to obtain the target horizontal coordinates. Based on the reference vertical coordinates of the superimposed dimming areas of the same strong light source in different outdoor shooting devices, and the vertical distance between the driver's single pupil and each outdoor shooting device, vertical coordinate mapping is performed to obtain the target vertical coordinates. The location of the superimposed photodiode region is obtained by combining the target's horizontal and vertical coordinates.

[0009] Based on the dimming method based on high dynamic response image data described above, S5 further includes edge transition processing of the dimming area, generating a pre-judged dimming area along the direction of movement of the strong light source, and generating a dimming screen dimming instruction set.

[0010] Furthermore, the edge transition processing of the dimming area specifically includes: The dimming area extends outward by a preset distance, and the dimming intensity is reduced along the extension direction until the dimming intensity at the outer boundary of the extended area drops to 0.

[0011] Furthermore, the generation of the pre-judged dimming area along the direction of movement of the strong light source specifically includes: Detect the motion trajectory of strong light sources in outdoor environmental image data and calculate the relative speed of the strong light sources to the vehicle. The running speed is compared with multiple preset high-speed thresholds. If the speed exceeds the lowest preset high-speed threshold, the speed is classified and recorded. For strong light sources with different speed levels, a pre-defined dimming area is generated along the direction of motion, and the path length is positively correlated with the speed level.

[0012] Based on the above-described dimming method based on high dynamic response image data, before merging the dimming sub-regions corresponding to the same strong light source from different outdoor shooting devices in step S4, the method further includes scaling the coordinates of the dimming sub-regions obtained by the outdoor shooting devices with different axes using the same spatial reference system.

[0013] Based on the dimming method based on high dynamic response image data described above, the dimming area determined in step S5 is a closed area formed by the intersection position and the boundary of the dimming screen, which is used as the dimming area.

[0014] The present invention also provides a dimming system based on high dynamic response image data, comprising: Initialization module: Used to initialize parameters, including: preset light intensity threshold, preset strong light range threshold, and dimming transition strategy parameters; The dimming area division module is used to acquire outdoor environment image data, compare light intensity pixel by pixel, and mark pixels whose light intensity exceeds a preset light intensity threshold, assigning values ​​according to the amount of light intensity exceeding the threshold; classify all adjacent marked pixels into a strong light range, retain marked pixels whose area in the strong light range is not less than the preset strong light range threshold, and form a pre-dimming area; divide the area into several dimming sub-regions according to the values ​​assigned to the pixels in the pre-dimming area. Pupil positioning module: used to acquire driver's facial image, locate pupil spatial coordinates through multi-view coordinate recognition method, and calculate the relative position parameters of pupil with respect to the external shooting device and dimming screen; Dimming area fitting module: used to merge the dimming sub-regions corresponding to the same strong light source from different outdoor shooting devices to obtain superimposed dimming sub-regions; Based on the coordinate ratio of the driver's pupil to different outdoor shooting devices, the superimposed dimming area is mapped to the corresponding plane coordinates to obtain the position of the superimposed dimming area; Dimming area determination module: Taking the pupil as the origin, the superimposed dimming sub-area is extended along the backlight path of the shooting angle, and the intersection position of the extended path and the spatial coordinates of the dimming screen is calculated to determine the dimming area.

[0015] The present invention also provides a dimming device based on high dynamic response image data, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the dimming method based on high dynamic response image data.

[0016] The present invention also provides a medium for storing a computer program, wherein the computer program, when executed by a processor, implements the dimming method based on high dynamic response image data.

[0017] Beneficial effects This invention eliminates the need for spatial coordinate positioning based on strong light sources. By fitting an external image from the driver's field of vision and combining this with the driver's pupil positioning, the backlight path of the fitted dimming area is extended to the dimming screen to determine the dimming position. This effectively solves the problem of inaccurate positioning of strong external light in existing active dimming technologies. Furthermore, by performing transition processing and pre-judgment softening on obstructed areas, the error tolerance of the field of vision optical path is effectively improved. This allows for error correction of potential shading deviations such as bumps and shading omissions during high-speed vehicle travel, thus meeting the driver's observation comfort requirements. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the device used in the active dimming of the vehicle in Example 1.

[0019] Figure 2 This is a partial flowchart of the high dynamic response image processing in the active dimming of the vehicle in Example 1.

[0020] Figure 3 This is a schematic diagram illustrating the reverse extension of high dynamic response image data processing in active dimming of vehicles.

[0021] The attached figures are labeled as follows: 1-First fixed-position exterior camera group, 2-Second fixed-position exterior camera group, 3-Fixed-position interior camera group, 4-Dimming screen, 5-Driver, 6-Overlay dimming sub-area, 7-Plane containing the overlay dimming sub-area, 8-Dimming area, 9-Driver's physiological visual angle. Detailed Implementation

[0022] The following examples are intended to illustrate the present invention, and not to further limit the invention.

[0023] Example 1 This embodiment provides a dimming method based on high dynamic response image data. The basic hardware used by the vehicle in the active dimming process includes a first fixed-position outdoor camera group 1, a second fixed-position outdoor camera group 2, a fixed-position indoor camera group 3, a dimming screen 4, and a software computing system.

[0024] The first fixed-position exterior camera group 1 and the second fixed-position exterior camera group 2 each include at least two independent fixed-position cameras. The shooting direction of all cameras in the first fixed-position exterior camera group 1 and the second fixed-position exterior camera group 2 is the outside of the windshield, that is, the same direction as the driver's usual observation direction, but it is not excluded that there is a directional angle of no more than 90°.

[0025] In the first fixed-position exterior camera group 1 and the second fixed-position exterior camera group 2, at least one fixed-position exterior camera is located directly in front of or to the left of the driver's left eye, and at least one other fixed-position exterior camera is located directly in front of or to the right of the driver's right eye. Figure 1 As shown.

[0026] Using the line connecting the driver's eyes when looking straight ahead as the baseline, the first fixed-position external camera group 1, located directly in front of or to the left of the driver's left eye, and the second fixed-position external camera group 2, located directly in front of or to the right of the driver's right eye, should also be located on both sides of the baseline or simultaneously on the baseline. If more fixed-position cameras are installed, their positions should also meet the principle that all fixed-position external camera groups are not on the same side of the baseline, but can be on the baseline at the same time.

[0027] The function of the first fixed-position exterior camera group 1 and the second fixed-position exterior camera group 2 is to capture and acquire environmental information outside the windshield. The viewing angle of the first fixed-position exterior camera group 1 and the second fixed-position exterior camera group 2 is at least greater than the main viewing angle of the human eye, and preferably greater than the human eye's viewing angle, so as to obtain sufficient information covering the visible area when the human eye observes.

[0028] By utilizing a first fixed-position outdoor camera group 1 and a second fixed-position outdoor camera group 2, the acquired external environment information can be extracted during fitting to sufficiently cover the external environment information actually observed by the human eye. To reduce CCD device aging caused by long-term sunlight exposure, the first fixed-position outdoor camera group 1 and the second fixed-position outdoor camera group 2 can be equipped with an anti-ultraviolet film layer. To accurately identify strong light, the first fixed-position outdoor camera group 1 and the second fixed-position outdoor camera group 2 can also be equipped with a dark filter to attenuate light intensity and improve image capture and accuracy in strong light.

[0029] The cameras in the fixed-position interior camera group 3 can be monocular cameras, but binocular cameras are preferred. All cameras in the fixed-position interior camera group 3 capture the driver's face, but the direction of the driver's gaze can be disregarded if high precision is not required. To improve recognition performance in low light, the fixed-position interior camera group 3 may also include infrared night vision capabilities.

[0030] The dimming screen 4 includes a windshield and a liquid crystal array screen set based on the windshield. The liquid crystal array screen includes, but is not limited to, the basic structure of a liquid crystal screen: two polarizers, a liquid crystal interlayer, and transparent electrodes. The liquid crystal array screen does not use color, but only a black and white screen, that is, it only has the function of attenuating light intensity and does not have the function of filtering specific wavelengths of light, but other considerations besides anti-glare are not excluded.

[0031] The basic color-changing unit of the dimming screen 4 is a pixel, which is a square or hexagonal pixel geometry in the conventional display field, preferably between 0.1*0.1-10*10mm. Each pixel can display more than two gray levels. The liquid crystal array screen can be part of the windshield itself or independent of it, but the liquid crystal array screen and the windshield are parallel, and the area of ​​the liquid crystal array screen is no larger than that of the windshield. To further optimize the dimming effect, the dimming screen 4 can also be installed on light-transmitting structures used for driving observation, such as portholes or vehicle windows, outside the windshield.

[0032] Preferably, the pixels in this embodiment are 1*1mm hexagons. Each pixel can display 5 gray levels. The liquid crystal array screen is independent of the windshield, and its curvature is the same as the inner side of the windshield, fitting snugly against the inner side of the windshield.

[0033] The software computing system is embedded in the vehicle system. In terms of software architecture, it adopts a low-level multi-threaded concurrent processing mechanism based on the C++11 standard library. It directly requests instantiation of four independent processing threads from the operating system kernel in memory: a lightCameraThread thread for capturing external binocular data, an eyeCameraThread thread for capturing internal infrared facial data, a lightProcessThread thread for calculating strong light patterns, and an eyeProcessThread thread for tracking the spatial position of the human eye. To avoid read / write conflicts and bus congestion caused by traditional multi-threaded sharing of global variables, the system constructs a generic template class ThreadSafeQueue as a frame buffer pool in the heap memory. This buffer pool internally encapsulates a mutex lock; any thread must acquire ownership of the mutex lock before performing push or pull operations, preventing concurrent read / write operations by other threads. Simultaneously, combined with condition variables, when the queue is empty, the processing thread actively suspends and sleeps, yielding CPU time slices; when the acquisition thread pushes in a new frame, it immediately sends a wake-up signal. The maximum cache depth of the buffer pool is forcibly clamped to 2 frames. Once the camera capture frame rate is higher than the CPU processing frame rate, the system triggers a saturation overflow mechanism. By forcibly discarding the old image frames at the bottom of the queue and pushing in the latest frame, the underlying architecture absolutely guarantees "zero latency" and strong real-time data processing in high-dynamic driving scenarios.

[0034] The dimming method based on high dynamic response image data includes: S1: Initialization parameters, including: preset light intensity threshold, preset strong light range threshold, dimming transition strategy, intrinsic parameters and distortion parameters of the first fixed-position outdoor camera group 1, the second fixed-position outdoor camera group 2 and the fixed-position indoor camera group 3.

[0035] The preset light intensity threshold is a dynamic threshold that is related to the average ambient brightness. The principle is that when the average ambient brightness is high, the preset light intensity threshold is low, and when the average ambient brightness is low, the preset light intensity threshold is high.

[0036] The preset high light range threshold is the minimum starting adjustment area. Although a small high light area has a high unit light intensity within the range, the overall light intensity is weak and does not affect driving. In addition, a small high light area has limited positioning accuracy and is prone to high dimming deviation.

[0037] The dimming transition strategy parameters include rules related to edge transition processing and high-speed strong light prediction dimming.

[0038] The above parameter initialization settings can be configured according to the driver's own preferences.

[0039] In the specific implementation process, during the initialization phase, the OpenCV library's class engine is called to read the configuration file mounted on the local file system in read-only mode, and the structured tree data in the file is deserialized into the configuration structure in memory.

[0040] The specific hard parameters for loading are as follows: (1) Core parameters for strong light detection: The preset light intensity threshold is set to 230 (that is, only extremely bright pixels with gray values ​​≥ 230 are processed, gray value range [0, 255], the preset strong light range threshold is set to 10 pixels, and the morphological noise reduction convolution kernel size is set to 5×5 pixels.

[0041] (2) Screen physical mapping parameters: The physical absolute width of the screen is set to 70.0cm, the physical absolute height is set to 42.0cm, and the display pixel grid size is set to 1366 pixels wide and 768 pixels high. The absolute position of the screen's Z-axis space is defined as -2.0cm (located in front of the human eye coordinate system). The X-axis offset of the screen's physical geometric center relative to the origin of the human eye coordinate system is precisely set to 17.2cm, and the Y-axis offset is precisely set to -15.5cm. The positive Y-axis direction of image rendering is set downwards.

[0042] (3) Coordinate system three-dimensional rigid body transformation parameters (to realize the benchmark unification from the strong light camera system to the human eye camera system): Load the translation vector T=[-9.25,-45.0,-6.7] cm from the strong light camera to the human eye, and the reverse / same direction rotation matrix R=[-0.9887,0,0.1499;0,1.0,0;-0.1499,0,-0.9887] which determines the orientation mapping of the three coordinate axes.

[0043] (4) Internal optical calibration parameters of the binocular system: extract the focal length fx=1133.3, fy=1128.4 and optical center cx=308.4513, cy=279.6438 of the left camera of the human eye, as well as the fifth-order distortion coefficients [-0.1450,-0.0133,0.0,0.0,0.0]; simultaneously extract the intrinsic parameters of the right camera and the epipolar correction rotation matrix and translation vector T=[-64.5334,0.075,-0.4250] generated by calibration. At the same time, load the corresponding intrinsic parameters of the left and right cameras of the high-intensity light system (such as fx=1347.4 of the left camera, etc.), and finally complete the memory construction of the mathematical model of the entire vehicle system.

[0044] S2: Acquire outdoor environment image data, compare light intensity pixel by pixel, and mark pixels whose light intensity exceeds the preset light intensity threshold, assigning values ​​according to the amount of light intensity exceeding; classify all adjacent marked pixels into a strong light range, retain marked pixels whose area in the strong light range is not less than the preset strong light range threshold, and form a pre-adjustment area; divide the area into several dimming sub-regions according to the values ​​assigned to the pixels in the pre-adjustment area.

[0045] The first fixed-position outdoor camera group 1 and the second fixed-position outdoor camera group 2, located at multiple different positions, can obtain corresponding full-element outdoor environment image data. The full-element outdoor environment image data includes basic image information such as pixel position, pixel color, and pixel light intensity.

[0046] like Figure 2 As shown. For a certain strong light source, if its imaging position in the fixed-position outdoor camera A is... The imaging position in the fixed-position outdoor camera B is The location of the strong light observed by the human eye was obtained through fitting. Then, when the line connecting the human eye and the light source does not pass through the two cameras mentioned above, the condition is satisfied. ≠ ≠ .

[0047] Then, combining the preset light intensity threshold and the preset strong light range threshold, the image data of the entire outdoor environment is compared pixel by pixel. When the light intensity of a pixel exceeds the preset light intensity threshold, the pixel is marked, and a number of non-continuous values ​​are assigned according to the amount of excess relative to the preset light intensity threshold. Depending on the computing power of the computing system and the driving of the dimming screen, the number of values ​​should be 2-30, and 5 values ​​can be assigned. The higher the number of values, the more delicate the dimming, but generally, 30 or less is sufficient to meet the dimming needs.

[0048] Adjacent marked pixels form a strong light range. The area of ​​the obtained strong light range pixels is compared with a preset strong light range threshold. When the area of ​​the strong light range pixels is not less than the preset strong light range threshold, the marked pixels are retained, and the retained marked pixels form a pre-dimming region. Based on the pixel values ​​in the pre-dimming region, i.e., the required dimming intensity, the pre-dimming region is divided into several dimming sub-regions. Each dimming sub-region has the same or similar dimming intensity. Similar means that the difference in dimming intensity is within 50% of the total number of levels.

[0049] In the specific implementation process, during the outdoor image processing thread, a 1280×480 resolution RAW format composite image is first extracted and hard-segmented into two independent 640×480 matrices (i.e., left and right views) using the Rect operator. Subsequently, epipolar correction remapping is performed on the matrices to eliminate lens optical distortion.

[0050] For the remapped color matrix, call the cvtColor function and utilize... (in y Represents the grayscale value of a single channel. r Representing the red channel, g Represents a green channel. b The floating-point weighted algorithm (representing the blue channel) collapses the RGB three-channel color of each pixel into a single-channel 8-bit grayscale (0-255) matrix.

[0051] Next, the binarization function accelerated by the AVX instruction set is executed. Using the preset light intensity threshold of 230 defined in the configuration, all array elements in the matrix below 230 are cleared to zero (turned black) through bitwise operations, and array elements not less than 230 are forcibly set to 255 (pure white).

[0052] The morphological filtering stage then begins, first constructing a 5×5 elliptical structuring element two-dimensional array. This array is then used as a sliding window to traverse the entire image and perform an erosion operation: replacing the central pixel with the local minimum within the window's coverage area, thereby eliminating fine free noise and burrs. Immediately following, a dilation operation is performed: replacing the central pixel with the local maximum within the window, thereby rebridging and mending the eroded and broken strong light core regions.

[0053] After binarization and noise reduction, the boundary tracking function based on the Suzuki85 algorithm is called. This algorithm performs clockwise vector topology tracing along the edges of connected regions with a pixel value of 255, and sets the parameters to RETR_EXTERNAL (reject nested extraction, only capture the outermost edge) and CHAIN_APPROX_SIMPLE (use the line segment endpoint compression method to discard redundant intermediate points on the same straight line).

[0054] A loop is then initiated, and for each acquired set of edge polygon points, Green's formula is called to calculate the absolute number of pixels enclosed by the closed region. Judgment logic is added to directly erase tiny light spots with a calculated area of ​​no more than 10 pixels from the memory container. The final set of compliant 2D polygon points is then packaged and stored in a structure, becoming the initial dimming core area boundary with high confidence.

[0055] S3: Acquire five facial images of the driver, locate the spatial coordinates of the pupils using a multi-view coordinate recognition method, and calculate the relative spatial position parameters of the pupils with respect to the fixed-position outdoor camera and the dimming screen.

[0056] Preferably, after acquiring the driver's facial image using the fixed-position interior camera group 3, the spatial coordinates of the pupil are located using a deep learning-based convolutional neural network human eye recognition method.

[0057] In the specific implementation, the interior image processing thread first reads single-frame matrix data from the interior infrared camera via non-blocking calls. To ensure compatibility with different computer vision libraries, a generic template class is used to wrap the OpenCV matrix address space into an image structure supported by the Dlib library with zero overhead, without any memory copying.

[0058] Next, the core detector of the Dlib library is instantiated. This detector uses a Histogram of Oriented Gradients (HOG) feature extractor combined with a linear support vector machine to perform a full-image sliding window convolution, successfully selecting the absolute boundary coordinates of the face.

[0059] Subsequently, the captured face bounding box is used as a parameter and fed into a cascaded regression tree (ERT) facial feature point predictor pre-initialized from the face model file. The predictor uses the regression tree forest to locate the two-dimensional pixel coordinates of 68 key facial feature points in milliseconds. Precise cropping is achieved using array indexing: six vertices with indices 36 to 41 are extracted to represent the left eyelid contour, and six vertices with indices 42 to 47 are extracted to represent the right eyelid contour.

[0060] The arithmetic mean of the summation of the X and Y axes and the division by 6 is calculated for each of the six coordinates to obtain the precise sub-pixel center coordinates of the left and right pupils.

[0061] After obtaining the 2D coordinates of the pupils in the left and right views, the binocular triangulation function, encapsulated by the mathematical principles of singular value decomposition and direct linear transformation, is called. This function substitutes the pre-initialized binocular projection matrices P1 and P2 and the corresponding 2D point sets into a system of homogeneous linear equations of the form Am=0 (where A is a constant and m represents the homogeneous coordinate matrix / solution vector), and solves to obtain a four-dimensional homogeneous coordinate matrix containing four components.

[0062] Divide the first three dimensions of the matrix (space vector) by the fourth dimension (homogeneous scaling factor). W The uncalibrated centimeter-level coordinates of the pupil in the reference physical three-dimensional coordinate system of the car camera were calculated.

[0063] Then, the coordinate is multiplied by the Z-axis scaling factor z_scale=1.2 written in the configuration to complete the calibration compensation from the camera coordinate system to the absolute coordinate system of the carriage.

[0064] Finally, the compensated 3D coordinates are injected into the instantiated Kalman filter container. This container maintains a 6×6 state transition probability matrix (using Newtonian kinematic linear equations to describe the object's position and velocity), through... The state extrapolation and gain correction covariance matrix operations are continuously performed with a fixed update step size every second. The final output is an extremely smooth three-dimensional coordinate of the pupil space (x_eye, y_eye, z_eye) that filters out the tiny high-frequency tremors of the human body (x_eye, y_eye, z_eye represent the coordinates of the pupil in the X, Y, and Z axes, respectively).

[0065] S4: Merge the light-adjusting sub-regions corresponding to the same strong light source from different outdoor shooting devices to obtain the superimposed light-adjusting sub-region; Based on the coordinate ratio of the driver's pupil to different outdoor shooting devices, the superimposed dimming area is mapped to the corresponding plane coordinates to obtain the position of the superimposed dimming area.

[0066] Step S4 is preceded by: Calculate the average RGB value of each dimming sub-region, classify them into suspected signal colors and non-signal colors, and optimize the dimming sub-regions of suspected signal colors.

[0067] For example, based on the initial settings or the driver's own preferences, the dimming intensity of the dimming sub-area of ​​suspected signal colors can be reduced, or dimming can be stopped altogether.

[0068] Suspected signal colors refer to colors that correspond to the commonly used red, green, and yellow land, sea, or air traffic indicator lights of the vehicle in question. These colors also include the vehicle's own navigation lights, brake lights, and other signal lights. All other colors are considered non-signal colors.

[0069] In the specific implementation process, a pure black (all pixels are 0) single-channel mask matrix of the same size (640×480) as the source image is allocated in the heap memory. Then, the drawContours function is called, and its fourth parameter, the line width, is forcibly set to -1, thereby triggering the polygon rasterization scan line filling algorithm, which draws each qualified set of strong light polygon points retained by S2 as a pure white solid block on the black mask.

[0070] Next, this black-and-white mask, along with the initial color exterior image, is passed as parameters to the mean function. This function iterates through the matrix at the lower level, checking the mask's state pixel by pixel. Only for the color image pixel regions in the mask corresponding to a pixel value of 255 (pure white), it accumulates the values ​​of the B, G, and R components, and finally divides by the total number of effective pixels to accurately determine the RGB physical average intensity of that specific luminous region.

[0071] After obtaining the average RGB value, a non-linear color space mathematical transformation formula is executed to convert the RGB model into the HSV model, which is more sensitive to human visual perception. In the OpenCV system, hue H is mapped to an integer dimension of a cylindrical coordinate system from 0 to 180.

[0072] Then, a nested set of conditional statements is executed: if the extracted hue value H falls within the red range of [0,10] or [160,180], the yellow range of [15,35], or the green range of [35,85], and its color saturation S parameter exceeds 50 (excluding the influence of strong white overexposure), then the area is determined to be a traffic light source. Once the conditions are met, the built-in boolean property of the BrightSpotInfo structure instance is immediately overridden, and it is completely discarded from the subsequent stereo positioning queue and rendering buffer pool, thus completely eliminating the danger of obscuring the traffic lights at the intersection from a physical mechanism perspective.

[0073] Before merging the dimming sub-regions corresponding to the same strong light source from different outdoor shooting devices as described in S4, it also includes scaling the coordinates of the dimming sub-regions obtained by different outdoor shooting devices along different axes using the same spatial reference system.

[0074] In the specific implementation process, the dimming zone coordinates of the first fixed-position outdoor camera group 1 and the second fixed-position outdoor camera group 2, which have different axial coordinates (i.e., shooting direction normals), are first scaled equiaxially according to the same reference frame, so that they are close to the center of the photograph. Since there is no distance information, one or more fixed scaling factors in one dimension are needed for approximation. The scaling factor in each dimension can be calculated by mathematical calculation or experiment of the optical path to obtain a numerical range for the actual distance range where strong light sources usually appear, and then a representative value is taken from it.

[0075] Therefore, the size of the dimming area captured and identified by the first fixed-position outdoor camera group 1 and the second fixed-position outdoor camera group 2, after being weighted by an isoaxial scaling factor, should be consistent with the size subjectively observed by the driver 5's naked eye. In practice, since the actual distance of the light source is much greater than the distance between the pupils and the first fixed-position outdoor camera group 1 and the second fixed-position outdoor camera group 2, if the accuracy requirement is not high or the computing power is limited, isoaxial scaling can be omitted or the default isoaxial scaling factor of 1 can be used.

[0076] Then, taking the same dimming sub-region as a unit, the same dimming sub-region captured by different fixed-position outdoor cameras from the same strong light source is superimposed, retaining the overlapping and non-overlapping parts, to obtain the area and graphic shape information of the superimposed dimming sub-region, and the strong light pixel assignment of the merged dimming sub-region remains unchanged.

[0077] Furthermore, as described in S4, the superimposed light-adjusting area is mapped to the corresponding planar coordinates according to the coordinate ratio between the driver's pupil and different outdoor shooting devices to obtain the position of the superimposed light-adjusting area, specifically: Based on the reference horizontal coordinates of the superimposed dimming areas of the same strong light source in different outdoor shooting devices, and the horizontal distance between the driver's single pupil and each outdoor shooting device, the horizontal coordinates are mapped to obtain the target horizontal coordinates. Based on the reference vertical coordinates of the superimposed dimming areas of the same strong light source in different outdoor shooting devices, and the vertical distance between the driver's single pupil and each outdoor shooting device, vertical coordinate mapping is performed to obtain the target vertical coordinates. The location of the superimposed photodiode region is obtained by combining the target's horizontal and vertical coordinates.

[0078] For example, suppose the horizontal coordinate of a point light source in the image of the first fixed-position outdoor camera group 1 is... Its horizontal coordinates in the image of the second fixed-position outdoor camera group 2 are The horizontal distance between the driver's pupil and the first fixed-position external camera group 1 is The horizontal distance between the pupil and the second fixed-position outdoor camera group 2 is Then the horizontal coordinate of the light source relative to the pupil is... for: .

[0079] Similarly, to improve accuracy, the obtained position of the photodiode region is scaled equiaxially, with a scaling factor of 1.

[0080] When there are more than two cameras, prioritize dynamically selecting the two cameras with the smallest line-of-sight angle and the best image quality from the camera group (e.g., the main cameras located on the left and right sides respectively), and then directly apply the above formula.

[0081] The above describes the core algorithm for eliminating parallax. This fitting method obtains the driver's position and field of vision's composition information by processing the captured information of the surrounding environment. Traditional autonomous driving vision systems require computationally expensive stereo matching algorithms to obtain the absolute depth along the Z-axis to generate 3D point clouds. In contrast, this method uses mathematical dimensionality reduction logic to directly perform zero-depth parallax algebraic elimination in the two-dimensional pixel domain.

[0082] For the left and right view dimming contour point sets determined to be non-signal light, the absolute geometric centroid coordinates of the contours are obtained by calling the moments image spatial moment function. The horizontal distance between the pupil and the fixed-position exterior cameras A and B is retrieved from shared memory using a thread mutex lock. and After simplifying the contour vertices using the algorithm, a high-speed traversal loop is initiated. The coordinates of each independent two-dimensional point on the contour boundary are extracted and substituted into the aforementioned formula. Through this low-level algebraic operation that linearly maps the parallax offset rate to the pupil distance distribution rate, the two distorted polygons with viewing angle deviations captured by the high-light camera are retranslated and fused into a single two-dimensional point array. This array is the "parallax-free superimposed contour" that conforms to the current driver's physical eye line of sight. This method directly eliminates parallax in the two-dimensional pixel domain, completely abandoning three-dimensional depth calculation and minimizing computational overhead.

[0083] S5: Using the pupil as the origin, extend the superimposed dimming sub-area along the backlight path of the shooting angle, calculate the intersection position of the extended path and the spatial coordinates of the dimming screen, and determine the dimming area.

[0084] The dimming area is defined as the closed area formed by the intersection position and the boundary of the dimming screen 4.

[0085] like Figure 3 As shown, based on the plane 7 where the superimposed dimming sub-region is located, the superimposed dimming sub-region 6 extends along the backlight path of the shooting angle. The intersection position of the extension path and the spatial coordinates of the dimming screen 4 is calculated. For areas that exceed the range of the driver's physiological visual angle 9, they are directly ignored because they are beyond the range of human vision. Finally, the dimming area 8 is determined. θ The angle between the driver's line of sight 5 and the normal of the dimming screen 4 is the reference angle for the extension of the backlight path.

[0086] In the specific implementation process, the spatial coordinates of the dimming screen 4 are obtained according to the spatial plane mathematical description of the dimming screen 4. The obtained superimposed dimming sub-region 6 is extended from the pupil position along the backlight path of the shooting angle of the first fixed position outdoor camera group 1 and the second fixed position outdoor camera group 2, i.e., outward. The intersection position of the dimming screen 4 with its coordinates is the position where the dimming screen needs to dim.

[0087] In the underlying program implementation of this step, the calculation of the actual physical depth (Z-axis coordinate) of the external strong light source is completely abandoned. First, the three-dimensional coordinates (x_eye, y_eye, z_eye) of the driver's five pupils are extracted as the geometric viewpoint center.

[0088] Next, the superimposed dimming sub-region generated in S4 is mapped onto a preset relative virtual imaging plane, which is defined at a position in front of the pupil equal to the camera's calibrated focal length. A calculation loop is initiated, using the pupil coordinates as the sole starting point and each two-dimensional contour vertex on the virtual imaging plane as a direction guide point for spatial vectors, constructing a cluster of geometric three-dimensional ray vectors "shooting forward from the eye".

[0089] Since the physical shape and spatial parameters of the dimming screen 4 within the carriage are known constants, the dimming screen 4 is abstracted into spatial mathematical equations (such as planar equations or parametric surface equations). Each spatial ray equation constructed simultaneously is subjected to algebraic intersection operations with the mathematical equations of the dimming screen 4 to directly calculate the physical intersection points (x_screen, y_screen, z_screen) between the line-of-sight beam and the physical dimming screen.

[0090] Subsequently, using a coordinate system transformation algorithm, these absolute physical intersection points are subtracted from the screen's offset relative to the origin (17.2cm, -15.5cm), and then proportionally mapped to the 1366×768 liquid crystal electrical signal pixel grid according to the screen's physical size of 70×42cm.

[0091] Finally, the polygon scan line filling algorithm is called to forcibly overwrite the closed polygon formed by connecting the intersections of these mapped pixels on the mask matrix in main memory with the value 255 (pure white occlusion area).

[0092] The entire calculation process described above utilizes only the ray-penetration projection logic between the pupil, virtual 2D outline, and display screen, fundamentally eliminating the dependence on the three-dimensional coordinates of external light sources and perfectly achieving high dynamic and high-precision occlusion that does not depend on external distance.

[0093] S6: Perform edge transition processing on dimming area 8, and generate a pre-judged dimming area along the direction of movement of the strong light source, and generate a dimming screen dimming instruction set.

[0094] The edge transition processing is used to compensate for incomplete shading caused by dimming accuracy and to eliminate the discontinuity between the shading and non-shading areas.

[0095] Preferably, the edge transition processing of the dimming area 8 specifically includes: The dimming area 8 extends outward by a preset distance, and the dimming intensity is reduced along the extension direction, for example, in a linear manner, until the dimming intensity at the outer boundary of the extension area drops to 0.

[0096] In the specific implementation process, in order to avoid the driver's pupils from contracting violently due to the harsh edges of the black blocks on the screen, the generated mask containing the pure white blocks is subjected to bitwise inversion based on the AVX instruction set: the arithmetic instruction is executed to change the pure white core light-blocking area to the value 0, and all other background pixels to non-zero.

[0097] Subsequently, the computationally intensive distance transformation function is invoked. This function employs a chamfered distance calculation logic using two raster scans, specifying a mask template based on Euclidean metric and a 5×5 size. For each of the millions of non-zero background pixels on the screen, it calculates the shortest straight-line distance between each pixel and the nearest core occlusion area boundary with a value of 0 (the distance increases linearly with the outward extension). A floating-point threshold constant, maxDistance = 50.0, is set (the number of pixels corresponding to a 1mm physical span calculated from the liquid crystal pixel density). The threshold processing function is then called, passing in a truncation parameter, to forcibly truncate all values ​​in the matrix with a calculated distance greater than 50.0 to 50.0.

[0098] Next, the matrix multiplication overload operator is invoked to multiply all floating-point distance values ​​across the entire map by a scaling factor (255.0 / 50.0), and the memory data type is forcibly converted from 32-bit floating-point to 8-bit unsigned integer. This operation generates a smoothly decaying terrain map in the matrix, transitioning smoothly from an inner pixel value of 0 to an outer pixel value of 255.

[0099] Finally, a Gaussian blur function is applied to the topographic map matrix, and a 15×15 large-size Gaussian bell-shaped probability kernel is used for two-dimensional spatial convolution to deeply smooth out the linear Mach band stripe visual artifacts caused by integer conversion truncation. The grayscale matrix array generated in this stage is finally converted into graphics card driver interface commands and output to the dimming LCD screen to accurately present a perfectly soft black spot with 5-level grayscale steps and a gradual attenuation from dark to light.

[0100] In addition, the generation of the pre-judgment dimming area is to address the issue of dimming lag or delay caused by limited computing power or camera frame rate when strong light sources move rapidly relative to the vehicle in high-speed scenarios.

[0101] Furthermore, the generation of the pre-judged dimming area along the direction of movement of the strong light source specifically includes: Detect the motion trajectory of strong light sources in outdoor environmental image data and calculate the relative speed of the strong light sources to the vehicle. The running speed is compared with multiple preset high-speed thresholds. If the speed exceeds the lowest preset high-speed threshold, the speed is classified and recorded. For strong light sources with different speed levels, a pre-defined dimming area is generated along the direction of motion, and the path length is positively correlated with the speed level.

[0102] In practical implementation, to anticipate occlusion delays caused by frame rate limitations during high-speed driving, this invention allocates a dedicated Kalman filter memory object independently for each tracked strong light source entity. The state transition matrix of this object is defined as follows: .(in The system state vector represents the current moment. This represents the system state vector from the previous moment. k Represents the state transition matrix. To control the input matrix, As an external control input, in the scenario where this invention is applied, it is typically only possible to "observe" oncoming vehicles, but not to actively "control" their movement. Therefore, external control input... It is set to 0 by default, and only keeps [the value]. Make pure physics predictions based on inertia and velocity.

[0103] As the video progresses frame by frame, the system state for the current period is obtained by continuously extracting the posterior frame update matrix object. This is achieved through algebraic expressions. (in It refers to The velocity components of the shaft, It refers to The velocity vector magnitude of the current strong light source is calculated in real time (the velocity component of the axis). When the magnitude value exceeds the preset first-level discrete high-speed threshold variable (e.g., 5.0 cm / s) in the code, the corresponding advance prediction time constant factor T_predict is immediately extracted through multi-branch judgment logic based on the order in which the velocity falls.

[0104] The current velocity vector is then multiplied by the prediction constant to derive the two-dimensional physical pixel offset vector that the system will inevitably experience in the next moment due to the delay. When rendering screen graphics, instead of simply drawing the original array of light spot vertices, an additional loop is used to accumulate the offset vectors of all vertices on the original contour to generate a set of predicted points.

[0105] Finally, the polygon scan line function is called to wrap and stitch the original point set boundary with the predicted point set boundary using the convex hull algorithm. This is displayed on the screen as a comet-shaped "tail" black polygon that grows out of the central black block in the opposite direction of the velocity. This geometric structure can preemptively implement spatial physical occlusion interception along the trajectory of the future high dynamic light source in the next millisecond interval when the light source undergoes a violent physical displacement.

[0106] This invention eliminates the need for spatial coordinate positioning based on strong light sources. By fitting an external image from the driver's field of vision and combining this with the driver's pupil positioning, the backlight path of the fitted dimming area is extended to the dimming screen to determine the dimming position. This effectively solves the problem of inaccurate positioning of strong external light in existing active dimming technologies. Furthermore, by performing transition processing and pre-judgment softening on obstructed areas, the error tolerance of the field of vision optical path is effectively improved. This allows for error correction of potential shading deviations such as bumps and shading omissions during high-speed vehicle travel, thus meeting the driver's observation comfort requirements.

[0107] The system employs intelligent dimming that is selective, positional, and gentle on the viewer in response to the dynamic movement of the vehicle and the changing position of external light. At the same time, it does not actively block light from the external field of view that is not strong, which greatly ensures the driver's sensitivity to environmental observation, especially in low-light environments. It also avoids blocking important potential traffic signs such as red and green lights, thus ensuring driving safety.

[0108] The present invention also provides a dimming system based on high dynamic response image data, comprising: Initialization module: Used to initialize parameters, including: preset light intensity threshold, preset strong light range threshold, and dimming transition strategy parameters; The dimming area division module is used to acquire outdoor environment image data, compare light intensity pixel by pixel, and mark pixels whose light intensity exceeds a preset light intensity threshold, assigning values ​​according to the amount of light intensity exceeding the threshold; classify all adjacent marked pixels into a strong light range, retain marked pixels whose area in the strong light range is not less than the preset strong light range threshold, and form a pre-dimming area; divide the area into several dimming sub-regions according to the values ​​assigned to the pixels in the pre-dimming area. Pupil positioning module: used to acquire driver's facial image, locate pupil spatial coordinates through multi-view coordinate recognition method, and calculate the relative position parameters of pupil with respect to the external shooting device and dimming screen; Dimming area fitting module: used to merge the dimming sub-regions corresponding to the same strong light source from different outdoor shooting devices to obtain superimposed dimming sub-regions; Based on the coordinate ratio of the driver's pupil to different outdoor shooting devices, the superimposed dimming area is mapped to the corresponding plane coordinates to obtain the position of the superimposed dimming area; Dimming area determination module: Taking the pupil as the origin, the superimposed dimming sub-area is extended along the backlight path of the shooting angle, and the intersection position of the extended path and the spatial coordinates of the dimming screen is calculated to determine the dimming area.

[0109] The present invention also provides a dimming device based on high dynamic response image data, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the dimming method based on high dynamic response image data.

[0110] The present invention also provides a medium for storing a computer program, wherein the computer program, when executed by a processor, implements the dimming method based on high dynamic response image data.

[0111] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A dimming method based on high dynamic response image data, characterized in that, include: S1: Initialization parameters, including: preset light intensity threshold, preset strong light range threshold, and dimming transition strategy parameters; S2: Acquire outdoor environment image data, compare light intensity pixel by pixel, and mark pixels whose light intensity exceeds the preset light intensity threshold, assigning values ​​according to the amount of light intensity exceeding the threshold; classify all adjacent marked pixels into a strong light range, retain marked pixels whose area in the strong light range is not less than the preset strong light range threshold, and form a pre-adjustment area; divide the area into several dimming sub-regions according to the values ​​assigned to the pixels in the pre-adjustment area. S3: Acquire the driver's facial image, locate the pupil spatial coordinates using a multi-view coordinate recognition method, and calculate the relative position parameters of the pupil with respect to the external shooting device and the dimming screen; S4: Merge the light-adjusting sub-regions corresponding to the same strong light source from different outdoor shooting devices to obtain the superimposed light-adjusting sub-region; Based on the coordinate ratio of the driver's pupil to different outdoor shooting devices, the superimposed dimming area is mapped to the corresponding plane coordinates to obtain the position of the superimposed dimming area; S5: Using the pupil as the origin, extend the superimposed dimming sub-area along the backlight path of the shooting angle, calculate the intersection position of the extended path and the spatial coordinates of the dimming screen, and determine the dimming area.

2. The dimming method based on high dynamic response image data according to claim 1, characterized in that, S4 describes mapping the superimposed dimming area to the corresponding planar coordinates according to the coordinate ratio between the driver's pupil and different outdoor shooting devices to obtain the position of the superimposed dimming area, specifically as follows: Based on the reference horizontal coordinates of the superimposed dimming areas of the same strong light source in different outdoor shooting devices, and the horizontal distance between the driver's single pupil and each outdoor shooting device, the horizontal coordinates are mapped to obtain the target horizontal coordinates. Based on the reference vertical coordinates of the superimposed dimming areas of the same strong light source in different outdoor shooting devices, and the vertical distance between the driver's single pupil and each outdoor shooting device, vertical coordinate mapping is performed to obtain the target vertical coordinates. The location of the superimposed photodiode region is obtained by combining the target's horizontal and vertical coordinates.

3. The dimming method based on high dynamic response image data according to claim 1, characterized in that, S5 also includes edge transition processing of the dimming area, and generating a pre-judged dimming area along the direction of movement of the strong light source, and generating a dimming screen dimming instruction set.

4. The dimming method based on high dynamic response image data according to claim 3, characterized in that, The edge transition processing of the dimming area specifically includes: The dimming area extends outward by a preset distance, and the dimming intensity is reduced along the extension direction until the dimming intensity at the outer boundary of the extended area drops to 0.

5. The dimming method based on high dynamic response image data according to claim 3, characterized in that, The generation of the pre-judged dimming area along the direction of movement of the strong light source is specifically as follows: Detect the motion trajectory of strong light sources in outdoor environmental image data and calculate the relative speed of the strong light sources to the vehicle. The running speed is compared with multiple preset high-speed thresholds. If the speed exceeds the lowest preset high-speed threshold, the speed is classified and recorded. For strong light sources with different speed levels, a pre-defined dimming area is generated along the direction of motion, and the path length is positively correlated with the speed level.

6. The dimming method based on high dynamic response image data according to claim 1, characterized in that, Before merging the dimming sub-regions corresponding to the same strong light source from different outdoor shooting devices as described in S4, it further includes scaling the coordinates of the dimming sub-regions obtained by the outdoor shooting devices with different axes using the same spatial reference system.

7. The dimming method based on high dynamic response image data according to claim 1, characterized in that, The dimming area described in S5 is a closed area formed by the intersection of the dimming position and the boundary of the dimming screen.

8. A dimming system based on high dynamic response image data, characterized in that, include: Initialization module: Used to initialize parameters, including: preset light intensity threshold, preset strong light range threshold, and dimming transition strategy parameters; The dimming area division module is used to acquire outdoor environment image data, compare light intensity pixel by pixel, and mark pixels whose light intensity exceeds a preset light intensity threshold, assigning values ​​according to the amount of light intensity exceeding the threshold; classify all adjacent marked pixels into a strong light range, retain marked pixels whose area in the strong light range is not less than the preset strong light range threshold, and form a pre-dimming area; divide the area into several dimming sub-regions according to the values ​​assigned to the pixels in the pre-dimming area. Pupil positioning module: used to acquire driver's facial image, locate pupil spatial coordinates through multi-view coordinate recognition method, and calculate the relative position parameters of pupil with respect to the external shooting device and dimming screen; Dimming area fitting module: used to merge the dimming sub-regions corresponding to the same strong light source from different outdoor shooting devices to obtain superimposed dimming sub-regions; Based on the coordinate ratio of the driver's pupil to different outdoor shooting devices, the superimposed dimming area is mapped to the corresponding plane coordinates to obtain the position of the superimposed dimming area; Dimming area determination module: Taking the pupil as the origin, the superimposed dimming sub-area is extended along the backlight path of the shooting angle, and the intersection position of the extended path and the spatial coordinates of the dimming screen is calculated to determine the dimming area.

9. A dimming device based on high dynamic response image data, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the dimming method based on high dynamic response image data as described in any one of claims 1-7.

10. A medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the dimming method based on high dynamic response image data as described in any one of claims 1-7.

Images