Environment perception and decision-making method and system of large model in automatic driving

Abstract

The invention provides an environmental perception and decision-making method and system for a large model in automatic driving, and the method comprises the steps: generating light intensity distribution data carrying Stokes parameters through obtaining an original road image of a multispectral channel; calculating a polarization spectrum coupling coefficient of weather medium scattering noise based on a Stokes parameter and the wavelength of a multispectral channel, performing compensation processing on the original road image by combining the light intensity distribution data, generating a polarization spectrum joint compensation image, and performing smoothing and strengthening processing on the polarization spectrum joint compensation image to generate a multi-level feature map; performing noise suppression and feature propagation operation on the multi-level feature map to reconstruct an object contour shielded by a weather medium, obtaining object space distribution after reconstruction, and generating an obstacle avoidance control instruction of the autonomous vehicle in combination with road boundary geometric features; according to the technical scheme provided by the invention, the driving safety and the autonomous navigation capability under complex weather conditions are improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and system for environmental perception and decision-making of a large model in autonomous driving. Background Art

[0002] In low-visibility weather conditions such as heavy rain or fog, autonomous vehicles face significant challenges in identifying road boundaries and obstacles. Severe weather conditions can degrade the quality of information obtained by sensors. For example, camera images may become blurred, and lidar data may be subject to errors due to interference from water droplets or fog particles. Therefore, in such environments, vehicles need to rely on highly accurate perception systems to accurately identify their surroundings and ensure safe driving. Furthermore, autonomous driving systems must be able to quickly respond to changing road conditions, including sudden obstacles or sharp turns. This requires perception technology to be both highly accurate and capable of real-time processing.

[0003] Currently, a widely used approach is to use deep learning models to analyze and process multi-sensor fusion data, particularly by combining visible light and infrared imaging technologies to enhance visual perception in adverse weather conditions. However, existing approaches have some significant drawbacks. For example, deep learning models rely on large amounts of labeled data for training, making data collection difficult for rare or extreme weather conditions, resulting in poor model performance in these scenarios. Furthermore, it is difficult to accurately compensate for optical noise caused by weather, leading to inaccurate decision-making. Summary of the Invention

[0004] The present invention provides a method and system for environmental perception and decision-making of a large model in autonomous driving, which is used to solve the problems in the existing technology such as poor model performance in rare or extreme weather conditions, difficulty in accurately compensating for optical noise caused by weather, and inaccurate final decision results.

[0005] In a first aspect, the present invention provides a method for environmental perception and decision-making using a large model in autonomous driving, comprising: Obtain original road images with multispectral channels; generating light intensity distribution data carrying Stokes parameters based on the polarization state distribution of the original road image; Calculating the polarization spectral coupling coefficient of the weather medium scattering noise based on the Stokes parameters and the wavelength of the multi-spectral channel; performing compensation processing on the original road image according to the polarization-spectrum coupling coefficient and the light intensity distribution data to generate a polarization-spectrum joint compensation image, and performing smoothing and enhancement processing on the polarization-spectrum joint compensation image to generate a multi-level feature map; Performing noise suppression and feature propagation operations on the multi-level feature maps to reconstruct the contours of objects obscured by weather media and obtain a reconstructed spatial distribution of the objects; Obstacle avoidance control instructions for the autonomous driving vehicle are generated based on the reconstructed spatial distribution of objects and the geometric features of the road boundaries.

[0006] Optionally, calculating the polarization spectral coupling coefficient of the weather medium scattering noise based on the Stokes parameters and the wavelength of the multispectral channel includes: Based on the Stokes parameters, the polarization ellipse azimuth is calculated to determine the light intensity component distribution in each polarization direction; Dividing the wavelength of the multispectral channel into a high scattering sensitivity band and a low scattering sensitivity band according to a scattering sensitivity threshold; Calculating the difference in light intensity component distribution corresponding to the high scattering sensitive band and the low scattering sensitive band, and calculating the discrete differential value of the light intensity component distribution difference as the wavelength of the multispectral channel changes; Performing a negative logarithmic transformation operation on the atmospheric transmittance parameter to generate an atmospheric optical action quantity, and combining the discrete differential value to generate an uncorrected coupling quantity; The uncorrected coupling amount is linearly corrected according to the atmospheric density parameter to generate a corrected coupling amount, and the corrected coupling amount is used as the polarization spectrum coupling coefficient of the weather medium scattering noise.

[0007] Optionally, performing a linear correction on the uncorrected coupling amount according to an atmospheric density parameter to generate a corrected coupling amount, and using the corrected coupling amount as a polarization spectrum coupling coefficient of weather medium scattering noise, includes: Obtain real-time atmospheric density values ​​and call atmospheric density reference values ​​from a pre-stored meteorological parameter database; Calculating a density ratio of the real-time atmospheric density value to the atmospheric density reference value, and performing boundary truncation processing on the density ratio to obtain a truncated density ratio; Inputting the truncated density ratio into a dual-interval linear function to obtain a density correction factor, wherein the dual-interval linear function includes a first linear function and a second linear function, the first linear function is used when the density ratio is less than or equal to a preset critical value, and the second linear function is used when the density ratio is greater than the preset critical value; A scalar multiplication operation is performed on the density correction factor and the uncorrected coupling amount to generate a corrected coupling amount, and the corrected coupling amount is used as the polarization spectrum coupling coefficient of the weather medium scattering noise.

[0008] Optionally, the original road image is compensated according to the polarization-spectrum coupling coefficient and the light intensity distribution data to generate a polarization-spectrum joint compensation image, and the polarization-spectrum joint compensation image is smoothed and enhanced to generate a multi-level feature map, including: constructing a polarization modulation factor matrix based on the polarization spectral coupling coefficient and the Stokes parameter; Performing a dot multiplication operation on the light intensity distribution data using the polarization modulation factor matrix to generate polarization-enhanced light intensity data; Calculating the light intensity ratio of the polarization-enhanced light intensity data and the original road image to generate a spectral attenuation ratio parameter based on the light intensity ratio, and generating an attenuation compensation coefficient corresponding to each spectral channel in combination with real-time visibility; performing compensation processing on the original road image according to the attenuation compensation coefficient to generate a polarization-spectrum joint compensation image; performing smoothing processing on the polarization spectrum joint compensation image to generate a basic feature map, and performing texture enhancement processing on the basic feature map to generate a texture feature map; The basic feature map and the texture feature map are spliced ​​in channel dimension to generate a multi-level feature map.

[0009] Optionally, calculating the light intensity ratio of the polarization-enhanced light intensity data and the original road image to generate a spectral attenuation ratio parameter based on the light intensity ratio, and generating an attenuation compensation coefficient corresponding to each spectral channel in combination with real-time visibility, including: Calculating the light intensity ratio of the polarization-enhanced light intensity data to the original road image, and performing a logarithmic domain conversion operation on the light intensity ratio to generate a logarithmic attenuation parameter; Obtain real-time visibility and call pre-stored benchmark visibility; Calculating a visibility offset between the real-time visibility and the reference visibility; linearly weighting the logarithmic attenuation parameter and the visibility offset to generate an unnormalized compensation parameter; performing an exponential domain inverse transform operation on the unnormalized compensation parameter to generate a spectral attenuation ratio parameter; The spectral attenuation ratio parameter is multiplied by the real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel.

[0010] Optionally, performing noise suppression and feature propagation operations on the multi-level feature maps to reconstruct the contours of objects obscured by weather media to obtain a reconstructed spatial distribution of the objects, including: Calculating a statistical correlation value of each channel based on the channel data of the multi-level feature map, and comparing the statistical correlation value with a preset threshold to generate a channel activation state identifier; Determining a channel weight vector according to the channel activation state identifier; Fusing the multi-level feature map with the channel weight vector to generate a noise suppression feature map; Performing multiple rounds of iterative feature propagation operations on the noise suppression feature map, terminating the feature propagation operation when the number of iterative rounds reaches a preset number, and generating a diffusion stabilization feature map; A second-order gradient tensor calculation is performed on the diffusion stability feature map to obtain the feature change rate tensor of each pixel point to determine the distribution of curvature extreme value points. The curvature extreme value points of adjacent pixels are connected according to the curvature extreme value point distribution to reconstruct the outline of the object obscured by the weather medium and generate the reconstructed object spatial distribution.

[0011] Optionally, generating obstacle avoidance control instructions for the autonomous driving vehicle based on the reconstructed spatial distribution of objects and road boundary geometric features includes: Based on the origin of the reference coordinate system of the reconstructed object space distribution, the origin of the ego-vehicle coordinates is defined, and the position vectors corresponding to each discrete point on the obstacle surface relative to the reference coordinate system are extracted; Perform semantic segmentation on the road boundary in the polarization-spectral joint compensation image to generate an initial road boundary mask; Performing topological optimization on the initial road boundary mask to generate road boundary geometric features, so as to fit the boundary constraint equation of the drivable area according to the road boundary geometric features; orthogonally projecting each discrete point onto the reference coordinate system according to the position vector to generate an obstacle projection point set; Calculating the minimum Euclidean distance vector between the obstacle projection point set and the vehicle coordinate origin to solve the obstacle avoidance direction offset angle, combining the boundary constraint equation to construct a fan-shaped safety area, and generating a parameterized trajectory cluster within the fan-shaped safety area; Calculating the average value of the distance change rate from the trajectory point set to the obstacle projection point set based on the parameterized trajectory cluster, and taking the parameterized trajectory with the smallest average distance change rate as the optimal obstacle avoidance trajectory; Generate obstacle avoidance control instructions for the autonomous driving vehicle based on the optimal obstacle avoidance trajectory.

[0012] In a second aspect, the present invention provides an environment perception and decision-making system for a large model in autonomous driving, comprising: An acquisition module is used to obtain original road images with multispectral channels; A first generating module is configured to generate light intensity distribution data carrying Stokes parameters based on the polarization state distribution of the original road image; a calculation module, configured to calculate a polarization spectral coupling coefficient of weather medium scattering noise based on the Stokes parameters and the wavelength of the multispectral channel; a compensation module, configured to perform compensation processing on the original road image according to the polarization-spectrum coupling coefficient and the light intensity distribution data to generate a polarization-spectrum joint compensation image, and to perform smoothing and enhancement processing on the polarization-spectrum joint compensation image to generate a multi-level feature map; a processing module, configured to perform noise suppression and feature propagation operations on the multi-level feature maps to reconstruct the contours of objects obscured by weather media and obtain a reconstructed spatial distribution of the objects; The second generation module is used to generate obstacle avoidance control instructions for the autonomous driving vehicle based on the reconstructed object spatial distribution and road boundary geometric characteristics.

[0013] In a third aspect, the present invention provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a large model environmental perception and decision-making method in autonomous driving as described in the first aspect above.

[0014] In a fourth aspect, the present invention provides a computer storage medium storing a computer program. When the computer program is executed by a computer, it implements the environmental perception and decision-making method of a large model in autonomous driving as described in the first aspect.

[0015] In the present invention, an original road image of a multi-spectral channel is obtained; based on the polarization state distribution of the original road image, light intensity distribution data carrying Stokes parameters is generated; based on the Stokes parameters and the wavelength of the multi-spectral channel, the polarization spectrum coupling coefficient of the weather medium scattering noise is calculated; according to the polarization spectrum coupling coefficient and the light intensity distribution data, the original road image is compensated to generate a polarization spectrum joint compensation image, and the polarization spectrum joint compensation image is smoothed and enhanced to generate a multi-level feature map; noise suppression and feature propagation operations are performed on the multi-level feature map to reconstruct the outline of objects obscured by the weather medium to obtain the reconstructed object spatial distribution; according to the reconstructed object spatial distribution and the road boundary geometric characteristics, an obstacle avoidance control instruction for the autonomous driving vehicle is generated. The technical solution provided by this invention overcomes the limitations of visible light through multispectral imaging, capturing original information about roads and obstacles obscured by weather media in heavy rain or foggy scenes, thus addressing the information loss of existing single-spectral sensors in extreme weather conditions. It utilizes polarization state distribution to analyze the directional characteristics of light wave propagation, separating scattered noise from light reflected from real objects, overcoming the core issue of image blur caused by scattering of rain and fog particles in existing solutions. Physical modeling quantifies the differential scattering effects of weather media on multispectral channels, enabling accurate assessment of noise intensity, thus overcoming the limitations of traditional deep learning models that rely on labeled data and cannot model dynamic scattering patterns. Scattered noise is used to inversely compensate for light intensity distortion and restore texture details of objects obscured by weather. Smoothing and enhancement processing can extract cross-resolution features, improving the robustness of edge and contour recognition in low-visibility environments. Channel statistical characteristics are integrated with geometric topology reconstruction to restore the complete three-dimensional structure of obscured objects under strong noise interference, addressing the object fracture and deformation distortion problems caused by existing segmentation methods in rain and fog. Optimal trajectories are generated by incorporating the spatial geometric constraints of the reconstructed environment, enabling real-time safety decision-making under complex weather conditions and addressing the response delays and trajectory planning failures caused by perception distortion in existing systems. Furthermore, the light intensity distribution in the polarization direction is calculated based on the Stokes parameters, the spectral bands are divided according to the scattering sensitivity, and the discrete differential relationship of the light intensity difference between the bands is established; the negative logarithmic transformation of the atmospheric transmittance is integrated to generate the uncorrected coupling parameter, and finally it is dynamically corrected by the real-time atmospheric density to generate the polarization spectrum coupling coefficient; the physical characteristics of the scattering noise are quantified by polarization spectrum coupling, overcoming the defects of the existing deep learning model that relies on massive labeled data and cannot explain the root cause of optical interference, and reducing the dependence on the scale and quality of training data; real-time correction of atmospheric density is introduced to make the polarization spectrum coupling coefficient automatically adapt to changes in rain and fog concentration, solve the problem that the traditional fixed parameter compensation method fails under sudden weather changes, and improve the perception stability of complex meteorological conditions.

[0016] These and other aspects of the present invention will become more readily apparent from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are only some embodiments of the present application, and the drawings can be obtained by those skilled in the art without creative effort.

[0018] Figure 1 A flow chart of an environment perception and decision method of a large model in autonomous driving provided by the present application is shown; Figure 2 A structural schematic diagram of an environment perception and decision system of a large model in autonomous driving provided by the present application is shown; Figure 3 A structural schematic diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0019] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0020] In some of the descriptions in the specification and claims of the present application and the above-mentioned drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that these operations can be executed in the order appearing in this text or in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this text are used to distinguish different messages, devices, modules, etc., and do not represent the order of sequence, nor do "first" and "second" represent different types.

[0021] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present application.

[0022] In low-visibility weather conditions like heavy rain or fog, autonomous vehicles face significant challenges in identifying road boundaries and obstacles. This is primarily due to a significant degradation in the quality of information acquired by sensors, such as blurred camera images and interference with LiDAR data. Existing technologies rely on large amounts of labeled data for training, making data collection for rare or extreme weather conditions challenging. This limits model performance in these scenarios and makes it difficult to effectively compensate for perception errors caused by optical noise, impacting decision accuracy. Figure 1 The present invention provides a flowchart of a method for environmental perception and decision-making of a large model in autonomous driving. Figure 1 As shown, the method includes: Step 101: Acquire a multi-spectral channel original road image; In this step, the original road image refers to the unprocessed road scene data directly obtained through the multispectral imaging system, which contains pixel-level radiation intensity information of multiple spectral channels such as visible light, near-infrared, and short-wave infrared, and is used to reflect the original environmental state interfered by scattered noise in rainy and foggy weather.

[0023] In an embodiment of the present invention, raw road image data in the visible, near-infrared, and short-wave infrared bands is synchronously collected by a vehicle-mounted multispectral imaging device, where each pixel contains radiation intensity information from multiple spectral channels. This step uses a spectroscopic prism and a multi-channel sensor array to achieve spectral separation, ensuring the ability to capture different wavelengths penetrating the medium even in heavy rain or fog.

[0024] Step 102: generating light intensity distribution data carrying Stokes parameters based on the polarization state distribution of the original road image; In this step, the polarization state distribution refers to the intensity distribution characteristics of the polarization direction formed after the incident light wave is reflected on the surface of the object. It is calculated by the intensity difference in the four polarization directions of 0°, 45°, 90°, and 135°, and is used to separate the medium scattering noise from the real object reflection signal. Stokes parameters refer to the group of physical quantities (S0, S1, S2) that describe the polarization state of the light wave. They are generated based on the polarization image through the Mueller matrix transformation and are used to quantify the polarization modulation characteristics. The light intensity distribution data refers to a multispectral light intensity data set that integrates the Stokes parameters. Each pixel contains the intensity values ​​of the three polarization dimensions S0, S1, and S2, which are used to characterize the optical polarization properties and energy distribution of each point in the scene.

[0025] In an embodiment of the present invention, polarization analysis is performed on a road image, and a polarization camera is used to capture intensity images in four polarization directions: 0°, 45°, 90°, and 135°. Stokes parameters (S0, S1, S2) are generated through Mueller matrix calculation, where S0 represents the total light intensity, S1 represents the difference in polarization intensity between 0° and 90°, and S2 represents the difference in polarization intensity between 45° and 135°. Finally, light intensity distribution data of the fused polarization state is output.

[0026] Step 103: Calculating the polarization-spectral coupling coefficient of the weather medium scattering noise based on the Stokes parameters and the wavelength of the multi-spectral channel; In an embodiment of the present invention, the difference in light intensity component distribution between a high scattering-sensitive band (e.g., 400-600 nm) and a low scattering-sensitive band (e.g., 800-1000 nm) is first calculated based on the Stokes parameter S1 component. This difference is then divided by the wavelength interval to obtain a discrete differential value. An uncorrected coupling value is generated by combining the negative logarithmic transformation value of the atmospheric transmittance. Finally, a piecewise linear correction is performed on the uncorrected coupling value using the real-time atmospheric density parameter to obtain the polarization spectral coupling coefficient.

[0027] Step 104: Compensating the original road image according to the polarization-spectrum coupling coefficient and the light intensity distribution data to generate a polarization-spectrum joint compensation image, and smoothing and enhancing the polarization-spectrum joint compensation image to generate a multi-level feature map. In an embodiment of the present invention, a polarization modulation factor matrix is ​​constructed using the polarization-spectral coupling coefficient, and is multiplied pixel by pixel with the light intensity distribution data to generate polarization-enhanced light intensity data. An attenuation compensation coefficient is calculated based on the polarization-enhanced light intensity data, the light intensity ratio of the original road image, and the real-time visibility. Weighted compensation is performed on the original road image to obtain a polarization-spectrum joint compensation image. Finally, a basic feature map is extracted through Gaussian smoothing, and a Laplace edge detection operator is used to enhance the texture feature map. The two are then channel-jointed to generate a multi-level feature map.

[0028] Step 105: performing noise suppression and feature propagation operations on the multi-level feature map to reconstruct the outline of the object obscured by the weather medium and obtain the reconstructed object spatial distribution; In an embodiment of the present invention, the statistical correlation value of each channel of the multi-level feature map is calculated, and a channel activation identifier is generated by comparing it with a preset threshold; a channel weight vector is determined based on the identifier to perform feature fusion on the multi-level feature map and the channel weight vector to generate a noise suppression feature map; multiple rounds of iterative diffusion are performed on the noise suppression feature map, and a diffusion-stabilized feature map is obtained after the iteration is terminated. The second-order gradient tensor is calculated on the map to determine the distribution of curvature extreme points, and the curvature extreme points of adjacent pixels are connected based on the distribution to generate a reconstructed object space distribution of the obscured object.

[0029] Step 106: Generate obstacle avoidance control instructions for the autonomous driving vehicle based on the reconstructed object spatial distribution and road boundary geometric features; In this step, the obstacle avoidance control command refers to a set of operating instructions that can be parsed by the vehicle actuator, including the steering wheel angle control value (calculated based on the trajectory curvature radius) and the speed constraint value (calculated based on the maximum trajectory curvature and the centripetal acceleration threshold), which is used to achieve real-time dynamic obstacle avoidance.

[0030] In an embodiment of the present invention, the position vectors corresponding to each discrete point on the obstacle surface relative to the reference coordinate system are extracted based on the origin of the reference coordinate system of the spatial distribution of the reconstructed object; semantic segmentation and topological optimization of the road boundary are performed based on the polarization spectrum joint compensation image to generate boundary geometric features; after fitting the boundary constraint equation according to the boundary geometric features, each discrete point is orthogonally projected into the reference coordinate system to calculate the minimum Euclidean distance vector between each obstacle projection point and the origin of the vehicle coordinate system, and the obstacle avoidance direction offset angle is solved; combined with the boundary constraint equation, a fan-shaped safety area is constructed and a parameterized trajectory cluster is generated, from which the parameterized trajectory with the smallest average distance change rate is selected as the optimal obstacle avoidance trajectory to generate obstacle avoidance control instructions for the autonomous driving vehicle.

[0031] The embodiments of the present invention utilize Stokes parameters to analyze the polarization-spectral coupling characteristics, accurately separating medium scattering noise in heavy rain or fog, and solving the image blur and point cloud distortion problems caused by scattering interference in existing cameras. The dynamic generation of polarization-spectral coupling coefficients replaces the strong dependence of traditional deep learning on labeled data, improving the generalization ability of rare weather scenarios. The spatial distribution of objects is quickly reconstructed through noise suppression feature propagation, and obstacle avoidance instructions are generated in combination with optimal obstacle avoidance trajectory optimization, reducing environmental perception and decision-making delays to a safety response threshold that meets the requirements of autonomous driving.

[0032] The present invention provides a specific embodiment, step 103, calculating the polarization spectral coupling coefficient of the weather medium scattering noise based on the Stokes parameters and the wavelength of the multi-spectral channel, specifically includes the following steps: Step 301: Calculating the azimuth of the polarization ellipse based on the Stokes parameters to determine the distribution of light intensity components in each polarization direction; In this step, the azimuth of the polarization ellipse refers to the angle between the vibration direction of the polarized light field vector and the reference axis. It is calculated using arctan(S2 / S1) and is used to determine the main polarization direction. The intensity component distribution refers to the set of light intensity values ​​decomposed into the four polarization directions of horizontal, vertical, 45°, and 135°. It is calculated based on the Stokes parameters using formulas such as the horizontal component.

[0033] In an embodiment of the present invention, the polarization ellipse azimuth θ is calculated by analyzing the ratio of the Stokes parameter S1 component to the S2 component (θ=0.5×arctan(S2 / S1)). According to the polarization ellipse azimuth, the incident light is decomposed into four orthogonal polarization directions: horizontal, vertical, 45°, and 135°. The light intensity component value in each direction is calculated respectively. Specifically, the light intensity component value in the horizontal direction = S0+S1, the light intensity component value in the vertical direction = S0-S1, the light intensity component value in the 45° direction = S0+S2, and the light intensity component value in the 135° direction = S0-S2. Finally, a light intensity component distribution containing four-directional polarization intensities is generated.

[0034] Step 302: dividing the wavelengths of the multi-spectral channel into a high scattering sensitivity band and a low scattering sensitivity band according to a scattering sensitivity threshold; In this step, the scattering sensitivity threshold refers to a critical wavelength value (600 nm) for dividing the scattering intensity of the spectral band, which is set based on the inverse fourth power relationship between scattering intensity and wavelength in the Rayleigh scattering law. The high scattering sensitivity band refers to the blue-green light band (400-600 nm) with a wavelength less than the scattering sensitivity threshold, and its scattering intensity increases by the fourth power as the wavelength decreases. The low scattering sensitivity band refers to the near-infrared band (700-1000 nm) with a wavelength greater than the scattering sensitivity threshold, and its scattering intensity is significantly lower than that of the visible light band.

[0035] In the embodiment of the present application, the preset scattering sensitivity threshold is 600 nm, the blue-green light channel (400-600 nm) with a wavelength less than 600 nm is divided into a high scattering sensitivity band, and the infrared channel (700-1000 nm) with a wavelength greater than 600 nm is divided into a low scattering sensitivity band.

[0036] Step 303: calculating the light intensity component distribution difference amount corresponding to the high scattering sensitivity band and the low scattering sensitivity band respectively, and calculating the discrete differential value of the light intensity component distribution difference amount changing with the wavelength of the multi-spectral channel; In this step, the light intensity component distribution difference amount refers to the absolute difference value of the average value of the polarized light intensity between the high scattering sensitivity band and the low scattering sensitivity band, reflecting the differential influence of scattering on polarized light. The discrete differential value refers to the discrete approximate derivative of the light intensity component distribution difference amount changing with the wavelength.

[0037] In the embodiment of the present application, the difference between the average intensity of the horizontal polarization component in the high scattering sensitivity band and the average intensity of the horizontal polarization component in the low scattering sensitivity band is calculated respectively as the light intensity component distribution difference amount ΔI; then the difference amount is divided by the center wavelength interval Δλ (such as Δλ=50 nm) of the adjacent spectral channel to generate the discrete differential value δ=ΔI / Δλ, to quantify the rate of change of the scattering intensity with the wavelength.

[0038] Step 304: performing a negative logarithmic conversion operation on the atmospheric transmittance parameter to generate an atmospheric optical action amount, and combining the discrete differential value to generate an uncorrected coupling amount; In this step, the atmospheric optical action amount refers to the physical quantity after the negative natural logarithmic conversion of the atmospheric transmittance, which is equivalent to the atmospheric optical thickness. The uncorrected coupling amount refers to the product of the discrete differential value and the atmospheric optical action amount, which represents the scattering noise intensity without considering the density change.

[0039] In an embodiment of the present invention, the atmospheric transmittance τ collected by the meteorological sensor is obtained, and a negative natural logarithm transformation is performed on it to obtain the atmospheric optical action δ, that is, δ=-ln(τ); the discrete differential value is multiplied by the atmospheric optical action to generate an uncorrected coupling amount.

[0040] Step 305: performing linear correction on the uncorrected coupling value according to the atmospheric density parameter to generate a corrected coupling value, and using the corrected coupling value as the polarization spectrum coupling coefficient of the weather medium scattering noise; In this step, the corrected coupling parameter, defined as the coupling parameter after piecewise linear correction using the atmospheric density parameter, is output as the final polarization-spectral coupling coefficient. Weather medium scattering noise refers to the non-uniform scattering effect of rain and fog particles on incident light, which distorts the sensor's received signal. The polarization-spectral coupling coefficient is a physical parameter that quantifies the intensity of scattering noise, integrating polarization, spectral, and meteorological characteristics.

[0041] In an embodiment of the present invention, the atmospheric density value is obtained in real time, the density ratio thereof to the atmospheric density reference value is calculated, and the density ratio is subjected to boundary truncation processing to obtain a truncated density ratio; when the truncated density ratio is less than or equal to a preset critical value, a first linear function is used to generate a density correction factor; when the truncated density ratio is greater than the preset critical value, a second linear function is used to generate a density correction factor; the uncorrected coupling amount and the density correction factor are multiplied to obtain a corrected coupling amount, which is used as the polarization spectrum coupling coefficient.

[0042] The embodiment of the present invention analyzes the directional characteristics of rain and fog particle scattering through the polarization ellipse azimuth angle, combined with dual-band spectral difference analysis, to solve the problem that traditional methods cannot separate multi-wavelength scattering noise; introduces the logarithmic transformation of atmospheric transmittance and real-time density correction, so that the polarization spectrum coupling coefficient automatically adapts to changes in rain and fog concentration, overcoming the defect of fixed parameter models that are inaccurate under sudden weather changes; based on the band division and discrete differential calculation values ​​of the Rayleigh scattering law, verifiable physical-level noise quantization is achieved, eliminating the dependence of deep learning on labeled data.

[0043] The present invention provides a specific embodiment, step 305, performing a linear correction on the uncorrected coupling value according to the atmospheric density parameter to generate a corrected coupling value, and using the corrected coupling value as the polarization spectrum coupling coefficient of the weather medium scattering noise, specifically comprising the following steps: Step 311: Acquire a real-time atmospheric density value and call an atmospheric density reference value from a pre-stored meteorological parameter database; In this step, the atmospheric density reference value refers to the standard sea level atmospheric density value of 1.225 kg / m³, which is stored in the meteorological parameter database and serves as the reference value for density correction.

[0044] In the embodiment of the present application, the atmospheric density value (unit: kg / m³) of the environment where the vehicle is located is collected in real time by the vehicle-mounted atmospheric density sensor; and the standard sea level atmospheric density reference value (which is a fixed value of 1.225 kg / m³) is called from the weather parameter database pre-installed in the automatic driving system.

[0045] Step 312: calculating the density ratio of the real-time atmospheric density value and the atmospheric density reference value, and performing boundary truncation processing on the density ratio to obtain the truncated density ratio; In this step, the density ratio parameter refers to the quotient value of the real-time atmospheric density value and the atmospheric density reference value, reflecting the degree of deviation of the current atmospheric density from the standard state. The truncated density ratio parameter refers to the density ratio after boundary truncation processing, and the value range is limited to the interval [0.6, 2.5], corresponding to the atmospheric density variation limit of-2000 meters to 4000 meters in altitude.

[0046] In the embodiment of the present application, the density ratio = real-time atmospheric density value ÷ atmospheric density reference value; when the density ratio is less than the preset lower threshold value 0.6, it is forcibly set to 0.6, and when it is greater than the preset upper threshold value 2.5, it is forcibly set to 2.5, and finally the truncated density ratio parameter is output to ensure that the parameter is within the physically reasonable range.

[0047] Step 313: inputting the truncated density ratio into a double-interval linear function to obtain a density correction factor, wherein the double-interval linear function includes a first linear function and a second linear function, the first linear function is used when the density ratio is less than or equal to a preset threshold value, and the second linear function is used when the density ratio is greater than the preset threshold value; In this step, the double-interval linear function refers to a segmented linear function with a threshold value 1.0 as the dividing point, including a first linear function and a second linear function. The first linear function refers to a linear function used in normal pressure or high pressure environment, which is in the form of: density correction factor = density ratio × weight A + bias A. The second linear function refers to a linear function used in low pressure environment, which is in the form of: density correction factor = density ratio × weight B + bias B. The preset threshold value refers to the dividing point 1.0 of the double-interval linear function, corresponding to the standard sea level atmospheric density state.

[0048] In the embodiment of the present application, the preset threshold value is 1.0 (corresponding to the sea level reference): if the truncated density ratio ≤ 1.0, the first linear function is used to calculate the density correction factor, i.e. density correction factor = truncated density ratio × first weight coefficient + first bias; if > 1.0, the second linear function is used, i.e. density correction factor = truncated density ratio × second weight coefficient + second bias.

[0049] Step 314: performing a scalar multiplication operation on the density correction factor and the uncorrected coupling quantity to generate a corrected coupling quantity, and taking the corrected coupling quantity as a polarization spectrum coupling coefficient of weather medium scattering noise; In the embodiment of the present application, density correction factor * uncorrected coupling quantity = corrected coupling quantity, and the corrected coupling quantity is taken as the polarization spectrum coupling coefficient of weather medium scattering noise, so as to facilitate subsequent compensation processing.

[0050] The embodiment of the present application solves the failure problem of the fixed compensation coefficient in the highland / basin and other pressure mutation areas; the boundary truncation processing avoids numerical overflow under extreme weather (such as typhoon / high altitude), and ensures the robustness of the system under complex weather conditions; the double-interval function matches the nonlinear relationship between atmospheric density and scattering coefficient, so that the noise quantization is more in line with the physical law.

[0051] The present application provides a specific embodiment, step 104, according to the polarization spectrum coupling coefficient and the light intensity distribution data, the original image of the road is compensated to generate a polarization spectrum joint compensation image, and the polarization spectrum joint compensation image is smoothed and strengthened to generate a multi-level feature map, which specifically includes the following steps: Step 401: constructing a polarization modulation factor matrix based on the polarization spectrum coupling coefficient and the Stokes parameter; In this step, the polarization modulation factor matrix refers to a matrix composed of the product of the polarization spectrum coupling coefficient and the S1 component of the Stokes parameter, which is used to enhance the signal strength of the polarization sensitive area.

[0052] In the embodiment of the present application, the polarization spectrum coupling coefficient is multiplied by the S1 component in the Stokes parameter to generate a polarization modulation factor matrix, which has the same size as the light intensity distribution data, and each element value is the product of the polarization spectrum coupling coefficient and the S1 value of the corresponding pixel point, which is used to enhance the signal strength of the polarization sensitive area.

[0053] Step 402: performing a point multiplication operation on the light intensity distribution data using the polarization modulation factor matrix to generate polarization enhanced light intensity data; In this step, the polarization enhanced light intensity data refers to the output of the light intensity distribution data after the point multiplication operation of the polarization modulation factor matrix, which reflects the scene light intensity distribution after the polarization characteristics are enhanced.

[0054] In the embodiment of the present application, the polarization modulation factor matrix and the S0 component (total light intensity) in the light intensity distribution data are multiplied pixel by pixel, that is, the output value of each pixel point is equal to the product of the S0 value of the point and the polarization modulation factor matrix, to generate an enhanced light intensity data set that fuses the polarization characteristics.

[0055] Step 403: calculate the light intensity ratio of the polarization enhanced light intensity data and the road original image, to generate a spectral attenuation proportion parameter according to the light intensity ratio, and generate an attenuation compensation coefficient corresponding to each spectral channel in combination with real-time visibility; In this step, the light intensity ratio refers to the quotient value of the polarization enhanced light intensity data and the pixel value corresponding to the road original image, which represents the degree of medium attenuation. The spectral attenuation proportion parameter refers to the physical quantity obtained by converting the light intensity ratio by a negative natural logarithm, which is equivalent to the attenuation coefficient in the spectral dimension. The attenuation compensation coefficient refers to the exponential operation result of fusing the spectral attenuation proportion parameter and the visibility offset, which is used to reversely compensate the light intensity loss of the original image.

[0056] In the embodiment of the present application, the following operations are performed for each spectral channel: the light intensity ratio of the polarization enhanced light intensity data and the original image is calculated; the logarithmic domain conversion operation is performed on the light intensity ratio to generate a logarithmic attenuation parameter; the real-time visibility value is obtained and the visibility offset thereof from the reference visibility is calculated; the logarithmic domain conversion operation and the visibility offset are linearly weighted according to the preset weight to generate an unnormalized compensation parameter; the exponential domain inverse conversion operation is performed on the unnormalized parameter to generate a spectral attenuation proportion parameter, and the attenuation compensation coefficient of each channel is calculated in combination with the real-time visibility.

[0057] Step 404: compensate the road original image according to the attenuation compensation coefficient to generate a polarization spectrum joint compensation image; In this step, the polarization spectrum joint compensation image refers to the output of the road original image after being corrected pixel by pixel by the attenuation compensation coefficient, which simultaneously overcomes the polarization distortion and spectral attenuation.

[0058] In the embodiment of the present application, each pixel value of the road original image is multiplied by the attenuation compensation coefficient of the corresponding spectral channel to perform pixel-level compensation, and the compensation formula is: compensated pixel value = original pixel value x attenuation compensation coefficient, and finally a polarization spectrum joint compensation image fused with polarization and spectral compensation is output.

[0059] Step 405: perform smoothing processing on the polarization spectrum joint compensation image to generate a basic feature map, and perform texture enhancement processing on the basic feature map to generate a texture feature map; In this step, the basic feature map refers to the low-pass filtering result of the polarization spectrum joint compensation image after Gaussian smoothing processing, which retains the main structure information of the image. The texture feature map refers to the high-frequency feature map generated by convolution of the Laplacian operator on the basic feature map, which enhances the edge and detail texture.

[0060] In the embodiment of the present application, a Gaussian kernel function (size 15x15) is used to perform convolution smoothing operation on the compensation image to generate a basic feature map; and then a Laplacian edge detection operator (3x3 kernel) convolution operation is performed on the basic feature map to enhance the high-frequency texture information and generate a texture feature map.

[0061] Step 406: performing channel dimension splicing on the basic feature map and the texture feature map to generate a multi-level feature map; In this step, the multi-level feature map refers to the fusion feature of the basic feature map and the texture feature map spliced ​​along the channel dimension, which contains both macro structure and micro detail information.

[0062] In an embodiment of the present invention, the basic feature map and the texture feature map are spliced ​​along the channel axis. If the size of the basic feature map is H×W×C1 and the texture feature map is H×W×C2, the size of the output multi-level feature map is H×W×(C1+C2).

[0063] The embodiment of the present invention restores the original texture of the target object in heavy rain with low visibility through joint compensation of polarization and spectrum; Gaussian smoothing and Laplace sharpening synergistically extract the macro structure and micro edges of the image, solving the problem of detail annihilation in rain and fog caused by traditional single-scale processing; channel stitching replaces the feature pyramid to reduce computing resource consumption and meet the real-time requirements of autonomous driving.

[0064] The present invention provides a specific embodiment, step 403, calculating the light intensity ratio of the polarization-enhanced light intensity data and the original road image, generating a spectral attenuation ratio parameter based on the light intensity ratio, and combining it with real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel, specifically including the following steps: Step 411: Calculating the light intensity ratio of the polarization-enhanced light intensity data and the original road image, and performing a logarithmic domain conversion operation on the light intensity ratio to generate a logarithmic attenuation parameter; In this step, the logarithmic attenuation parameter refers to the physical quantity after the light intensity ratio is converted by the negative natural logarithm, which is equivalent to the optical thickness of the atmosphere.

[0065] In this embodiment of the present invention, the ratio of the polarization-enhanced light intensity data to the light intensity at the corresponding position in the original road image is calculated for each pixel. A negative natural logarithm operation is then performed on this ratio to generate a logarithmic attenuation parameter, calculated using the formula: logarithmic attenuation parameter = -ln(light intensity ratio). This parameter is equivalent to the optical thickness of the atmosphere and reflects the degree of attenuation of light radiation by the medium.

[0066] Step 412: Acquire real-time visibility and call pre-stored benchmark visibility; In this step, the benchmark visibility refers to the visibility reference value under standard clear weather (1000 meters), which is stored in the system configuration file and used as a benchmark for meteorological condition comparison.

[0067] In an embodiment of the present invention, the current visibility value (unit: meter) is collected in real time by the vehicle-mounted meteorological sensor; at the same time, the benchmark visibility threshold (fixed value 1000 meters) is called from a pre-stored configuration file, and the threshold is set based on the international standard clear weather visibility.

[0068] Step 413: Calculating the visibility offset between the real-time visibility and the reference visibility; In this step, the visibility offset refers to the absolute value of the difference between the real-time visibility and the reference visibility, which is used to quantify the severity of the weather.

[0069] In the embodiment of the present invention, the visibility offset=|reference visibility-real-time visibility|).

[0070] Step 414: linearly weighting the logarithmic attenuation parameter and the visibility offset to generate an unnormalized compensation parameter; In the embodiment of the present invention, the logarithmic attenuation parameter is multiplied by a first weight coefficient (0.7), and the visibility offset is multiplied by a second weight coefficient (0.3). The two products are then added together to generate an unnormalized compensation parameter, i.e., the unnormalized compensation parameter = logarithmic attenuation parameter × 0.7 + visibility offset × 0.3.

[0071] Step 415: performing an exponential domain inverse transform operation on the unnormalized compensation parameter to generate a spectral attenuation ratio parameter; In this step, the unnormalized compensation parameter refers to the intermediate value of the linear superposition of the logarithmic attenuation parameter and the visibility offset according to the preset weight, which serves as the input carrier of the exponential inverse transformation.

[0072] In the embodiment of the present invention, an exponential operation with the natural constant e as the base is performed on the unnormalized compensation parameter to generate a spectral attenuation ratio parameter, that is, the spectral attenuation ratio parameter = e unnormalized compensation parameter, which represents the light intensity gain multiple to be compensated.

[0073] Step 416: performing a product operation on the spectral attenuation ratio parameter and the real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel; In the embodiment of the present invention, the spectral attenuation ratio parameter is multiplied by the normalized value of the real-time visibility parameter (i.e., real-time visibility / 1000) to generate the attenuation compensation coefficient of each spectral channel, i.e., attenuation compensation coefficient = spectral attenuation ratio parameter × (real-time visibility / 1000).

[0074] The embodiment of the present invention accurately quantifies the absorption and scattering effects of the medium through logarithmic conversion of the light intensity ratio, overcoming the problem of traditional methods relying on empirical parameters; it integrates the visibility offset to dynamically adjust the compensation intensity, and maintains stable performance in scenarios with sudden visibility changes (such as fog).

[0075] The present invention provides a specific embodiment, step 105, performing noise suppression and feature propagation operations on the multi-level feature map to reconstruct the outline of the object obscured by the weather medium to obtain the reconstructed object spatial distribution, specifically comprising the following steps: Step 501: Calculating a statistical correlation value of each channel based on the channel data of the multi-level feature map, and comparing the statistical correlation value with a preset threshold to generate a channel activation state identifier; In this step, channel data refers to the set of all pixel values ​​of a single channel in the multi-level feature map, with a size of height × width, reflecting the feature response of a specific abstract level. The statistical correlation value refers to the Pearson correlation coefficient between the two channel data, which is used to measure the similarity of feature responses between channels. The preset threshold refers to the critical value (such as 0.7) for determining whether a channel is activated. It is set based on the experience of feature correlation, and the channel is considered valid when it exceeds this value. The channel activation state identifier refers to a binary identifier sequence consisting of 0 or 1, where 1 indicates that the channel is strongly correlated with other channels (correlation coefficient ≥ 0.7) and the feature should be retained.

[0076] In an embodiment of the present invention, the pixel value matrix of each channel of the multi-level feature map is extracted, and the statistical correlation value between any two channels is calculated. The calculation formula is: statistical correlation value = covariance ÷ product of the standard deviations of the two channels. Each statistical correlation value is compared with a preset threshold (such as 0.7). If the statistical correlation value is ≥0.7, an activation flag 1 is generated, otherwise 0 is generated, and finally a binary channel activation state flag is generated.

[0077] Step 502: Determine a channel weight vector according to the channel activation state identifier; In this step, the channel weight vector refers to a list of weight coefficients assigned to each channel, which is calculated based on the activation marker ratio and used for feature fusion weighting.

[0078] In an embodiment of the present invention, the number of channel activation state identifiers of 1 for each channel is counted, and its proportion to the total number of channels is calculated; the channel weight of each channel = corresponding proportion × weight scaling factor + basic weight; all channel weight values ​​are combined into a channel weight vector.

[0079] Step 503: Fusing the multi-level feature map with the channel weight vector to generate a noise suppression feature map; In this step, the noise suppression feature map refers to a single-channel image obtained by weighted fusion of multiple levels of feature maps by channel weight vectors, which suppresses irrelevant channel noise.

[0080] In an embodiment of the present invention, channel weighting is performed on each channel of the multi-level feature map. Specifically, the channel data is multiplied by the corresponding channel weight value, and then summed along the channel dimension to generate a fused single-channel noise suppression feature map.

[0081] Step 504: performing multiple rounds of iterative feature propagation operations on the noise suppression feature map, terminating the feature propagation operation when the number of iterative rounds reaches a preset number, and generating a diffusion stabilization feature map; In this step, the preset number of iterations refers to the upper limit of the number of feature propagation iterations (e.g., 8). Experimental verification has proven that this balances reconstruction accuracy and computational efficiency. The diffusion-stable feature map is the feature map that converges after multiple rounds of iterative propagation and contains enhanced object structure information.

[0082] In an embodiment of the present invention, multiple rounds of iterative feature propagation operations are performed on the noise suppression feature map. In each round of iteration, the current noise suppression feature map and the initial noise suppression feature map are weighted and summed according to a preset ratio, and a 5×5 Gaussian convolution is performed on the weighted result to update the current feature map. When the number of iterations reaches a preset number (such as 8 times), the operation is terminated to generate a diffusion stabilization feature map.

[0083] Step 505: Performing a second-order gradient tensor calculation on the diffusion stability feature map to obtain a feature change rate tensor for each pixel to determine a curvature extreme point distribution. Connecting the curvature extreme points of adjacent pixels based on the curvature extreme point distribution to reconstruct the contour of the object obscured by the weather medium and generate a reconstructed object spatial distribution. In this step, the feature change rate tensor refers to the feature Figure 2 The matrix group consisting of the order partial derivatives contains derivative information in the four directions of XX, XY, YX, and YY. The curvature extreme point distribution refers to the spatial location set of pixels with the largest eigenvalue > 0.8 in the characteristic rate of change tensor, representing the turning point of the object contour. The curvature extreme point refers to a single pixel in the curvature extreme point distribution, reflecting the location of the geometric mutation of the contour. Weather media refers to atmospheric particles such as rain and fog that cause light scattering, resulting in the obscuration of the target object. The reconstructed object spatial distribution refers to the three-dimensional point cloud data generated based on the triangulation of the curvature extreme points, describing the complete geometric shape of the obscured object.

[0084] In an embodiment of the present invention, the second-order partial derivatives of the diffusion stability feature map in the X and Y directions are calculated; a Hessian matrix of each pixel point is constructed (whose matrix elements include the XX direction derivative, the XY direction derivative, the YX direction derivative, and the YY direction derivative); the eigenvalues ​​of the Hessian matrix are solved, and the points with the maximum eigenvalue greater than 0.8 are marked as curvature extreme points; Delaunay triangulation is performed on the curvature extreme points in the 8-neighborhood, and the spatial distribution of the reconstructed object is generated based on the triangular mesh.

[0085] The embodiment of the present invention improves the effective feature signal-to-noise ratio in rainstorm scenes through channel correlation weighted fusion; the triangular connection of curvature extreme points restores the topological structure of vehicles obscured by fog.

[0086] The present invention provides a specific embodiment, step 106, generating obstacle avoidance control instructions for the autonomous driving vehicle based on the reconstructed object spatial distribution and road boundary geometric features, specifically comprising the following steps: Step 601: Based on the origin of the reference coordinate system of the reconstructed object space distribution, define the origin of the ego-vehicle coordinate system, and extract the position vector corresponding to each discrete point on the obstacle surface relative to the reference coordinate system; In this step, the origin of the reference coordinate system refers to the zero point of the 3D coordinate system of the reconstructed object space, typically set to the center of mass of the vehicle. The origin of the ego-vehicle coordinate system refers to the zero point (0,0,0) of the vehicle's local coordinate system, defined by the origin of the reference coordinate system. A discrete point refers to a single 3D coordinate point in the obstacle surface point cloud, containing a position vector (x,y,z). The position vector is a 3D space vector pointing from the origin of the ego-vehicle coordinate system to the discrete obstacle point.

[0087] In this embodiment of the present invention, the origin of the three-dimensional coordinate system of the reconstructed object spatial distribution (usually the vehicle's current position) is used as the vehicle coordinate origin; the point cloud data of the obstacle surface is traversed, and the three-dimensional coordinates (x, y, z) of each discrete point relative to the origin are recorded to generate a position vector.

[0088] Step 602: semantically segment the road boundary in the polarization-spectral joint compensation image to generate an initial road boundary mask; In this step, the initial road boundary mask refers to the binary road area image generated by semantic segmentation, and white pixels represent the road area.

[0089] In an embodiment of the present invention, a pre-trained fully convolutional network is used to process the polarization spectrum joint compensation image and output a binary segmentation map of the road area; holes are filled through morphological closing operations to generate an initial road boundary mask, in which the non-road area is 0 and the road area is 1.

[0090] Step 603: Topologically optimize the initial road boundary mask to generate road boundary geometric features, and fit the boundary constraint equation of the drivable area based on the road boundary geometric features; orthogonally project each discrete point onto the reference coordinate system based on the position vector to generate an obstacle projection point set; In this step, the road boundary geometry refers to the parameterized curve of the road boundary after topology optimization, defined by the coordinates of the control points. The drivable area refers to the safe passage area enclosed by the road boundary geometry. The boundary constraint equation refers to the mathematical equation describing the road boundary curve. The reference coordinate system refers to the right-handed three-dimensional rectangular coordinate system established with the ego-vehicle coordinate origin. The obstacle projection point set refers to the two-dimensional coordinate set of the discrete obstacle points after orthogonal projection onto the XY plane.

[0091] In the embodiment of the present application, a skeleton extraction operation is performed on the initial mask to obtain a single-pixel wide boundary line; isolated line segments with a length less than 10 pixels are removed to generate an optimized boundary point set, which is fitted with a cubic Bezier curve to generate a road boundary geometric feature; a boundary constraint equation, such as a straight line equation ax+by+c=0 or a curve parameter equation, is solved based on the curve control point. The Z coordinate of the position vector of each obstacle is set to zero (ignoring the height dimension), and is projected onto the XY plane of the reference coordinate system to generate a two-dimensional obstacle projection point set (the formula is projection point=(x,y,0)).

[0092] Step 604: Calculate the minimum Euclidean distance vector of the obstacle projection point set from the origin of the ego vehicle coordinate system to solve the obstacle avoidance direction offset angle, construct a sector safety area in combination with the boundary constraint equation, and generate a parameterized trajectory cluster in the sector safety area; In this step, the minimum Euclidean distance vector refers to the two-dimensional vector in the obstacle projection point set that is closest to the origin of the ego vehicle. The obstacle avoidance direction offset angle refers to the angle between the minimum Euclidean distance vector and the vehicle forward direction (the positive direction of the Y axis). The sector safety area refers to a sector plane area with the obstacle avoidance direction offset angle as the central axis and the road boundary as the constraint. The parameterized trajectory curve cluster refers to a set of cubic Bezier curves described by parameter equations generated in the sector area.

[0093] In the embodiment of the present application, the minimum Euclidean distance vector of the obstacle projection point set from the origin (0,0) of the ego vehicle coordinate system is calculated, and the angle between the minimum Euclidean distance vector and the vehicle forward direction (the Y axis) is the obstacle avoidance direction offset angle θ. A sector safety area is constructed with θ as the central axis and the opening angle φ of the road boundary equation as the opening. Seven cubic Bezier curves are uniformly generated in the sector safety area as a parameterized trajectory curve cluster.

[0094] Step 605: According to the parameterized trajectory cluster, calculate the average distance change rate of the trajectory point set to the obstacle projection point set, and select the parameterized trajectory with the minimum average distance change rate as the optimal obstacle avoidance trajectory; In this step, the trajectory point set refers to a sequence of discrete point coordinates sampled at equal arc lengths on a single trajectory curve. The average distance change rate refers to the average value of the absolute value of the distance change rate along the arc length of the trajectory point set to the obstacle point set. The parameterized trajectory curve refers to a curve defined by a control point parameter equation. The optimal obstacle avoidance trajectory refers to the parameterized trajectory curve with the minimum average distance change rate.

[0095] In the embodiment of the present application, 50 points are sampled for each trajectory to generate a trajectory point set; the distance D of each trajectory point to the nearest obstacle is calculated; the average value of the absolute value of the distance change rate dD / ds along the trajectory arc length s is calculated; and the trajectory with the minimum average value is selected as the optimal obstacle avoidance trajectory.

[0096] Step 606: Generate obstacle avoidance control instructions for the autonomous driving vehicle based on the optimal obstacle avoidance trajectory; In an embodiment of the present invention, the extreme value of the curvature radius of the optimal obstacle avoidance trajectory is calculated; the steering wheel deflection angle is calculated based on the vehicle wheelbase, that is, the steering wheel deflection angle = arctan(vehicle wheelbase ÷ extreme value of the curvature radius); the speed constraint is calculated based on the trajectory length and maximum curvature, and the speed constraint = √(trajectory length ÷ maximum curvature); and an instruction group of the steering wheel deflection angle and the speed constraint is output as an obstacle avoidance control instruction for the autonomous driving vehicle.

[0097] The embodiment of the present invention uses the average distance change rate index to screen out the trajectory with the gentlest change in distance to obstacles, thereby reducing the accident rate; and integrates the fan-shaped area generation with road boundary constraints to improve the success rate of sharp curve scenarios.

[0098] Figure 2 The present invention provides a schematic diagram of the structure of a large model environment perception and decision-making system in autonomous driving, such as Figure 2 As shown, the system includes: An acquisition module 21 is used to acquire original road images of multispectral channels; A first generating module 22 is configured to generate light intensity distribution data carrying Stokes parameters based on the polarization state distribution of the original road image; A calculation module 23 is used to calculate the polarization spectrum coupling coefficient of the weather medium scattering noise based on the Stokes parameters and the wavelength of the multi-spectral channel; The compensation module 24 is used to perform compensation processing on the original road image according to the polarization spectrum coupling coefficient and the light intensity distribution data to generate a polarization spectrum joint compensation image, and smooth and enhance the polarization spectrum joint compensation image to generate a multi-level feature map.

[0099] A processing module 25 is configured to perform noise suppression and feature propagation operations on the multi-level feature maps to reconstruct the contours of objects obscured by weather media and obtain a reconstructed spatial distribution of the objects; The second generation module 26 is used to generate obstacle avoidance control instructions for the autonomous driving vehicle based on the reconstructed object spatial distribution and road boundary geometric characteristics.

[0100] Figure 2 The large model described above can be used in the environment perception and decision-making system of autonomous driving. Figure 1 The implementation principles and technical effects of the large-scale model-based environmental perception and decision-making method for autonomous driving described in the illustrated embodiment are not further elaborated. The specific manner in which each module and unit performs operations in the large-scale model-based environmental perception and decision-making system for autonomous driving described in the aforementioned embodiment has been described in detail in the relevant embodiments of the method and will not be further elaborated here.

[0101] In one possible design, Figure 2 A large model of the embodiment shown in the environment perception and decision-making system in autonomous driving can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32; The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0102] The processing component 32 is used for the above Figure 1 The embodiment provides a large-model environment perception and decision-making method in autonomous driving.

[0103] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0104] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0105] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0106] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0107] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0108] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0109] The embodiment of the present invention further provides a computer storage medium storing a computer program, which can achieve the above-mentioned Figure 1 The illustrated embodiment is a large-model environment perception and decision-making method in autonomous driving.

[0110] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0111] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0112] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A large-scale model environment perception and decision-making method for autonomous driving, characterized by: include: Obtain original road images with multispectral channels; generating light intensity distribution data carrying Stokes parameters based on the polarization state distribution of the original road image; Calculating the polarization spectral coupling coefficient of the weather medium scattering noise based on the Stokes parameters and the wavelength of the multi-spectral channel; performing compensation processing on the original road image according to the polarization-spectrum coupling coefficient and the light intensity distribution data to generate a polarization-spectrum joint compensation image, and performing smoothing and enhancement processing on the polarization-spectrum joint compensation image to generate a multi-level feature map; Performing noise suppression and feature propagation operations on the multi-level feature maps to reconstruct the contours of objects obscured by weather media and obtain a reconstructed spatial distribution of the objects; Obstacle avoidance control instructions for the autonomous driving vehicle are generated based on the reconstructed spatial distribution of objects and the geometric features of the road boundaries.

2. The method according to claim 1, characterized in that Calculating the polarization spectrum coupling coefficient of the weather medium scattering noise based on the Stokes parameters and the wavelength of the multi-spectral channel includes: Based on the Stokes parameters, the polarization ellipse azimuth is calculated to determine the light intensity component distribution in each polarization direction; Dividing the wavelength of the multispectral channel into a high scattering sensitivity band and a low scattering sensitivity band according to a scattering sensitivity threshold; Calculating the difference in light intensity component distribution corresponding to the high scattering sensitive band and the low scattering sensitive band, and calculating the discrete differential value of the light intensity component distribution difference as the wavelength of the multispectral channel changes; Performing a negative logarithmic transformation operation on the atmospheric transmittance parameter to generate an atmospheric optical action quantity, and combining the discrete differential value to generate an uncorrected coupling quantity; The uncorrected coupling amount is linearly corrected according to the atmospheric density parameter to generate a corrected coupling amount, and the corrected coupling amount is used as the polarization spectrum coupling coefficient of the weather medium scattering noise.

3. The method according to claim 2, characterized in that The uncorrected coupling amount is linearly corrected according to the atmospheric density parameter to generate a corrected coupling amount, and the corrected coupling amount is used as the polarization spectrum coupling coefficient of the weather medium scattering noise, including: Obtain real-time atmospheric density values ​​and call atmospheric density reference values ​​from a pre-stored meteorological parameter database; Calculating a density ratio of the real-time atmospheric density value to the atmospheric density reference value, and performing boundary truncation processing on the density ratio to obtain a truncated density ratio; Inputting the truncated density ratio into a dual-interval linear function to obtain a density correction factor, wherein the dual-interval linear function includes a first linear function and a second linear function, the first linear function is used when the density ratio is less than or equal to a preset critical value, and the second linear function is used when the density ratio is greater than the preset critical value; A scalar multiplication operation is performed on the density correction factor and the uncorrected coupling amount to generate a corrected coupling amount, and the corrected coupling amount is used as the polarization spectrum coupling coefficient of the weather medium scattering noise.

4. The method according to claim 1, wherein The method further comprises: performing compensation processing on the original road image according to the polarization-spectrum coupling coefficient and the light intensity distribution data to generate a polarization-spectrum joint compensation image; and performing smoothing and enhancement processing on the polarization-spectrum joint compensation image to generate a multi-level feature map, including: constructing a polarization modulation factor matrix based on the polarization spectral coupling coefficient and the Stokes parameter; Performing a dot multiplication operation on the light intensity distribution data using the polarization modulation factor matrix to generate polarization-enhanced light intensity data; Calculating the light intensity ratio of the polarization-enhanced light intensity data and the original road image to generate a spectral attenuation ratio parameter based on the light intensity ratio, and generating an attenuation compensation coefficient corresponding to each spectral channel in combination with real-time visibility; performing compensation processing on the original road image according to the attenuation compensation coefficient to generate a polarization-spectrum joint compensation image; performing smoothing processing on the polarization spectrum joint compensation image to generate a basic feature map, and performing texture enhancement processing on the basic feature map to generate a texture feature map; The basic feature map and the texture feature map are spliced ​​in channel dimension to generate a multi-level feature map.

5. The method according to claim 4, characterized in that Calculating the light intensity ratio of the polarization-enhanced light intensity data and the original road image to generate a spectral attenuation ratio parameter based on the light intensity ratio, and generating an attenuation compensation coefficient corresponding to each spectral channel in combination with real-time visibility, including: Calculating the light intensity ratio of the polarization-enhanced light intensity data to the original road image, and performing a logarithmic domain conversion operation on the light intensity ratio to generate a logarithmic attenuation parameter; Obtain real-time visibility and call pre-stored benchmark visibility; Calculating a visibility offset between the real-time visibility and the reference visibility; linearly weighting the logarithmic attenuation parameter and the visibility offset to generate an unnormalized compensation parameter; performing an exponential domain inverse transform operation on the unnormalized compensation parameter to generate a spectral attenuation ratio parameter; The spectral attenuation ratio parameter is multiplied by the real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel.

6. The method according to claim 1, characterized in that Noise suppression and feature propagation operations are performed on the multi-level feature maps to reconstruct the outline of the object obscured by the weather medium to obtain the reconstructed object spatial distribution, including: Calculating a statistical correlation value of each channel based on the channel data of the multi-level feature map, and comparing the statistical correlation value with a preset threshold to generate a channel activation state identifier; Determining a channel weight vector according to the channel activation state identifier; Fusing the multi-level feature map with the channel weight vector to generate a noise suppression feature map; Performing multiple rounds of iterative feature propagation operations on the noise suppression feature map, terminating the feature propagation operation when the number of iterative rounds reaches a preset number, and generating a diffusion stabilization feature map; A second-order gradient tensor calculation is performed on the diffusion stability feature map to obtain the feature change rate tensor of each pixel point to determine the distribution of curvature extreme value points. The curvature extreme value points of adjacent pixels are connected according to the curvature extreme value point distribution to reconstruct the outline of the object obscured by the weather medium and generate the reconstructed object spatial distribution.

7. The method according to claim 1, characterized in that Generate obstacle avoidance control instructions for the autonomous driving vehicle based on the reconstructed object spatial distribution and road boundary geometric features, including: Based on the origin of the reference coordinate system of the reconstructed object space distribution, the origin of the ego-vehicle coordinates is defined, and the position vectors corresponding to each discrete point on the obstacle surface relative to the reference coordinate system are extracted; Perform semantic segmentation on the road boundary in the polarization-spectral joint compensation image to generate an initial road boundary mask; Performing topological optimization on the initial road boundary mask to generate road boundary geometric features, so as to fit the boundary constraint equation of the drivable area according to the road boundary geometric features; orthogonally projecting each discrete point onto the reference coordinate system according to the position vector to generate an obstacle projection point set; Calculating the minimum Euclidean distance vector between the obstacle projection point set and the vehicle coordinate origin to solve the obstacle avoidance direction offset angle, combining the boundary constraint equation to construct a fan-shaped safety area, and generating a parameterized trajectory cluster within the fan-shaped safety area; Calculating the average value of the distance change rate from the trajectory point set to the obstacle projection point set based on the parameterized trajectory cluster, and taking the parameterized trajectory with the smallest average distance change rate as the optimal obstacle avoidance trajectory; Generate obstacle avoidance control instructions for the autonomous driving vehicle based on the optimal obstacle avoidance trajectory.

8. A large-scale model environment perception and decision-making system for autonomous driving, characterized by: include: An acquisition module is used to obtain original road images with multispectral channels; A first generating module is configured to generate light intensity distribution data carrying Stokes parameters based on the polarization state distribution of the original road image; a calculation module, configured to calculate a polarization spectral coupling coefficient of weather medium scattering noise based on the Stokes parameters and the wavelength of the multispectral channel; a compensation module, configured to perform compensation processing on the original road image according to the polarization-spectrum coupling coefficient and the light intensity distribution data to generate a polarization-spectrum joint compensation image, and to perform smoothing and enhancement processing on the polarization-spectrum joint compensation image to generate a multi-level feature map; a processing module, configured to perform noise suppression and feature propagation operations on the multi-level feature maps to reconstruct the contours of objects obscured by weather media and obtain a reconstructed spatial distribution of the objects; The second generation module is used to generate obstacle avoidance control instructions for the autonomous driving vehicle based on the reconstructed object spatial distribution and road boundary geometric characteristics.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the environmental perception and decision-making method of a large model in autonomous driving as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, the environment perception and decision-making method of a large model in autonomous driving is implemented as described in any one of claims 1 to 7.

Images