An environment perception and decision method and system of a large model in autonomous driving

Abstract

The application provides an environment perception and decision-making method and system of a large model in automatic driving, generates light intensity distribution data carrying Stokes parameters by acquiring a road original image of multiple spectral channels; calculates a polarization spectrum coupling coefficient of weather medium scattering noise based on the Stokes parameters and the wavelengths of the multiple spectral channels, combines the light intensity distribution data, compensates the road original image, generates a polarization spectrum joint compensation image, and performs smoothing and strengthening processing on the polarization spectrum joint compensation image to generate a multi-level feature map; performs noise suppression and feature propagation operations on the multi-level feature map to reconstruct the object contour hidden by the weather medium, obtains a reconstructed object spatial distribution, combines a road boundary geometric feature, and generates an obstacle avoidance control instruction of an automatic driving vehicle; and the technical scheme provided by the application improves the driving safety and autonomous navigation ability under complex weather conditions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to an environment perception and decision-making method and system of a large model in automatic driving. BACKGROUND

[0002] In low-visibility weather conditions such as heavy rain or fog, autonomous vehicles face significant challenges in identifying road boundaries and obstacles. Due to the adverse weather conditions, the quality of information obtained by sensors decreases, such as the images captured by cameras may become blurred, and the laser radar data may also have errors due to the interference of water droplets or fog particles. Therefore, in such an environment, the vehicle needs to rely on a highly accurate perception system to accurately identify the surrounding environment and ensure driving safety. In addition, the autonomous driving system also needs to be able to quickly respond to changing road conditions, including sudden obstacles or sharp turns, which requires not only high-precision perception technology but also real-time processing capability.

[0003] Currently, a widely used method is to use deep learning models to analyze and process multi-sensor fusion data, especially combining visible light and infrared imaging technology to enhance visual perception ability in adverse weather conditions. However, the existing scheme has some significant defects. For example, deep learning models rely on a large amount of labeled data for training, making it difficult to collect data under rare or extreme weather conditions, resulting in poor performance of the model in these scenarios; it is difficult to accurately compensate for optical noise caused by weather, making the final decision result inaccurate, etc. SUMMARY

[0004] The present application provides an environment perception and decision-making method and system of a large model in automatic driving, to solve the problem of poor model performance under rare or extreme weather conditions, difficulty in accurately compensating for optical noise caused by weather, and inaccurate final decision results in the prior art.

[0005] In a first aspect, the present application provides an environment perception and decision-making method of a large model in automatic driving, comprising:

[0006] obtaining a road original image of multiple spectral channels;

[0007] generating light intensity distribution data carrying Stokes parameters based on the polarization state distribution of the road original image;

[0008] calculating the polarization spectrum coupling coefficient of weather medium scattering noise based on the Stokes parameters and the wavelengths of the multiple spectral channels;

[0009] According to the polarization spectrum coupling coefficient and the light intensity distribution data, the road original image is compensated to generate a polarization spectrum joint compensation image, and the polarization spectrum joint compensation image is smoothed and strengthened to generate a multi-level feature map;

[0010] The multi-level feature map is subjected to noise suppression and feature propagation operations to reconstruct the object profile obscured by the weather medium, to obtain a reconstructed object spatial distribution;

[0011] According to the reconstructed object spatial distribution and the road boundary geometric feature, an obstacle avoidance control instruction of the autonomous vehicle is generated.

[0012] Optionally, based on the Stokes parameters and the wavelengths of the multi-spectral channels, a polarization spectrum coupling coefficient of the weather medium scattering noise is calculated, comprising:

[0013] Based on the Stokes parameters, a polarization ellipse azimuth angle is solved to determine the light intensity component distribution of each polarization direction;

[0014] According to a scattering sensitivity threshold, the wavelengths of the multi-spectral channels are divided into a high scattering sensitivity band and a low scattering sensitivity band;

[0015] The light intensity component distribution difference amount corresponding to the high scattering sensitivity band and the low scattering sensitivity band is calculated, and the discrete differential value of the light intensity component distribution difference amount with respect to the wavelength of the multi-spectral channel is calculated;

[0016] A negative logarithmic conversion operation is performed on the atmospheric transmittance parameter to generate an atmospheric optical action amount, and the discrete differential value is combined to generate an uncorrected coupling amount;

[0017] According to the atmospheric density parameter, the uncorrected coupling amount is linearly corrected to generate a corrected coupling amount, and the corrected coupling amount is taken as the polarization spectrum coupling coefficient of the weather medium scattering noise.

[0018] Optionally, according to the atmospheric density parameter, the uncorrected coupling amount is linearly corrected to generate a corrected coupling amount, and the corrected coupling amount is taken as the polarization spectrum coupling coefficient of the weather medium scattering noise, comprising:

[0019] An real-time atmospheric density value is obtained, and an atmospheric density reference value is called from a pre-stored meteorological parameter database;

[0020] The density ratio of the real-time atmospheric density value and the atmospheric density reference value is calculated, and the density ratio is subjected to boundary truncation processing to obtain a truncated density ratio;

[0021] inputting the truncated density ratio into a double-interval linear function to obtain a density correction factor, wherein the double-interval linear function comprises a first linear function and a second linear function, the first linear function is used when the density ratio is less than or equal to a preset threshold value, and the second linear function is used when the density ratio is greater than the preset threshold value;

[0022] performing a scalar multiplication operation on the density correction factor and the uncorrected coupling quantity to generate a corrected coupling quantity, and taking the corrected coupling quantity as a polarization spectrum coupling coefficient of weather medium scattering noise.

[0023] Optionally, according to the polarization spectrum coupling coefficient and the light intensity distribution data, a polarization spectrum joint compensation image is generated by performing compensation processing on the road original image, and a multi-level feature map is generated by performing smoothing and strengthening processing on the polarization spectrum joint compensation image, comprising:

[0024] Based on the polarization spectrum coupling coefficient and the Stokes parameter, a polarization modulation factor matrix is constructed;

[0025] The polarization modulation factor matrix is used to perform point multiplication operation on the light intensity distribution data to generate polarization enhanced light intensity data;

[0026] The light intensity ratio of the polarization enhanced light intensity data and the road original image is calculated to generate a spectral attenuation proportion parameter according to the light intensity ratio, and an attenuation compensation coefficient corresponding to each spectral channel is generated in combination with real-time visibility;

[0027] According to the attenuation compensation coefficient, a polarization spectrum joint compensation image is generated by performing compensation processing on the road original image;

[0028] The polarization spectrum joint compensation image is smoothed to generate a basic feature map, and the basic feature map is texture-enhanced to generate a texture feature map;

[0029] The basic feature map and the texture feature map are spliced in the channel dimension to generate a multi-level feature map.

[0030] Optionally, the light intensity ratio of the polarization enhanced light intensity data and the road original image is calculated to generate a spectral attenuation proportion parameter according to the light intensity ratio, and an attenuation compensation coefficient corresponding to each spectral channel is generated in combination with real-time visibility, comprising:

[0031] The light intensity ratio of the polarization enhanced light intensity data and the road original image is calculated, and a logarithmic attenuation parameter is generated by performing a logarithmic domain conversion operation on the light intensity ratio;

[0032] Real-time visibility is obtained, and a pre-stored reference visibility is called;

[0033] calculating a visibility offset of the real-time visibility and the reference visibility;

[0034] linearly weighting the logarithmic attenuation parameter and the visibility offset to generate an unnormalized compensation parameter;

[0035] performing an exponential domain inverse transform operation on the unnormalized compensation parameter to generate a spectral attenuation ratio parameter;

[0036] performing a product operation on the spectral attenuation ratio parameter and the real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel.

[0037] Optionally, performing noise suppression and feature propagation operations on the multi-level feature map to reconstruct the object profile obscured by the weather medium to obtain a reconstructed object spatial distribution, including:

[0038] calculating a statistical correlation value of each channel based on the channel data of the multi-level feature map, and comparing the statistical correlation value with a preset threshold to generate a channel activation state identifier;

[0039] determining a channel weight vector according to the channel activation state identifier;

[0040] fusing the multi-level feature map and the channel weight vector to generate a noise suppression feature map;

[0041] performing a multi-round iteration feature propagation operation on the noise suppression feature map, and terminating the feature propagation operation when the iteration round reaches a preset number to generate a diffusion stable feature map;

[0042] performing a second-order gradient tensor calculation on the diffusion stable feature map to obtain a feature change rate tensor of each pixel point to determine a curvature extreme point distribution, and connecting curvature extreme points of adjacent pixels according to the curvature extreme point distribution to reconstruct the object profile obscured by the weather medium to generate a reconstructed object spatial distribution.

[0043] Optionally, generating an obstacle avoidance control instruction of an autonomous vehicle according to the reconstructed object spatial distribution and a road boundary geometric feature, including:

[0044] defining a vehicle coordinate origin based on an origin of a reference coordinate system of the reconstructed object spatial distribution, and extracting a position vector corresponding to each discrete point on the obstacle surface relative to the reference coordinate system;

[0045] performing semantic segmentation on a road boundary in the polarimetric spectral joint compensation image to generate an initial road boundary mask;

[0046] topology optimization is performed on the initial road boundary mask to generate road boundary geometric features, and a drivable area boundary constraint equation is fitted according to the road boundary geometric features;

[0047] According to the position vector, each discrete point is orthogonally projected into the reference coordinate system to generate a set of obstacle projection points;

[0048] The minimum Euclidean distance vector between the set of obstacle projection points and the origin of the ego-vehicle coordinate system is calculated to solve an obstacle avoidance direction offset angle, a sector safety region is constructed in combination with the boundary constraint equation, and a parameterized trajectory cluster is generated in the sector safety region;

[0049] According to the parameterized trajectory cluster, the average distance change rate of the trajectory point set to the set of obstacle projection points is calculated, and the parameterized trajectory with the minimum average distance change rate is taken as the optimal obstacle avoidance trajectory;

[0050] According to the optimal obstacle avoidance trajectory, an obstacle avoidance control instruction for the autonomous vehicle is generated.

[0051] In a second aspect, the present application provides an environment perception and decision system for automatic driving of a large model, comprising:

[0052] An acquisition module is configured to acquire a road original image of a multi-spectral channel;

[0053] A first generation module is configured to generate light intensity distribution data carrying Stokes parameters based on the polarization state distribution of the road original image;

[0054] A calculation module is configured to calculate a polarization spectrum coupling coefficient of weather medium scattering noise based on the Stokes parameters and the wavelengths of the multi-spectral channel;

[0055] A compensation module is configured to perform compensation processing on the road original image according to the polarization spectrum coupling coefficient and the light intensity distribution data to generate a polarization spectrum joint compensation image, and to perform smoothing and strengthening processing on the polarization spectrum joint compensation image to generate a multi-level feature map;

[0056] A processing module is configured to perform noise suppression and feature propagation operations on the multi-level feature map to reconstruct an object contour obscured by a weather medium, and obtain a reconstructed object spatial distribution;

[0057] A second generation module is configured to generate an obstacle avoidance control instruction for an autonomous vehicle according to the reconstructed object spatial distribution and road boundary geometric features.

[0058] In a third aspect, the present application provides a computing device, comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the environment perception and decision method of a large model in autonomous driving as described in the first aspect above.

[0059] In a fourth aspect, the present application provides a computer storage medium, storing a computer program, when the computer program is executed by a computer, implementing the environment perception and decision method of a large model in autonomous driving as described in the first aspect.

[0060] In the present application, a road original image of a multi-spectral channel is acquired; based on a polarization state distribution of the road original image, light intensity distribution data carrying Stokes parameters is generated; based on the Stokes parameters and wavelengths of the multi-spectral channel, a polarization spectrum coupling coefficient of weather medium scattering noise is calculated; according to the polarization spectrum coupling coefficient and the light intensity distribution data, the road original image is compensated to generate a polarization spectrum joint compensation image, and the polarization spectrum joint compensation image is smoothed and strengthened to generate a multi-level feature map; the multi-level feature map is subjected to noise suppression and feature propagation operations to reconstruct an object contour obscured by a weather medium, to obtain a reconstructed object spatial distribution; and based on the reconstructed object spatial distribution and road boundary geometric features, an obstacle avoidance control instruction for an autonomous vehicle is generated. The technical scheme provided by the present application breaks through the visible light limit through multi-spectral imaging, captures original information of roads and obstacles obscured by weather media in heavy rain or fog scenes, solves the defect of information loss of existing single-spectral sensors in extreme weather, separates scattered noise from real object reflected light by analyzing the light wave propagation direction characteristics of the polarization state distribution, overcomes the core problem of image blurring caused by rain and fog particle scattering in existing schemes, quantifies the differential scattering influence of weather media on the multi-spectral channel through physical modeling to achieve accurate evaluation of noise intensity, breaks through the limitations of traditional deep learning models that rely on labeled data and cannot model dynamic scattering rules, recovers the texture details of objects obscured by weather based on reverse compensation of scattered noise intensity distortion, and the smoothing and strengthening processing can extract cross-resolution features to improve the robustness of edge and contour recognition in low-visibility environments. The fusion of channel statistical characteristics and geometric topology reconstruction restores the complete three-dimensional structure of the obscured object under strong noise interference, solves the problem of object breakage and deformation distortion caused by existing segmentation methods in rain and fog, and generates an optimal trajectory in combination with the spatial geometric constraints of the reconstructed environment to realize real-time and safe decision-making under complex weather conditions, and makes up for the defects of response delay and trajectory planning failure caused by the perception distortion of existing systems. Further, based on the Stokes parameter, the light intensity distribution of the polarization direction is calculated, the spectral bands are divided according to the scattering sensitivity, the discrete differential relationship of the light intensity difference between the bands is established, the uncorrected coupling parameters are generated by fusing the negative logarithmic conversion quantity of the atmospheric transmittance, and finally the polarization spectrum coupling coefficient is dynamically corrected by the real-time atmospheric density. The scattering noise physical characteristics are quantified by the polarization spectrum coupling, the defects of the existing deep learning model that relies on a large amount of labeled data and cannot explain the root cause of optical interference are overcome, the dependence on the scale and quality of the training data is reduced, the atmospheric density is introduced for real-time correction, the polarization spectrum coupling coefficient automatically adapts to the change of rain and fog concentration, the problem of failure of the traditional fixed parameter compensation method in sudden weather is solved, and the perception stability in complex weather conditions is improved.

[0061] These aspects or other aspects of the present application will be more apparent in the following description of the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0063] Figure 1 A flow chart of an environment perception and decision method of a large model in autonomous driving provided by the present application is shown;

[0064] Figure 2 A structural schematic diagram of an environment perception and decision system of a large model in autonomous driving provided by the present application is shown;

[0065] Figure 3 A structural schematic diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0066] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0067] In some of the descriptions in the specification and claims of the present application and the above-mentioned drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that these operations can be executed or in parallel without the order in which they appear in this text. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and these operations can be executed in sequence or in parallel. It should be noted that the "first", "second", etc. in this text are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence. Also, "first" and "second" are not different types.

[0068] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0069] In low-visibility weather conditions such as rain or fog, autonomous vehicles face significant challenges in identifying road boundaries and obstacles, mainly due to the significant decline in the quality of information obtained by sensors, such as blurred camera images and disturbed laser radar data. The existing technology relies on a large amount of labeled data for training, making it difficult to collect data for rare or extreme weather conditions, resulting in limited performance of the model in such scenarios and difficulty in effectively compensating for perception errors caused by optical noise, affecting decision accuracy. Figure 1 A flowchart of an environment perception and decision-making method for large models in autonomous driving is provided for embodiments of the present application, as shown in Figure 1 The method comprises the following steps:

[0070] Step 101: Obtain a road original image in multiple spectral channels;

[0071] In this step, the road original image refers to the unprocessed road scene data obtained directly by a multispectral imaging system, containing pixel-level radiation intensity information in multiple spectral channels such as visible light, near-infrared, and short-wave infrared, used to reflect the original environmental state disturbed by scattered noise in rainy and foggy weather.

[0072] In embodiments of the present application, the road original image data in visible light, near-infrared, and short-wave infrared bands is synchronously collected by a vehicle-mounted multispectral imaging device, where each pixel point contains radiation intensity information in multiple spectral channels; this step uses a spectroscope and a multi-channel sensor array to realize spectral separation, ensuring the ability to capture different wavelengths penetrating the medium under heavy rain or fog conditions.

[0073] Step 102: Based on the polarization state distribution of the road original image, generate light intensity distribution data carrying Stokes parameters;

[0074] In this step, the polarization state distribution refers to the polarization direction intensity distribution characteristics formed after the reflection of incident light waves on the surface of an object, calculated by the intensity difference of four polarization directions of 0°, 45°, 90°, and 135°, used to separate medium scattering noise and real object reflection signals. Stokes parameters refer to a group of physical quantities (S0, S1, S2) that describe the polarization state of light waves, generated based on polarization images through Mueller matrix transformation, used to quantify polarization modulation characteristics. Light intensity distribution data refers to a multispectral light intensity data set that fuses Stokes parameters, each pixel containing intensity values in three polarization dimensions of S0, S1, and S2, used to represent the optical polarization properties and energy distribution of each point in the scene.

[0075] In the embodiment of the present application, polarization analysis is performed on the road image, and intensity images of four polarization directions of 0°, 45°, 90° and 135° are captured by using a polarization camera; Stokes parameters (S0, S1, S2) are calculated by using a Mueller matrix, wherein S0 represents total light intensity, S1 represents the difference between the intensities of 0° and 90° polarized light, and S2 represents the difference between the intensities of 45° and 135° polarized light, and finally the light intensity distribution data of the fused polarization state are output.

[0076] Step 103: based on the Stokes parameters and the wavelengths of the multi-spectral channels, the polarization spectral coupling coefficient of the weather medium scattering noise is calculated;

[0077] In the embodiment of the present application, first, the light intensity component distribution difference between the high scattering sensitive waveband (such as 400-600nm) and the low scattering sensitive waveband (such as 800-1000nm) is calculated according to the Stokes parameter S1 component; then the difference is divided by the wavelength interval to obtain the discrete differential value; the uncorrected coupling amount is generated by combining the negative logarithmic conversion value of the atmospheric transmittance; finally, the uncorrected coupling amount is segmented and linearly corrected by using the real-time atmospheric density parameter, to obtain the polarization spectral coupling coefficient.

[0078] Step 104: based on the polarization spectral coupling coefficient and the light intensity distribution data, the road original image is compensated to generate a polarization spectral joint compensation image, and the polarization spectral joint compensation image is smoothed and strengthened to generate a multi-level feature map;

[0079] In the embodiment of the present application, the polarization spectral coupling coefficient is used to construct a polarization modulation factor matrix, which is multiplied by the light intensity distribution data pixel by pixel to generate polarization enhanced light intensity data; based on the polarization enhanced light intensity data, the light intensity ratio of the road original image and the real-time visibility, the attenuation compensation coefficient is calculated, the road original image is weighted and compensated to obtain the polarization spectral joint compensation image; finally, the basic feature map is extracted by Gaussian smoothing processing, the texture feature map is strengthened by Laplace edge detection operator, and the two are channel spliced to generate a multi-level feature map.

[0080] Step 105: noise suppression and feature propagation operations are performed on the multi-level feature map to reconstruct the object contour obscured by the weather medium, to obtain the reconstructed object spatial distribution.

[0081] In the embodiment of the present application, the statistical correlation values of each channel of the multi-level feature map are calculated, and a channel activation identifier is generated by comparing with a preset threshold; the channel weight vector is determined according to the identifier, so as to perform feature fusion on the multi-level feature map and the channel weight vector, and generate a noise suppression feature map; and after a plurality of rounds of iterative diffusion are performed on the noise suppression feature map, a diffusion stable feature map is obtained, and a second-order gradient tensor calculation is performed on the diffusion stable feature map, so as to determine the curvature extreme point distribution, and the curvature extreme points of adjacent pixels are connected according to the distribution, and a reconstructed object space distribution of the occluded object is generated.

[0082] Step 106: generating an obstacle avoidance control instruction of the autonomous vehicle according to the reconstructed object space distribution and the road boundary geometric feature;

[0083] In this step, the obstacle avoidance control instruction indicates a set of operation instructions that can be parsed by the vehicle execution mechanism, including a steering wheel deflection control amount (calculated based on the trajectory curvature radius) and a speed constraint value (calculated based on the maximum curvature of the trajectory and the centripetal acceleration threshold), which are used to realize real-time dynamic obstacle avoidance.

[0084] In the embodiment of the present application, the position vectors corresponding to each discrete point on the surface of the obstacle relative to the reference coordinate system are extracted according to the reference coordinate system origin of the reconstructed object space distribution; the road boundary semantic segmentation and topological optimization are performed based on the polarization spectrum joint compensation image to generate the boundary geometric feature; after fitting the boundary constraint equation according to the boundary geometric feature, the orthogonal projection of each discrete point to the reference coordinate system is performed to calculate the minimum Euclidean distance vector between each obstacle projection point and the origin of the ego-vehicle coordinate, and the obstacle avoidance direction offset angle is solved; the fan-shaped safety area is constructed in combination with the boundary constraint equation, and a parameterized trajectory cluster is generated, and the parameterized trajectory with the minimum average distance change rate is selected as the optimal obstacle avoidance trajectory, so as to generate the obstacle avoidance control instruction of the autonomous vehicle.

[0085] In the embodiment of the present application, the polarization spectrum coupling characteristics are analyzed by using the Stokes parameters, the medium scattering noise in heavy rain or heavy fog is accurately separated, the image blurring and point cloud distortion problems caused by scattering interference of the existing camera are solved; the dynamic generation of the polarization spectrum coupling coefficient replaces the strong dependence of the traditional deep learning on the labeled data, and the generalization ability of the rare weather scene is improved; the object space distribution is quickly reconstructed by the noise suppression feature propagation, the obstacle avoidance instruction is generated by combining the optimal obstacle avoidance trajectory, and the environment perception and decision delay are reduced to the safety response threshold that can meet the automatic driving.

[0086] The present application provides a specific embodiment, step 103, based on the Stokes parameters and the wavelength of the multi-spectral channel, the polarization spectrum coupling coefficient of the weather medium scattering noise is calculated, which specifically includes the following steps:

[0087] Step 301: based on the Stokes parameters, the polarization ellipse azimuth angle is calculated to determine the light intensity component distribution of each polarization direction;

[0088] In this step, the polarization ellipse azimuth angle refers to the angle between the polarization light electric field vector vibration direction and the reference axis, which is calculated by arctan(S2 / S1) and is used to determine the polarization main direction. The light intensity component distribution refers to the light intensity value set decomposed according to the horizontal, vertical, 45° and 135° four polarization directions, which is calculated based on the Stokes parameters through the horizontal component formula.

[0089] In the embodiment of the application, the ratio relationship between the S1 component and the S2 component of the Stokes parameters is analyzed, and the polarization ellipse azimuth angle θ (θ=0.5×arctan(S2 / S1)) is calculated. According to the polarization ellipse azimuth angle, the incident light is decomposed into horizontal, vertical, 45° and 135° four orthogonal polarization directions, and the light intensity component values of each direction are calculated. Specifically, the light intensity component value of the horizontal direction is S0+S1, the light intensity component value of the vertical direction is S0-S1, the light intensity component value of the 45° direction is S0+S2, and the light intensity component value of the 135° direction is S0-S2. Finally, the light intensity component distribution containing four polarization intensity is generated.

[0090] Step 302: according to the scattering sensitivity threshold, the wavelengths of the multi-spectral channel are divided into a high scattering sensitive band and a low scattering sensitive band;

[0091] In this step, the scattering sensitivity threshold refers to the critical wavelength value (600nm) for dividing the scattering intensity of the spectral band, which is set based on the inverse fourth power relationship between the scattering intensity and the wavelength in the Rayleigh scattering law. The high scattering sensitive band refers to the blue-green light band (400-600nm) with a wavelength less than the scattering sensitivity threshold, and the scattering intensity thereof increases in the fourth power with the decrease of the wavelength. The low scattering sensitive band refers to the near-infrared band (700-1000nm) with a wavelength greater than the scattering sensitivity threshold, and the scattering intensity thereof is significantly lower than that of the visible light band.

[0092] In the embodiment of the application, the preset scattering sensitivity threshold is 600nm, the blue-green light channel (400-600nm) with a wavelength less than 600nm is divided into a high scattering sensitive band, and the infrared channel (700-1000nm) with a wavelength greater than 600nm is divided into a low scattering sensitive band.

[0093] Step 303: the light intensity component distribution difference amount corresponding to the high scattering sensitive band and the low scattering sensitive band is calculated respectively, and the discrete differential value of the light intensity component distribution difference amount with the wavelength change of the multi-spectral channel is calculated;

[0094] In this step, the light intensity component distribution difference amount refers to the absolute difference between the average values of the polarized light intensity in the high scattering sensitive wave band and the low scattering sensitive wave band, reflecting the differential influence of scattering on the polarized light. The discrete differential value refers to the discrete approximate derivative of the light intensity component distribution difference amount with respect to the wavelength.

[0095] In the embodiment of the present application, the difference between the average intensity of the horizontal polarization component in the high scattering sensitive wave band and the average intensity of the horizontal polarization component in the low scattering sensitive wave band is calculated respectively as the light intensity component distribution difference amount ΔI; and the difference amount is divided by the center wavelength interval Δλ (such as Δλ=50nm) of the adjacent spectral channels to generate the discrete differential value δ=ΔI / Δλ, so as to quantify the rate of change of the scattering intensity with respect to the wavelength.

[0096] Step 304: performing a negative logarithmic conversion operation on the atmospheric transmittance parameter to generate an atmospheric optical action amount, and combining the discrete differential value to generate an uncorrected coupling amount;

[0097] In this step, the atmospheric optical action amount refers to the physical quantity after the negative natural logarithmic conversion of the atmospheric transmittance, which is equivalent to the atmospheric optical thickness. The uncorrected coupling amount refers to the product of the discrete differential value and the atmospheric optical action amount, which represents the scattering noise intensity without considering the density change.

[0098] In the embodiment of the present application, the atmospheric transmittance τ collected by the meteorological sensor is obtained, and the negative natural logarithmic conversion is performed to obtain the atmospheric optical action amount δ, i.e. δ=-ln(τ); and the discrete differential value is multiplied by the atmospheric optical action amount to generate the uncorrected coupling amount.

[0099] Step 305: performing linear correction on the uncorrected coupling amount according to the atmospheric density parameter to generate a corrected coupling amount, and taking the corrected coupling amount as the polarized spectral coupling coefficient of the weather medium scattering noise;

[0100] In this step, the corrected coupling amount refers to the coupling parameter after the piecewise linear correction of the atmospheric density parameter, which is output as the final polarized spectral coupling coefficient. The weather medium scattering noise refers to the non-uniform scattering effect of rain and fog particles on the incident light, which causes distortion of the sensor received signal. The polarized spectral coupling coefficient refers to the physical parameter quantifying the scattering noise intensity, which integrates the characteristics of polarization, spectrum and meteorology.

[0101] In the embodiment of the present application, the atmospheric density value is obtained in real time, the density ratio of the atmospheric density value and the atmospheric density reference value is calculated, and the boundary truncation processing is performed on the density ratio to obtain the truncated density ratio; when the truncated density ratio is less than or equal to the preset threshold value, the first linear function is used to generate the density correction factor, and when the truncated density ratio is greater than the preset threshold value, the second linear function is used to generate the density correction factor; the uncorrected coupling amount and the density correction factor are multiplied to obtain the corrected coupling amount, which is taken as the polarized spectral coupling coefficient.

[0102] The embodiment of the present application analyzes the directional characteristics of rain and fog particle scattering through the polarization ellipse azimuth angle, solves the problem that the traditional method cannot separate the multi-wavelength scattering noise, and combines the double-band spectral difference analysis; the logarithmic conversion of atmospheric transmittance and real-time correction of density are introduced, so that the polarization spectrum coupling coefficient automatically adapts to the change of rain and fog concentration, and the defects of the fixed parameter model in the sudden weather are overcome; based on the band division and discrete differential calculation value of the Rayleigh scattering law, the physical level noise quantization that can be verified is realized, and the dependence on the labeled data of the deep learning is got rid of.

[0103] The present application provides a specific embodiment, step 305, linearly correcting the uncorrected coupling quantity according to the atmospheric density parameter to generate a corrected coupling quantity, and taking the corrected coupling quantity as the polarization spectrum coupling coefficient of weather medium scattering noise, specifically comprising the following steps:

[0104] Step 311: obtaining a real-time atmospheric density value, and calling an atmospheric density reference value from a pre-stored meteorological parameter database;

[0105] In this step, the atmospheric density reference value refers to the standard sea level atmospheric density value 1.225 kg / m³, which is stored in the meteorological parameter database as the reference value for density correction.

[0106] In the embodiment of the present application, the atmospheric density value (unit: kg / m³) of the environment where the vehicle is located is collected in real time by the vehicle-mounted atmospheric density sensor; at the same time, the standard sea level atmospheric density reference value (which is a fixed value 1.225 kg / m³) is called from the meteorological parameter database pre-installed in the automatic driving system.

[0107] Step 312: calculating the density ratio of the real-time atmospheric density value and the atmospheric density reference value, and performing boundary truncation processing on the density ratio to obtain a truncated density ratio;

[0108] In this step, the density ratio parameter refers to the quotient value of the real-time atmospheric density value and the atmospheric density reference value, reflecting the degree of deviation of the current atmospheric density from the standard state. The truncated density ratio parameter refers to the density ratio after boundary truncation processing, and the value range is limited to the interval [0.6, 2.5], corresponding to the atmospheric density variation limit of-2000 meters to 4000 meters above sea level.

[0109] In the embodiment of the present application, the density ratio = real-time atmospheric density value ÷ atmospheric density reference value; when the density ratio is less than the preset lower threshold 0.6, it is forcibly set to 0.6, and when it is greater than the preset upper threshold 2.5, it is forcibly set to 2.5, and finally the truncated density ratio parameter is output to ensure that the parameter is within the physically reasonable range.

[0110] Step 313: input the truncated density ratio into a double-interval linear function to obtain a density correction factor, wherein the double-interval linear function includes a first linear function and a second linear function, the first linear function is used when the density ratio is less than or equal to a preset threshold, and the second linear function is used when the density ratio is greater than the preset threshold;

[0111] In this step, the double-interval linear function refers to a segmented linear function with a threshold of 1.0, including a first linear function and a second linear function. The first linear function refers to a linear function used in normal or high pressure environment, which is in the form of: density correction factor = density ratio * weight A + bias A. The second linear function refers to a linear function used in low pressure environment, which is in the form of: density correction factor = density ratio * weight B + bias B. The preset threshold refers to the threshold of 1.0 of the double-interval linear function, which corresponds to the standard sea level atmospheric density state.

[0112] In the embodiment of the application, the preset threshold is 1.0 (corresponding to the sea level reference): if the truncated density ratio is less than or equal to 1.0, the first linear function is used to calculate the density correction factor, i.e. density correction factor = truncated density ratio * first weight coefficient + first bias); if it is greater than 1.0, the second linear function is used, i.e. density correction factor = truncated density ratio * second weight coefficient + second bias.

[0113] Step 314: performing a scalar multiplication operation on the density correction factor and the uncorrected coupling quantity to generate a corrected coupling quantity, and taking the corrected coupling quantity as the polarization spectrum coupling coefficient of the weather medium scattering noise;

[0114] In the embodiment of the application, the density correction factor * the uncorrected coupling quantity = the corrected coupling quantity, and the corrected coupling quantity is taken as the polarization spectrum coupling coefficient of the weather medium scattering noise, so as to facilitate subsequent compensation processing.

[0115] The embodiment of the application solves the failure problem of the fixed compensation coefficient in the plateau / basin and other pressure mutation areas; the boundary truncation processing avoids numerical overflow under extreme weather (such as typhoon / high altitude), and ensures the robustness of the system under complex weather conditions; the double-interval function matches the nonlinear relationship between atmospheric density and scattering coefficient, so that the noise quantization is more in line with the physical law.

[0116] The application provides a specific embodiment, step 104, according to the polarization spectrum coupling coefficient and the light intensity distribution data, the road original image is compensated to generate a polarization spectrum joint compensation image, and the polarization spectrum joint compensation image is smoothed and strengthened to generate a multi-level feature map, which specifically includes the following steps:

[0117] Step 401: constructing a polarization modulation factor matrix based on the polarization spectral coupling coefficient and the Stokes parameter;

[0118] In this step, the polarization modulation factor matrix refers to a matrix composed of the product of the polarization spectral coupling coefficient and the S1 component of the Stokes parameter, which is used to enhance the signal intensity of the polarization-sensitive region.

[0119] In the embodiment of the present application, the polarization spectral coupling coefficient is multiplied by the S1 component in the Stokes parameter to generate a polarization modulation factor matrix, which has the same size as the light intensity distribution data, and each element value is the product of the polarization spectral coupling coefficient and the S1 value of the corresponding pixel point, which is used to enhance the signal intensity of the polarization-sensitive region.

[0120] Step 402: performing a point multiplication operation on the light intensity distribution data using the polarization modulation factor matrix to generate polarization-enhanced light intensity data;

[0121] In this step, the polarization-enhanced light intensity data refers to the output of the light intensity distribution data after the point multiplication operation of the polarization modulation factor matrix, which reflects the light intensity distribution of the scene after the polarization characteristics are strengthened.

[0122] In the embodiment of the present application, the polarization modulation factor matrix is multiplied by the S0 component (total light intensity) in the light intensity distribution data pixel by pixel, that is, the output value of each pixel point is equal to the S0 value of the point multiplied by the polarization modulation factor matrix, to generate an enhanced light intensity data set that fuses the polarization characteristics.

[0123] Step 403: calculating the light intensity ratio of the polarization-enhanced light intensity data and the original image of the road to generate a spectral attenuation proportion parameter according to the light intensity ratio, and combining the real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel;

[0124] In this step, the light intensity ratio refers to the quotient value of the polarization-enhanced light intensity data and the corresponding pixel value of the original image of the road, which represents the degree of medium attenuation. The spectral attenuation proportion parameter refers to a physical quantity obtained by converting the light intensity ratio into a negative natural logarithm, which is equivalent to an attenuation coefficient in the spectral dimension. The attenuation compensation coefficient refers to the exponential operation result of fusing the spectral attenuation proportion parameter and the visibility offset, which is used to compensate for the loss of light intensity of the original image.

[0125] In the embodiment of the present application, the following operations are performed for each spectral channel: calculating the light intensity ratio of the polarization-enhanced light intensity data and the original image; performing a logarithmic domain conversion operation on the light intensity ratio to generate a logarithmic attenuation parameter; obtaining the real-time visibility value and calculating the visibility offset thereof from the reference visibility; performing linear weighting on the logarithmic domain conversion operation and the visibility offset according to a preset weight to generate an unnormalized compensation parameter; performing an exponential domain inverse transformation operation on the unnormalized parameter to generate a spectral attenuation proportion parameter, and calculating the attenuation compensation coefficient of each channel in combination with the real-time visibility.

[0126] Step 404: compensating the road original image according to the attenuation compensation coefficient to generate a polarization spectrum joint compensation image;

[0127] In this step, the polarization spectrum joint compensation image refers to the output of the road original image after pixel-by-pixel correction by the attenuation compensation coefficient, which overcomes the polarization distortion and spectral attenuation.

[0128] In the embodiment of the application, each pixel value of the road original image is multiplied by the attenuation compensation coefficient of the corresponding spectral channel to perform pixel-level compensation. The compensation formula is: compensated pixel value = original pixel value x attenuation compensation coefficient. Finally, the polarization spectrum joint compensation image is output, which fuses polarization and spectral compensation.

[0129] Step 405: smoothing the polarization spectrum joint compensation image to generate a basic feature map, and performing texture enhancement processing on the basic feature map to generate a texture feature map;

[0130] In this step, the basic feature map refers to the low-pass filtered result of the polarization spectrum joint compensation image after Gaussian smoothing processing, which retains the main structure information of the image. The texture feature map refers to the high-frequency feature map generated by convolution of the Laplacian operator on the basic feature map, which enhances the edge and detail texture.

[0131] In the embodiment of the application, a Gaussian kernel function (size 15x15) is used to perform convolution smoothing operation on the compensation image to generate a basic feature map. Then, a Laplace edge detection operator (3x3 kernel) convolution operation is performed on the basic feature map to enhance the high-frequency texture information and generate a texture feature map.

[0132] Step 406: concatenating the basic feature map and the texture feature map in the channel dimension to generate a multi-level feature map;

[0133] In this step, the multi-level feature map refers to the fusion feature obtained by concatenating the basic feature map and the texture feature map in the channel dimension, which contains both macroscopic structure and microscopic detail information.

[0134] In the embodiment of the application, the basic feature map and the texture feature map are concatenated along the channel axis. If the size of the basic feature map is HxWxC1 and the size of the texture feature map is HxWxC2, then the size of the output multi-level feature map is HxWx(C1+C2).

[0135] In the embodiment of the application, the polarization-spectrum dual-domain joint compensation is used to restore the original texture of the target object in heavy rain with low visibility. Gaussian smoothing and Laplace sharpening are used to extract the macroscopic structure and microscopic edge of the image, which solves the problem of detail annihilation in rain and fog in traditional single-scale processing. Channel concatenation replaces the feature pyramid to reduce the consumption of computing resources and meet the real-time requirements of automatic driving.

[0136] The present application provides a specific embodiment, step 403, calculating the intensity ratio of the polarization enhanced light intensity data and the road original image, generating a spectral attenuation ratio parameter according to the intensity ratio, combining real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel, specifically including the following steps:

[0137] Step 411: Calculate the intensity ratio of the polarization enhanced light intensity data and the road original image, and perform a logarithmic domain conversion operation on the intensity ratio to generate a logarithmic attenuation parameter;

[0138] In this step, the logarithmic attenuation parameter refers to the physical quantity after the negative natural logarithm conversion of the intensity ratio, which is equivalent to the atmospheric optical thickness.

[0139] In the embodiment of the present application, the intensity ratio of the polarization enhanced light intensity data and the light intensity of the corresponding position of the road original image is calculated for each pixel point; then the negative natural logarithm operation is performed on the ratio to generate a logarithmic attenuation parameter, the calculation formula is: logarithmic attenuation parameter = -ln(intensity ratio), which is equivalent to the atmospheric optical thickness, reflecting the attenuation degree of the medium to the light radiation.

[0140] Step 412: Obtain real-time visibility and call the pre-stored reference visibility;

[0141] In this step, the reference visibility refers to the visibility reference value (1000 meters) under standard clear weather, which is stored in the system configuration file and used as the meteorological state comparison reference.

[0142] In the embodiment of the present application, the current visibility value (unit: meters) is collected in real time by the vehicle-mounted meteorological sensor; at the same time, the reference visibility threshold (fixed value 1000 meters) is called from the pre-stored configuration file, which is based on the international standard clear weather visibility setting.

[0143] Step 413: Calculate the visibility offset of the real-time visibility and the reference visibility;

[0144] In this step, the visibility offset refers to the absolute value of the difference between the real-time visibility and the reference visibility, which is used to quantify the degree of meteorological severity.

[0145] In the embodiment of the present application, the visibility offset = |reference visibility-real-time visibility|).

[0146] Step 414: Linearly weight the logarithmic attenuation parameter and the visibility offset to generate an unnormalized compensation parameter;

[0147] In the embodiment of the present application, the logarithmic attenuation parameter is multiplied by a first weight coefficient (0.7), the visibility offset is multiplied by a second weight coefficient (0.3), and then the two products are added to generate an unnormalized compensation parameter, i.e., unnormalized compensation parameter = logarithmic attenuation parameter * 0.7 + visibility offset * 0.3.

[0148] Step 415: performing an exponential domain inverse transformation operation on the unnormalized compensation parameter to generate a spectral attenuation ratio parameter;

[0149] In this step, the unnormalized compensation parameter refers to an intermediate value obtained by linearly superimposing the logarithmic attenuation parameter and the visibility offset according to the preset weight, serving as an input carrier of the exponential inverse transformation.

[0150] In the embodiment of the present application, an exponential operation with a natural constant e is performed on the unnormalized compensation parameter to generate the spectral attenuation ratio parameter, i.e., spectral attenuation ratio parameter = e unnormalized compensation parameter, which represents the light intensity gain multiple to be compensated.

[0151] Step 416: performing a product operation on the spectral attenuation ratio parameter and the real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel;

[0152] In the embodiment of the present application, the spectral attenuation ratio parameter is multiplied by a normalized value of the real-time visibility parameter (i.e., real-time visibility / 1000) to generate the attenuation compensation coefficient of each spectral channel, i.e., attenuation compensation coefficient = spectral attenuation ratio parameter * (real-time visibility / 1000).

[0153] The embodiment of the present application accurately quantifies the medium absorption and scattering effect by logarithmic conversion of the light intensity ratio, overcomes the problem of relying on empirical parameters in the traditional method, and dynamically adjusts the compensation strength by fusing the visibility offset, thereby maintaining stable performance in the visibility mutation scene (such as a fog).

[0154] The present application provides a specific embodiment, step 105, performing noise suppression and feature propagation operations on the multi-level feature map to reconstruct the object contour obscured by the weather medium, to obtain a reconstructed object spatial distribution, specifically including the following steps:

[0155] Step 501: calculating a statistical correlation value of each channel based on the channel data of the multi-level feature map, and comparing the statistical correlation value with a preset threshold to generate a channel activation state identifier;

[0156] In this step, the channel data refers to a set of all pixel values of a single channel in the multi-level feature map, with a size of height x width, reflecting the feature response of a specific abstraction level. The statistical correlation value refers to the Pearson correlation coefficient between two channel data, used to measure the similarity of feature responses between channels. The preset threshold refers to the critical value (such as 0.7) for determining whether a channel is activated, which is set based on experience and is considered valid if it exceeds this value. The channel activation state identifier refers to a binary identification sequence composed of 0 or 1, with 1 indicating that the channel is strongly correlated (correlation coefficient >= 0.7) with other channels and the feature should be retained.

[0157] In the embodiment of the present application, the pixel value matrix of each channel of the multi-level feature map is extracted, and the statistical correlation value between any two channels is calculated, with the formula being: statistical correlation value = covariance / product of standard deviations of two channels. Each statistical correlation value is compared with the preset threshold (such as 0.7), and if the statistical correlation value >= 0.7, an activation identifier 1 is generated, otherwise 0 is generated. Finally, a binary channel activation state identifier is generated.

[0158] Step 502: determining a channel weight vector according to the channel activation state identifier;

[0159] In this step, the channel weight vector refers to a list of weight coefficients assigned to each channel, which is calculated based on the proportion of activation identifiers and used for feature fusion weighting.

[0160] In the embodiment of the present application, the number of channels with an activation state identifier of 1 is counted for each channel, and the proportion of the total number of channels is calculated. The channel weight of each channel = corresponding proportion x weight scaling factor + basic weight. All channel weight values are combined to form a channel weight vector.

[0161] Step 503: fusing the multi-level feature map with the channel weight vector to generate a noise suppression feature map;

[0162] In this step, the noise suppression feature map refers to a single channel image obtained by weighted fusion of the multi-level feature map with the channel weight vector, which suppresses irrelevant channel noise.

[0163] In the embodiment of the present application, channel weighting is performed on each channel of the multi-level feature map. Specifically, the channel data is multiplied by the corresponding channel weight value, and then summed along the channel dimension to generate a noise suppression feature map after fusion.

[0164] Step 504: performing a multi-round iterative feature propagation operation on the noise suppression feature map, and terminating the feature propagation operation when the number of iterations reaches a preset number, to generate a diffusion stable feature map;

[0165] In this step, the preset number of times refers to the upper limit number of feature propagation iteration rounds (such as 8 times), which can balance the reconstruction accuracy and computational efficiency through experimental verification. The diffusion stable feature map refers to the feature map converged after multiple rounds of iteration propagation, which contains enhanced object structure information.

[0166] In the embodiment of the application, the noise suppression feature map is subjected to a plurality of rounds of iteration feature propagation operation, each round of iteration performing: weighting and summing the current noise suppression feature map and the initial noise suppression feature map according to a preset ratio, and performing 5*5 Gaussian convolution on the weighted result to update the current feature map, and terminating when the iteration round reaches the preset number of times (such as 8 times), to generate a diffusion stable feature map.

[0167] Step 505: performing second-order gradient tensor calculation on the diffusion stable feature map to obtain the feature change rate tensor of each pixel point, to determine the curvature extreme point distribution, connecting the curvature extreme points of adjacent pixels according to the curvature extreme point distribution, to reconstruct the object contour obscured by the weather medium, and generate a reconstructed object spatial distribution;

[0168] In this step, the feature change rate tensor refers to a matrix group composed of second-order partial derivatives, containing XX, XY, YX, and YY derivative information. The curvature extreme point distribution refers to a set of spatial positions of pixel points with a maximum feature value greater than 0.8 in the feature change rate tensor, representing the turning points of the object contour. The curvature extreme point refers to a single pixel point in the curvature extreme point distribution, reflecting the geometric mutation position of the contour. The weather medium refers to atmospheric particles such as rain and fog that cause light scattering, causing the target object to be obscured. The reconstructed object spatial distribution refers to the three-dimensional point cloud data generated based on the Delaunay triangulation of the curvature extreme points, describing the complete geometric shape of the obscured object. Figure Two In the embodiment of the application, the second-order partial derivatives of the diffusion stable feature map in the X direction and the Y direction are calculated; the Hessian matrix of each pixel point is constructed (the matrix elements include the XX direction derivative, the XY direction derivative, the YX direction derivative, and the YY direction derivative); the eigenvalues of the Hessian matrix are solved, and the points with a maximum eigenvalue greater than 0.8 are marked as curvature extreme points; Delaunay triangulation connection is performed on the curvature extreme points in the 8-neighborhood, and the reconstructed object spatial distribution is generated based on the triangular mesh.

[0169] In the embodiment of the application, the channel correlation weighted fusion is used to improve the effective feature signal-to-noise ratio in heavy rain scenarios; and the curvature extreme point triangulation is used to restore the topological structure of the fog-obscured vehicle.

[0170] The application provides a specific embodiment, step 106, generating an obstacle avoidance control instruction for an autonomous vehicle according to the reconstructed object spatial distribution and the geometric features of the road boundary, specifically including the following steps:

[0171]

[0172] ​Step 601: defining a self-vehicle coordinate origin based on a reference coordinate system origin of a reconstructed object space distribution, while extracting a position vector corresponding to each discrete point on the surface of the obstacle relative to the reference coordinate system;

[0173] In this step, the reference coordinate system origin refers to the zero point of the three-dimensional coordinate system of the reconstructed object space distribution, which is usually set as the vehicle center position. The self-vehicle coordinate origin refers to the zero point of the vehicle local coordinate system defined with the reference coordinate system origin (0, 0, 0). The discrete point refers to a single three-dimensional coordinate point in the obstacle surface point cloud, which includes a position vector (x, y, z). The position vector refers to a three-dimensional space vector from the self-vehicle coordinate origin to the obstacle discrete point.

[0174] In the embodiment of the application, the three-dimensional coordinate system origin of the reconstructed object space distribution (usually the current position of the vehicle) is taken as the self-vehicle coordinate origin; the obstacle surface point cloud data is traversed, and the three-dimensional coordinates (x, y, z) of each discrete point relative to the origin are recorded to generate the position vector.

[0175] Step 602: performing semantic segmentation on the road boundary in the polarization spectrum joint compensation image to generate an initial road boundary mask;

[0176] In this step, the initial road boundary mask refers to the binary road region image generated by semantic segmentation, and the white pixels represent the road region.

[0177] In the embodiment of the application, the pre-trained full convolutional network is used to process the polarization spectrum joint compensation image, and a road region binary segmentation image is output; a morphological closing operation is performed to fill the holes to generate the initial road boundary mask, wherein the non-road region is 0 and the road region is 1.

[0178] Step 603: topologically optimizing the initial road boundary mask to generate road boundary geometric features, fitting a drivable area boundary constraint equation according to the road boundary geometric features, and orthogonally projecting each discrete point to the reference coordinate system according to the position vector to generate an obstacle projection point set;

[0179] In this step, the road boundary geometric feature refers to the parameterized curve of the road boundary after topological optimization, which is defined by the control point coordinates. The drivable area refers to the safe passing area surrounded by the road boundary geometric feature. The boundary constraint equation refers to the mathematical equation describing the road boundary curve. The reference coordinate system refers to the right-handed three-dimensional rectangular coordinate system established with the self-vehicle coordinate origin. The obstacle projection point set refers to the two-dimensional coordinate set after the orthogonal projection of the obstacle discrete point to the XY plane.

[0180] In the embodiment of the present application, a skeleton extraction operation is performed on the initial mask to obtain a single-pixel wide boundary line; isolated line segments with a length less than 10 pixels are removed to generate an optimized boundary point set, which is fitted with a cubic Bezier curve to generate a road boundary geometric feature; a boundary constraint equation, such as a straight line equation ax+by+c=0 or a curve parameter equation, is solved based on the curve control point. The Z coordinate of the position vector of each obstacle is set to zero (ignoring the height dimension), and is projected onto the XY plane of the reference coordinate system to generate a two-dimensional obstacle projection point set (the formula is projection point=(x,y,0)).

[0181] Step 604: Calculate the minimum Euclidean distance vector of the obstacle projection point set from the origin of the ego vehicle coordinate system to solve the obstacle avoidance direction offset angle, combine the boundary constraint equation, construct a fan-shaped safety region, and generate a parameterized trajectory cluster in the fan-shaped safety region;

[0182] In this step, the minimum Euclidean distance vector refers to the two-dimensional vector in the obstacle projection point set that is closest to the origin of the ego vehicle. The obstacle avoidance direction offset angle refers to the angle between the minimum Euclidean distance vector and the vehicle forward direction (Y-axis positive direction). The fan-shaped safety region refers to a fan-shaped plane region with the obstacle avoidance direction offset angle as the central axis and the road boundary as the constraint. The parameterized trajectory curve cluster refers to a set of cubic Bezier curves described by parameter equations generated within the fan-shaped region.

[0183] In the embodiment of the present application, the minimum Euclidean distance vector of the obstacle projection point set from the origin (0,0) of the ego vehicle coordinate system is calculated, and the angle between the vector and the vehicle forward direction (Y-axis) is the obstacle avoidance direction offset angle θ. A fan-shaped safety region is constructed with θ as the central axis and the opening angle φ of the road boundary equation as the opening. Seven cubic Bezier curves are uniformly generated in the fan-shaped safety region as the parameterized trajectory curve cluster.

[0184] Step 605: According to the parameterized trajectory cluster, calculate the average distance change rate of the trajectory point set to the obstacle projection point set, and select the parameterized trajectory with the smallest average distance change rate as the optimal obstacle avoidance trajectory.

[0185] In this step, the trajectory point set refers to a sequence of discrete point coordinates sampled at equal arc lengths on a single trajectory curve. The average distance change rate refers to the average value of the absolute value of the distance change rate along the arc length of the trajectory point set to the obstacle point set. The parameterized trajectory curve refers to a curve defined by control points using a parameter equation. The optimal obstacle avoidance trajectory refers to the parameterized trajectory curve with the smallest average distance change rate.

[0186] In the embodiment of the present application, 50 points are sampled for each trajectory to generate a trajectory point set; the distance D of each trajectory point to the nearest obstacle is calculated; the absolute value average of the distance change rate dD / ds along the trajectory arc length s is calculated; and the trajectory with the smallest average value is selected as the optimal obstacle avoidance trajectory.

[0187] Step 606: generating an obstacle avoidance control instruction of the autonomous vehicle according to the optimal obstacle avoidance trajectory;

[0188] In the embodiment of the application, the curvature radius extreme value of the optimal obstacle avoidance trajectory is calculated, the steering wheel deflection angle is calculated according to the vehicle wheelbase, that is, the steering wheel deflection angle = arctan (vehicle wheelbase ÷ curvature radius extreme value), the speed constraint is calculated based on the trajectory length and the maximum curvature, the speed constraint = sqrt (trajectory length ÷ maximum curvature), and the instruction set of the steering wheel deflection angle and the speed constraint is output as the obstacle avoidance control instruction of the autonomous vehicle.

[0189] The embodiment of the application filters out the trajectory with the most gentle change from the obstacle by the distance change rate average index, reduces the accident rate, and improves the success rate of sharp curve scenes by fusing the sector region generation of the road boundary constraint.

[0190] Figure 2 A structure diagram of an environment perception and decision system of a large model in autonomous driving is provided for the embodiment of the application, as shown in Figure 2 The system comprises:

[0191] An acquisition module 21 is configured to acquire a road original image of a multi-spectral channel.

[0192] A first generation module 22 is configured to generate light intensity distribution data carrying Stokes parameters based on a polarization state distribution of the road original image.

[0193] A calculation module 23 is configured to calculate a polarization spectrum coupling coefficient of weather medium scattering noise based on the Stokes parameters and wavelengths of the multi-spectral channel.

[0194] A compensation module 24 is configured to perform compensation processing on the road original image according to the polarization spectrum coupling coefficient and the light intensity distribution data to generate a polarization spectrum joint compensation image, and perform smoothing and strengthening processing on the polarization spectrum joint compensation image to generate a multi-level feature map.

[0195] A processing module 25 is configured to perform noise suppression and feature propagation operations on the multi-level feature map to reconstruct an object contour obscured by a weather medium to obtain a reconstructed object spatial distribution.

[0196] A second generation module 26 is configured to generate an obstacle avoidance control instruction of an autonomous vehicle according to the reconstructed object spatial distribution and road boundary geometric features.

[0197] Figure 2 The environment perception and decision system of the large model in autonomous driving can perform Figure 1The implementation principle and technical effects of the environment perception and decision method of the large model in autonomous driving according to the illustrated embodiment will not be described again. The specific manner in which each module, unit of the environment perception and decision system of the large model in autonomous driving according to the above embodiment performs operations has been described in detail in the embodiments related to the method, which will not be described in detail here.

[0198] In one possible design, Figure 2 The environment perception and decision system of the large model in autonomous driving according to the illustrated embodiment can be implemented as a computing device, such as Figure 3 As shown, the computing device can include a storage component 31 and a processing component 32.

[0199] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called for execution by the processing component 32.

[0200] The processing component 32 is configured to perform the above Figure 1 The environment perception and decision method of the large model in autonomous driving according to the illustrated embodiment.

[0201] The processing component 32 can include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic elements, for executing the above method.

[0202] The storage component 31 is configured to store various types of data to support the operation of the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0203] Of course, the computing device can also include other components, such as an input / output interface, a display component, a communication component, etc.

[0204] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0205] The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, etc.

[0206] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform, and the computing device can be a cloud server, and the processing component and the storage component can be basic server resources rented or purchased from the cloud computing platform.

[0207] The embodiment of the application further provides a computer storage medium storing a computer program, and the computer program can realize the above-mentioned method when being executed by a computer. Figure 1 The large model in the environment perception and decision method of the embodiment shown in the figure.

[0208] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-mentioned system, device and unit can refer to the corresponding processes in the foregoing method embodiments, and will not be described here.

[0209] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0210] Through the description of the foregoing embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and a necessary general hardware platform, and of course, it can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment or some parts of the embodiment.

[0211] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. A method for environment perception and decision-making of large models in autonomous driving, characterized in that, The method comprises the following steps: acquiring a road original image of a plurality of spectral channels; generating light intensity distribution data carrying Stokes parameters based on a polarization state distribution of the road original image; calculating a polarization spectrum coupling coefficient of weather medium scattering noise based on the Stokes parameters and wavelengths of the plurality of spectral channels; performing compensation processing on the road original image according to the polarization spectrum coupling coefficient and the light intensity distribution data, generating a polarization spectrum joint compensation image, and performing smoothing and strengthening processing on the polarization spectrum joint compensation image to generate a multi-level feature map; performing noise suppression and feature propagation operations on the multi-level feature map to reconstruct an object contour obscured by a weather medium, obtaining a reconstructed object spatial distribution; generating an obstacle avoidance control instruction of an autonomous vehicle according to the reconstructed object spatial distribution and a road boundary geometric feature; calculating a polarization spectrum coupling coefficient of weather medium scattering noise based on the Stokes parameters and wavelengths of the plurality of spectral channels, comprising: solving a polarization ellipse azimuth angle based on the Stokes parameters to determine a light intensity component distribution of each polarization direction; dividing the wavelengths of the plurality of spectral channels into a high scattering sensitivity band and a low scattering sensitivity band according to a scattering sensitivity threshold; calculating a light intensity component distribution difference quantity corresponding to the high scattering sensitivity band and the low scattering sensitivity band respectively, and calculating a discrete differential value of the light intensity component distribution difference quantity with respect to the wavelengths of the plurality of spectral channels; performing a negative logarithmic conversion operation on an atmospheric transmittance parameter to generate an atmospheric optical action quantity, and combining the discrete differential value to generate an uncorrected coupling quantity; performing linear correction on the uncorrected coupling quantity according to an atmospheric density parameter to generate a corrected coupling quantity, and taking the corrected coupling quantity as the polarization spectrum coupling coefficient of the weather medium scattering noise; performing linear correction on the uncorrected coupling quantity according to an atmospheric density parameter to generate a corrected coupling quantity, and taking the corrected coupling quantity as the polarization spectrum coupling coefficient of the weather medium scattering noise, comprising: acquiring a real-time atmospheric density value, and calling an atmospheric density reference value from a pre-stored meteorological parameter database; calculating a density ratio of the real-time atmospheric density value and the atmospheric density reference value, and performing boundary truncation processing on the density ratio to obtain a truncated density ratio; inputting the truncated density ratio into a double-interval linear function to obtain a density correction factor, wherein the double-interval linear function comprises a first linear function and a second linear function, the first linear function is used when the density ratio is less than or equal to a preset threshold, and the second linear function is used when the density ratio is greater than the preset threshold; performing scalar multiplication operation on the density correction factor and the uncorrected coupling quantity to generate a corrected coupling quantity, and taking the corrected coupling quantity as the polarization spectrum coupling coefficient of the weather medium scattering noise.

2. The method of claim 1, wherein, performing compensation processing on the road original image according to the polarization spectrum coupling coefficient and the light intensity distribution data, generating a polarization spectrum joint compensation image, and performing smoothing and strengthening processing on the polarization spectrum joint compensation image to generate a multi-level feature map, comprising: Construct a polarization modulation factor matrix based on the polarization spectrum coupling coefficient and the Stokes parameter; Perform a point multiplication operation on the light intensity distribution data by using the polarization modulation factor matrix to generate polarization enhanced light intensity data; Calculate the light intensity ratio of the polarization enhanced light intensity data and the road original image to generate a spectral attenuation ratio parameter according to the light intensity ratio, and combine the real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel; Perform compensation processing on the road original image according to the attenuation compensation coefficient to generate a polarization spectrum joint compensation image; Perform smoothing processing on the polarization spectrum joint compensation image to generate a basic feature map, and perform texture strengthening processing on the basic feature map to generate a texture feature map; Concatenate the basic feature map and the texture feature map in the channel dimension to generate a multi-level feature map.

3. The method of claim 2, wherein, Calculate the light intensity ratio of the polarization enhanced light intensity data and the road original image to generate a spectral attenuation ratio parameter according to the light intensity ratio, and combine the real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel, including: Calculate the light intensity ratio of the polarization enhanced light intensity data and the road original image, and perform a logarithmic domain conversion operation on the light intensity ratio to generate a logarithmic attenuation parameter; Obtain the real-time visibility and call the pre-stored reference visibility; Calculate the visibility offset of the real-time visibility and the reference visibility; Linearly weight the logarithmic attenuation parameter and the visibility offset to generate an unnormalized compensation parameter; Perform an exponential domain inverse transformation operation on the unnormalized compensation parameter to generate a spectral attenuation ratio parameter; Perform a product operation on the spectral attenuation ratio parameter and the real-time visibility to generate an attenuation compensation coefficient corresponding to each spectral channel.

4. The method of claim 1, wherein, Perform noise suppression and feature propagation operations on the multi-level feature map to reconstruct the object contour obscured by the weather medium to obtain the reconstructed object spatial distribution, including: Based on the channel data of the multi-level feature map, calculate the statistical correlation value of each channel, and compare the statistical correlation value with a preset threshold to generate a channel activation state identifier; Determine a channel weight vector according to the channel activation state identifier; Fuse the multi-level feature map and the channel weight vector to generate a noise suppression feature map; Perform a multi-round iteration feature propagation operation on the noise suppression feature map, and terminate the feature propagation operation when the iteration round reaches a preset number to generate a diffusion stable feature map; Perform a second-order gradient tensor calculation on the diffusion stable feature map to obtain a feature change rate tensor of each pixel point to determine a curvature extreme point distribution, connect the curvature extreme points of adjacent pixels according to the curvature extreme point distribution to reconstruct the object contour obscured by the weather medium, and generate a reconstructed object spatial distribution.

5. The method of claim 1, wherein, Generate an obstacle avoidance control instruction of an autonomous vehicle according to the reconstructed object spatial distribution and the road boundary geometric feature, including: Define the vehicle coordinate origin based on the reference coordinate system origin of the reconstructed object spatial distribution, and extract the position vector corresponding to each discrete point on the obstacle surface relative to the reference coordinate system; Perform semantic segmentation on the road boundary in the polarization spectrum joint compensation image to generate an initial road boundary mask; Perform topological optimization on the initial road boundary mask to generate road boundary geometric features, and fit a drivable area boundary constraint equation according to the road boundary geometric features; According to the position vector, orthogonally project each discrete point to the reference coordinate system to generate a set of obstacle projection points; Calculate the minimum Euclidean distance vector between the set of obstacle projection points and the origin of the ego vehicle coordinate system to solve the obstacle avoidance direction offset angle, combine the boundary constraint equation to construct a fan-shaped safety area, and generate a parameterized trajectory cluster in the fan-shaped safety area; According to the parameterized trajectory cluster, calculate the average distance change rate of the trajectory point set to the set of obstacle projection points, and take the parameterized trajectory with the minimum average distance change rate as the optimal obstacle avoidance trajectory; According to the optimal obstacle avoidance trajectory, generate an obstacle avoidance control instruction for the autonomous vehicle.

6. An environment perception and decision system in autonomous driving with large models, characterized in that, Comprise: An acquisition module configured to acquire a road original image of a plurality of spectral channels; A first generation module configured to generate light intensity distribution data carrying Stokes parameters based on the polarization state distribution of the road original image; A calculation module configured to calculate a polarization spectrum coupling coefficient of weather medium scattering noise based on the Stokes parameters and the wavelengths of the plurality of spectral channels; A compensation module configured to compensate the road original image according to the polarization spectrum coupling coefficient and the light intensity distribution data to generate a polarization spectrum joint compensation image, and perform smoothing and strengthening processing on the polarization spectrum joint compensation image to generate a multi-level feature map; A processing module configured to perform noise suppression and feature propagation operations on the multi-level feature map to reconstruct the object contour obscured by the weather medium, and obtain a reconstructed object spatial distribution; A second generation module configured to generate an obstacle avoidance control instruction for the autonomous vehicle according to the reconstructed object spatial distribution and road boundary geometric features; The calculation of the polarization spectrum coupling coefficient of weather medium scattering noise based on the Stokes parameters and the wavelengths of the plurality of spectral channels comprises: Based on the Stokes parameters, the polarization ellipse azimuth angle is solved to determine the light intensity component distribution of each polarization direction; According to the scattering sensitivity threshold, the wavelengths of the plurality of spectral channels are divided into high scattering sensitive bands and low scattering sensitive bands; Calculate the light intensity component distribution difference corresponding to the high scattering sensitive bands and the low scattering sensitive bands respectively, and calculate the discrete differential value of the light intensity component distribution difference with respect to the wavelengths of the plurality of spectral channels; Perform a negative logarithmic conversion operation on the atmospheric transmittance parameter to generate an atmospheric optical action quantity, and combine the discrete differential value to generate an uncorrected coupling quantity; According to the atmospheric density parameter, the uncorrected coupling quantity is linearly corrected to generate a corrected coupling quantity, which is taken as the polarization spectrum coupling coefficient of weather medium scattering noise; According to the atmospheric density parameter, the uncorrected coupling quantity is linearly corrected to generate a corrected coupling quantity, which is taken as the polarization spectrum coupling coefficient of weather medium scattering noise, comprising: Obtaining a real-time atmospheric density value, and calling an atmospheric density reference value from a pre-stored meteorological parameter database; Calculating a density ratio of the real-time atmospheric density value and the atmospheric density reference value, and performing boundary clipping processing on the density ratio to obtain a clipped density ratio; Inputting the clipped density ratio into a double-interval linear function to obtain a density correction factor, wherein the double-interval linear function includes a first linear function and a second linear function, the first linear function is used when the density ratio is less than or equal to a preset threshold value, and the second linear function is used when the density ratio is greater than the preset threshold value; Performing a scalar multiplication operation on the density correction factor and the uncorrected coupling quantity to generate a corrected coupling quantity, and taking the corrected coupling quantity as a polarization spectral coupling coefficient of weather medium scattering noise.

7. A computing device, comprising: The method comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to realize the environment perception and decision method of the large model in the automatic driving according to any one of claims 1-5.

8. A computer storage medium, characterized in that, The computer program is stored in the computer and is executed by the computer to realize the environment perception and decision method of the large model in the automatic driving according to any one of claims 1-5.

Images