Laser radar camera co-location perception method, system and medium

Abstract

The present invention relates to the field of co-location perception technology, and in particular to a co-location perception method, system and medium for a laser radar camera. The method comprises the following steps: extracting real-time ambient light data through an onboard camera lens on a sweeping robot and constructing a mobile collaborative perspective three-dimensional model to obtain a mobile collaborative perspective three-dimensional model; calculating the probability of environmental space distortion behavior based on the mobile collaborative perspective three-dimensional model to obtain environmental space distortion behavior probability data; performing spatial distortion structured identification on the environmental space distortion behavior probability data to obtain spatial distortion structured data; performing light distortion behavior correction processing according to the spatial distortion structured data to obtain light distortion behavior correction data; performing strategy gradient learning on the light distortion behavior correction data and constructing a light co-location perception model to obtain a light co-location perception model. The present invention makes the co-location perception technology more perfect by optimizing the co-location perception technology.

Description

Technical Field

[0001] The present invention relates to the field of co-location sensing technology, and in particular to a co-location sensing method, system and medium for a laser radar camera. Background Art

[0002] As two important sensors, LiDAR and camera play an important role in distance measurement and visual recognition on sweeping robots, respectively. However, they face many challenges in the data fusion process. LiDAR can provide high-precision distance information and work stably under various lighting conditions, but it lacks in-depth understanding of object details. Cameras can capture rich image information and details, but their performance is significantly affected by light changes, reflections, and occlusions, especially in low-light or complex environments, resulting in unstable positioning and object recognition accuracy. Therefore, how to effectively fuse the data of LiDAR and camera to improve perception accuracy and reliability has become a hot topic in current research. The co-location perception method of LiDAR and camera aims to solve the core problem in the fusion of such sensor data: how to ensure high accuracy of positioning and perception under the influence of interference factors such as uneven lighting, object occlusion, and environmental reflection. This method gradually improves the stability and accuracy of the fusion results through a refined processing flow, from sensor data preprocessing to deep fusion. In particular, in low-light or complex environments, the data from the camera on the sweeping robot cannot accurately reflect the real scene. At this time, the high-precision distance data provided by the lidar can help correct the lack of camera information and ensure that the system can stably and reliably perform environmental perception and object recognition. However, a traditional lidar camera co-location perception method has the problem of inaccurate analysis of ambient light changes, resulting in low accuracy in correcting environmental distortion during the reflection process. Summary of the invention

[0003] Based on this, it is necessary to provide a lidar camera co-location perception method, system and medium to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above-mentioned purpose, a laser radar camera co-location perception method, system and medium are provided, and the method comprises the following steps:

[0005] Step S1: extracting real-time ambient light data through an onboard camera lens to obtain real-time ambient light data; constructing a mobile collaborative perspective three-dimensional model based on the real-time ambient light data to obtain a mobile collaborative perspective three-dimensional model;

[0006] Step S2: Based on the mobile collaborative perspective three-dimensional model, the real-time environmental light data is evaluated for light reflection intensity to obtain light reflection intensity evaluation data; the light reflection intensity evaluation data is analyzed for structural interference effect to obtain light reflection structural interference effect data; the probability of environmental space distortion behavior is calculated based on the light reflection structural interference effect data to obtain environmental space distortion behavior probability data;

[0007] Step S3: performing spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data; performing light distortion behavior correction processing according to the spatial distortion structured data to obtain light distortion behavior correction data;

[0008] Step S4: Based on the policy gradient algorithm, policy gradient learning is performed on the light distortion behavior correction data to obtain light correction policy gradient data; a light co-location perception model is constructed according to the light correction policy gradient data to obtain the light co-location perception model, and the light co-location perception model is sent to the data cloud platform to execute the lidar camera co-location perception method.

[0009] The present invention uses an airborne camera lens to capture light information in the environment in real time to ensure the timeliness and accuracy of the data. A three-dimensional model is constructed through real-time data, and the model can reflect the real situation of the environment, including the distribution and influence of light. Based on the mobile collaborative perspective three-dimensional model, the real-time environmental light data is evaluated to quantify the intensity and characteristics of light reflection. By evaluating the data, the structural interference effect of light in the environment is analyzed, that is, the complex interactive influence of light reflection, refraction and diffraction on the surface of the object. According to the light reflection structural interference effect data, the probability of spatial distortion behavior occurring in the environment is calculated. These data are helpful to understand and predict the behavior of light in a specific environment, including its impact on visual perception and object shape cognition. By analyzing the probability data of environmental spatial distortion behavior, specific spatial distortion structured features, such as bending, twisting or deformation of light, are identified. Based on the identified distortion features, correction processing of light distortion behavior is performed to improve the accuracy of light transmission and the accuracy of environmental perception. The light distortion behavior correction data is learned and optimized using a policy gradient algorithm to find the optimal correction strategy. Optimized light correction strategy gradient data are obtained, which reflect how to adjust light transmission according to real-time environmental data to reduce distortion effects. Based on the results of policy gradient learning, a light co-location perception model is constructed. This model can adjust the position and angle of the laser radar and camera according to the actual light conditions in a real-time or near-real-time environment to achieve more accurate co-location perception. The optimized light co-location perception model is sent to the data cloud platform to execute the co-location perception method of the laser radar and camera in a wide range of application scenarios. By correcting the light distortion behavior and applying the light co-location perception model, the accuracy of environmental perception and spatial perception is improved, and errors and uncertainties are reduced. Therefore, the present invention is an improved processing of a traditional laser radar camera co-location perception method, which solves the problem that a traditional laser radar camera co-location perception method has an inaccurate analysis of ambient light changes, resulting in low accuracy of environmental distortion correction during the reflection process, improves the accuracy of ambient light change analysis, and improves the accuracy of environmental distortion correction during the reflection process.

[0010] Preferably, step S1 comprises the following steps:

[0011] Step S11: collecting real-time environmental data through an onboard camera lens to obtain real-time environmental data;

[0012] Step S12: extracting real-time ambient light data from the real-time ambient data to obtain real-time ambient light data;

[0013] Step S13: Obstacle real-time ranging is performed based on real-time environmental data by using a laser radar to obtain obstacle real-time ranging data;

[0014] Step S14: construct a mobile collaborative perspective three-dimensional model for the real-time environmental data according to the real-time environmental light data and the real-time obstacle ranging data to obtain a mobile collaborative perspective three-dimensional model.

[0015] The present invention uses an onboard camera lens to obtain real-time environmental data, which includes images, videos or other perception data. This step ensures that comprehensive information of the current environment is obtained. Light information, including light intensity, light distribution, etc., is extracted from the collected environmental data. These data are the basis for subsequent three-dimensional model construction and light reflection evaluation. Sensors such as laser radar are used to perform real-time ranging of obstacles based on real-time environmental data. These data not only contribute to environmental perception, but also provide an important basis for subsequent modeling and path planning. Combined with real-time environmental light data and real-time ranging data of obstacles, an accurate mobile collaborative perspective three-dimensional model is constructed. This model combines the physical properties of light and the spatial position of obstacles in the environment, and can provide accurate environmental visual information. By integrating different sources of real-time environmental data (vision, laser ranging), step S1 improves the comprehensive perception of the environment. The construction of the mobile collaborative perspective three-dimensional model is not only based on light data, but also combines the spatial information of obstacles, so it can provide more accurate environmental simulation and scene analysis capabilities. Collecting and processing real-time data enables the system to respond quickly to environmental changes, which is of great significance for instant decision-making and safety.

[0016] Preferably, step S2 comprises the following steps:

[0017] Step S21: performing light change fluctuation analysis on the real-time ambient light data based on the mobile collaborative perspective three-dimensional model to obtain light change fluctuation data;

[0018] Step S22: evaluating the light reflection intensity of the light change fluctuation data to obtain light reflection intensity evaluation data;

[0019] Step S23: performing structural interference effect analysis on the light reflection intensity evaluation data to obtain light reflection structural interference effect data;

[0020] Step S24: Calculate the probability of the environmental space distortion behavior according to the light reflection structure interference effect data and the light reflection intensity evaluation data to obtain the environmental space distortion behavior probability data.

[0021] The present invention analyzes the fluctuations of real-time environmental light data through a mobile collaborative perspective three-dimensional model. These fluctuations can be caused by factors such as different light sources, obstacles and terrain in the environment. It ensures the accurate capture of light change fluctuation data, which helps to understand the dynamic characteristics and change trends of light in the environment. Based on the light change fluctuation data, the reflection intensity of light in different areas and surfaces is evaluated. These data provide a quantitative analysis of the distribution of light in the environment. The light reflection intensity evaluation data directly affects the realism and accuracy of visual simulation, image rendering and augmented reality technology. The light reflection intensity evaluation data is further analyzed to explore the interference effect of light on different surfaces and structures. This includes optical phenomena such as reflection, refraction and diffraction. Structural interference effect data helps understand the complex propagation mode of light in a specific environment, and provides a basis for more accurate light simulation and environmental simulation. Based on the light reflection structural interference effect data and the light reflection intensity evaluation data, the probability of spatial distortion behavior occurring in the environment is calculated. These data are crucial to improving the visual accuracy in virtual reality, augmented reality and simulated environments, and help reduce visual misleading and inconsistency in user perception.

[0022] Preferably, step S23 includes the following steps:

[0023] Step S231: marking the light reflection path of the light reflection intensity evaluation data to obtain light reflection path marking data;

[0024] Step S232: performing light reflection interlaced structure analysis according to the light reflection path mark data to obtain light reflection interlaced structure data;

[0025] Step S233: evaluating the color difference between different reflected lights on the light reflection interlaced structure data according to the light reflection intensity evaluation data, to obtain the color difference data of the reflected light;

[0026] Step S234: calculating the light wave phase difference between different reflected light rays for the light reflection interlaced structure data according to the reflected light chromatic aberration data and the light reflection intensity evaluation data, to obtain the reflected interlaced structure light wave phase difference data;

[0027] Step S235: Perform structural interference effect analysis based on the phase difference data of the reflected interlaced structure light wave to obtain light reflection structure interference effect data.

[0028] The present invention can accurately track the reflection path of light in the environment by marking the path of the light reflection intensity evaluation data. These data are very important for analyzing the specific path and distance of light propagation. A detailed description of the light reflection behavior is provided, which provides basic data for the subsequent analysis of the light interlaced structure. Based on the light reflection path marking data, the interlaced reflection of light on different surfaces and structures is analyzed. These data reveal the reflection mode and path intersection phenomenon of light in a complex environment. By analyzing the light interlaced structure, the complexity of light reflection in the environment can be deeply understood, including factors such as multiple reflections and changes in reflection angles. According to the light reflection intensity evaluation data, the light color difference between different reflected light is evaluated. These data take into account the color change and attenuation of light during the reflection process. For virtual reality, augmented reality and image rendering applications, light color difference data is crucial to ensure the authenticity and color fidelity of visual scenes. Based on the light color difference data of the reflected light and the light reflection intensity evaluation data, the light wave phase difference between different reflected light is calculated. These data reflect the phase change of light waves during spatial propagation. Through the light wave phase difference data, the optical interference phenomenon can be quantified, providing a basis for the accurate simulation and emulation of environmental optical properties. Based on the phase difference data of the reflected staggered structure light waves, the interference effect of light on the structure is further analyzed. These data reveal the interference phenomenon caused by light reflection and propagation in the environment. The structural interference effect data provides key information for accurately simulating optical phenomena in complex environments, such as the design optimization of optical equipment and the prediction of visual performance under environmental conditions.

[0029] Preferably, step S24 comprises the following steps:

[0030] Step S241: calculating the optical path difference of different light reflections on the light reflection structure interference effect data to obtain the light reflection optical path difference data;

[0031] Step S242: performing medium reflection interference simulation on the light reflection structure interference effect data according to the light reflection optical path difference data to obtain medium reflection interference simulation data;

[0032] Step S243: performing polarization light interaction analysis on the light reflection structure interference effect data according to the medium reflection interference simulation data to obtain reflection structure polarization light interaction data;

[0033] Step S244: performing light distortion critical state analysis based on the reflection structure polarization light interaction data, the medium reflection interference simulation data, and the light reflection optical path difference data to obtain light distortion critical state data;

[0034] Step S245: performing regression interval estimation on the light distortion critical state data according to the light reflection intensity evaluation data to obtain light reflection distortion interval estimation data;

[0035] Step S246: Calculate the probability of environmental space distortion behavior based on the light reflection distortion interval estimation data to obtain environmental space distortion behavior probability data.

[0036] The present invention can quantify the path length difference of light in the process of spatial propagation by calculating the optical path difference of different light reflection paths. These data are very important for understanding the time delay and spatial distribution of light reflection. The optical path difference data provides an accurate measurement of the optical path length, which provides a basis for subsequent optical simulation and interference effect analysis. Based on the optical path difference data of light reflection, the optical interference phenomenon on the medium surface is simulated. These simulation data reflect the phase change and interference effect that occurs when the light is reflected on the medium surface. The medium reflection interference simulation data helps to evaluate the reflection characteristics of optical materials and provides an important basis for material selection and optical design. According to the medium reflection interference simulation data, the polarization state change and interaction of light on the surface of the structure are analyzed. These data are crucial for understanding the polarization behavior of light on complex surfaces. The polarization light interaction data helps to optimize the design of polarization filters, coatings and optical components, and improve the efficiency and performance of optical systems. Based on the reflection structure polarization light interaction data and the light reflection optical path difference data, the distortion and distortion state of light under specific conditions are analyzed. These data reflect the propagation restrictions and distortion of light in complex environments. Light distortion critical state data provides key information for predicting the performance of visual systems, sensors, and optical devices in complex environments, and helps optimize device design and application environment selection. Based on the light reflection distortion interval estimation data, the probability of optical distortion in the environment is calculated. These data help evaluate and predict the impact of optical distortion on visual systems and image processing. Environmental space distortion behavior probability data provides important visual environment predictions for applications such as virtual reality, augmented reality, and autonomous driving, supporting real-time decision-making and system optimization.

[0037] Preferably, performing regression interval estimation on the light distortion critical state data according to the light reflection intensity evaluation data comprises the following steps:

[0038] According to the light reflection intensity evaluation data, the light distortion critical state data is subjected to light distortion increasing law identification to obtain light distortion increasing law data;

[0039] Perform incremental stepwise regression analysis on the data of increasing law of light distortion to obtain the incremental stepwise regression data of distortion;

[0040] According to the distortion increasing stepwise regression data, multicollinearity constraint processing is performed on the light distortion increasing law data to obtain the collinear constraint data of the light distortion increasing law;

[0041] According to the collinear constraint data of the increasing law of light distortion, regression analysis is performed on the data of the increasing law of light distortion to obtain the collinear regression data of the increasing law of light distortion;

[0042] The regression interval is estimated based on the light distortion incremental collinear regression data to obtain the light reflection distortion interval estimation data.

[0043] The present invention identifies the law of increasing light distortion based on the light reflection intensity evaluation data. This step focuses on determining how optical distortion gradually increases or decreases under different conditions. The light distortion increasing law data is modeled as a mathematical equation using a stepwise regression analysis method. Stepwise regression helps to gradually add the most relevant variables to optimize the predictive ability of the model. The multicollinearity present in the distortion increasing law data is processed to ensure the robustness and reliability of the regression model. This step helps to reduce the redundant information between the variables in the model and improve the prediction accuracy. Based on the data processed with collinearity constraints, regression analysis is performed to further optimize the prediction model of increasing light distortion. This model can more accurately describe the trend and law of optical distortion. Regression interval estimation is performed using the collinear regression data of increasing light distortion. This step provides the range of occurrence of optical distortion, thereby helping to evaluate the performance and stability of the optical system under different environmental conditions.

[0044] Preferably, step S3 comprises the following steps:

[0045] Step S31: performing spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data;

[0046] Step S32: extracting features from the spatial distortion structured data to obtain spatial distortion structure feature data;

[0047] Step S33: performing a distortion structure multi-dimensional analysis on the spatial distortion structure data according to the spatial distortion structure feature data to obtain the distortion structure multi-dimensional feature data;

[0048] Step S34: performing light distortion behavior correction processing according to the distortion structure multi-dimensional feature data to obtain light distortion behavior correction data.

[0049] The present invention analyzes and identifies the probability data of light distortion behavior in the environment and organizes it into structured data. This step helps to convert complex environmental distortion behavior into an operational data form that can be further analyzed. A detailed description and statistical information of the environmental distortion phenomenon are provided, providing basic data for subsequent analysis. Key features, such as the intensity, frequency, spatial distribution and other characteristics of the distortion, are extracted from the structured distortion data. These features reflect important aspects of the distortion phenomenon and help to further understand the nature and behavior of light distortion. A detailed description and quantitative analysis of light distortion behavior are provided, providing a basis for subsequent processing and correction. Based on the spatial distortion structure feature data, multi-dimensional statistics and analysis, such as cluster analysis, principal component analysis, etc., are performed. These analyses can reveal the relationship and difference between different distortion patterns. It helps to identify and understand different types of light distortion patterns and provides in-depth data support for formulating correction strategies. Based on the multi-dimensional feature data of the distortion structure, a correction strategy for light distortion behavior is formulated. This includes adjusting the position of the optical device, optimizing the environmental settings or adopting digital correction technology. Through effective correction measures, the impact caused by environmental distortion in the optical system is reduced or eliminated, and the performance and stability of the system are improved.

[0050] Preferably, step S34 includes the following steps:

[0051] Step S341: calculating the curvature of the twisted structure multi-dimensional feature data to obtain the curvature data of the twisted structure;

[0052] Step S342: performing hierarchical feature processing on the multi-dimensional feature data of the twisted structure according to the twisted structure curvature data to obtain twisted structure curvature hierarchical data;

[0053] Step S343: calculating the layered light distortion index according to the layered data of the curvature of the distortion structure to obtain the layered light distortion index data;

[0054] Step S344: performing light dynamic parameter correction according to the layered light distortion index data to obtain layered light dynamic parameter correction data;

[0055] Step S345: performing light distortion behavior correction processing according to the layered light dynamic parameter correction data to obtain light distortion behavior correction data.

[0056] The present invention obtains the curvature data of each distorted structure by analyzing and calculating the multidimensional feature data of the distorted structure. These data reflect the specific form and curvature degree of the light distortion. A more detailed description of the light distortion behavior is provided to help identify the type and degree of distortion that needs to be corrected. Based on the distorted structure curvature data, hierarchical feature processing is performed to classify and analyze the multidimensional feature data according to different levels of distortion curvature. A specific analysis for different levels of distortion curvature is provided to help distinguish and understand the degree of influence of various distortion phenomena on the optical system. Based on the distorted structure curvature hierarchical data, the light distortion index of each level, such as distortion intensity, influence range, etc., is calculated. A more in-depth quantitative analysis of the light distortion behavior is provided to provide specific guidance and optimization direction for subsequent correction. According to the hierarchical light distortion index data, a dynamic parameter correction strategy for the optical system is formulated, involving operations such as lens adjustment and light source position change. By adjusting the parameters of the optical system in real time, the light distortion caused by the distortion is reduced or eliminated, and the accuracy and stability of the optical system are improved. The hierarchical light dynamic parameter correction data is applied to perform actual light distortion behavior correction processing to ensure that the system output meets the expected optical performance. Ultimately, effective management and optimization of light distortion behavior in optical systems was achieved, improving the system's functionality and application effects.

[0057] Preferably, the present invention further provides a laser radar camera co-location perception system, which is used to execute the laser radar camera co-location perception method as described above, and the laser radar camera co-location perception system comprises:

[0058] A perspective 3D model construction module is used to extract real-time ambient light data through an onboard camera lens to obtain real-time ambient light data; construct a mobile collaborative perspective 3D model based on the real-time ambient light data to obtain a mobile collaborative perspective 3D model;

[0059] The environment space distortion behavior probability calculation module is used to evaluate the light reflection intensity of the real-time environment light data based on the mobile collaborative perspective three-dimensional model to obtain the light reflection intensity evaluation data; perform structural interference effect analysis on the light reflection intensity evaluation data to obtain the light reflection structural interference effect data; perform environment space distortion behavior probability calculation based on the light reflection structural interference effect data to obtain the environment space distortion behavior probability data;

[0060] The correction processing module is used to perform spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data; perform light distortion behavior correction processing according to the spatial distortion structured data to obtain light distortion behavior correction data;

[0061] The co-location perception model building module is used to perform policy gradient learning on the light distortion behavior correction data based on the policy gradient algorithm to obtain the light correction policy gradient data; build the light co-location perception model according to the light correction policy gradient data to obtain the light co-location perception model, and send the light co-location perception model to the data cloud platform to execute the lidar camera co-location perception method.

[0062] The present invention also provides a computer program, which, when executed, implements the laser radar camera co-location perception method as described in any one of the above.

[0063] The beneficial effect of the present invention is to use an airborne camera lens to capture light information in the environment in real time, ensuring the timeliness and accuracy of the data. A three-dimensional model is constructed through real-time data, and the model can reflect the real situation of the environment, including the distribution and influence of light. Based on the mobile collaborative perspective three-dimensional model, the real-time environmental light data is evaluated to quantify the intensity and characteristics of light reflection. By evaluating the data, the structural interference effect of light in the environment is analyzed, that is, the complex interactive influence of light reflection, refraction and diffraction on the surface of the object. According to the light reflection structural interference effect data, the probability of spatial distortion behavior occurring in the environment is calculated. These data are helpful to understand and predict the behavior of light in a specific environment, including its impact on visual perception and object shape cognition. By analyzing the probability data of environmental spatial distortion behavior, specific spatial distortion structured features, such as bending, twisting or deformation of light, are identified. Based on the identified distortion features, correction processing of light distortion behavior is performed to improve the accuracy of light transmission and the accuracy of environmental perception. The light distortion behavior correction data is learned and optimized using a policy gradient algorithm to find the optimal correction strategy. Optimized light correction strategy gradient data are obtained, which reflect how to adjust light transmission according to real-time environmental data to reduce distortion effects. Based on the results of policy gradient learning, a light co-location perception model is constructed. This model can adjust the position and angle of the laser radar and camera according to the actual light conditions in a real-time or near-real-time environment to achieve more accurate co-location perception. The optimized light co-location perception model is sent to the data cloud platform to execute the co-location perception method of the laser radar and camera in a wide range of application scenarios. By correcting the light distortion behavior and applying the light co-location perception model, the accuracy of environmental perception and spatial perception is improved, and errors and uncertainties are reduced. Therefore, the present invention is an improved processing of a traditional laser radar camera co-location perception method, which solves the problem that a traditional laser radar camera co-location perception method has an inaccurate analysis of ambient light changes, resulting in low accuracy of environmental distortion correction during the reflection process, improves the accuracy of ambient light change analysis, and improves the accuracy of environmental distortion correction during the reflection process. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of the steps of a laser radar camera co-location perception method;

[0065] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;

[0066] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0067] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0068] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0069] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0070] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0071] To achieve this, please refer to Figures 1 to 3 , a laser radar camera co-location perception method, the method comprising the following steps:

[0072] Step S1: extracting real-time ambient light data through an onboard camera lens to obtain real-time ambient light data; constructing a mobile collaborative perspective three-dimensional model based on the real-time ambient light data to obtain a mobile collaborative perspective three-dimensional model;

[0073] Step S2: Based on the mobile collaborative perspective three-dimensional model, the real-time environmental light data is evaluated for light reflection intensity to obtain light reflection intensity evaluation data; the light reflection intensity evaluation data is analyzed for structural interference effect to obtain light reflection structural interference effect data; the probability of environmental space distortion behavior is calculated based on the light reflection structural interference effect data to obtain environmental space distortion behavior probability data;

[0074] Step S3: performing spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data; performing light distortion behavior correction processing according to the spatial distortion structured data to obtain light distortion behavior correction data;

[0075] Step S4: Based on the policy gradient algorithm, policy gradient learning is performed on the light distortion behavior correction data to obtain light correction policy gradient data; a light co-location perception model is constructed according to the light correction policy gradient data to obtain the light co-location perception model, and the light co-location perception model is sent to the data cloud platform to execute the lidar camera co-location perception method.

[0076] In the embodiment of the present invention, reference Figure 1 The above is a schematic flow chart of the steps of a laser radar camera co-location perception method of the present invention. In this example, the laser radar camera co-location perception method includes the following steps:

[0077] Step S1: extracting real-time ambient light data through an onboard camera lens to obtain real-time ambient light data; constructing a mobile collaborative perspective three-dimensional model based on the real-time ambient light data to obtain a mobile collaborative perspective three-dimensional model;

[0078] In an embodiment of the present invention, the ambient light data is captured in real time by the airborne camera lens on the sweeping robot. The airborne camera lens is installed on the drone and is equipped with a high dynamic range imaging sensor to ensure that the ambient light can be accurately captured under different lighting conditions. The real-time captured image is subjected to noise filtering and image enhancement by a preprocessing algorithm to reduce the impact of ambient light changes on the data. Next, based on these image data, a mobile collaborative perspective three-dimensional model is constructed using multi-perspective stereo vision technology. The specific process includes: matching feature points of image data acquired from different angles, generating a depth map using a parallax calculation method, and then synthesizing a three-dimensional model of the environment through a three-dimensional reconstruction algorithm. Ultimately, the resulting three-dimensional model can dynamically reflect the actual shape and lighting changes of the environment, providing an accurate spatial data basis for subsequent analysis.

[0079] Step S2: Based on the mobile collaborative perspective three-dimensional model, the real-time environmental light data is evaluated for light reflection intensity to obtain light reflection intensity evaluation data; the light reflection intensity evaluation data is analyzed for structural interference effect to obtain light reflection structural interference effect data; the probability of environmental space distortion behavior is calculated based on the light reflection structural interference effect data to obtain environmental space distortion behavior probability data;

[0080] In an embodiment of the present invention, after obtaining the mobile collaborative perspective three-dimensional model, the real-time environmental light data is evaluated for light reflection intensity by ray tracing technology. A high-precision ray tracing algorithm is used to calculate the reflection intensity of each surface point of the three-dimensional model. This process takes into account the position of the light source, the surface material properties, and the scattering effect of the ambient light. The light reflection intensity evaluation results are further processed to analyze their changing trends under different lighting conditions to generate light reflection intensity evaluation data. Based on this data, the interference effect analysis of the light reflection structure is performed. The reflection data is processed using a structural interference effect model to identify and quantify the interference patterns in the reflection image, such as spot and stripe effects, to obtain light reflection structure interference effect data. Further, using these interference effect data, an environmental space distortion behavior probability calculation model is applied. The model comprehensively considers the reflection intensity and interference effect of the ambient light, and through probabilistic statistical analysis, calculates the probability of spatial distortion behavior caused by structural interference when the light propagates in the environment, and finally obtains the environmental space distortion behavior probability data. This data can be used to analyze the light propagation characteristics in complex environments and provide support for the co-location perception of lidar and cameras.

[0081] Step S3: performing spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data; performing light distortion behavior correction processing according to the spatial distortion structured data to obtain light distortion behavior correction data;

[0082] In an embodiment of the present invention, spatial distortion structured identification is performed on the probability data of the environmental space distortion behavior. Using a spatial distortion identification algorithm, the probability data is first mapped into a three-dimensional space, and the data is gridded by a high-resolution grid generation algorithm to accurately divide the distorted area in the space. Next, a spatial segmentation analysis method is applied to identify the specific shape and distribution characteristics of the distorted area. The structure of the distorted area is described in detail by a spatial analysis tool, and a structured data set is generated. These data sets contain the geometric characteristics, distribution patterns and positions of the distorted area in three-dimensional space. Based on this structured data, light distortion behavior correction processing is performed. Using a light propagation model, the distorted area is analyzed by an inverse modeling method, and the propagation path of the light in these areas is calculated and corrected. The correction algorithm includes steps such as light refractive index adjustment and path correction to obtain light distortion behavior correction data, which can effectively compensate for the changes in the light in the distorted area and ensure the accuracy of the subsequent model.

[0083] Step S4: Based on the policy gradient algorithm, policy gradient learning is performed on the light distortion behavior correction data to obtain light correction policy gradient data; a light co-location perception model is constructed according to the light correction policy gradient data to obtain the light co-location perception model, and the light co-location perception model is sent to the data cloud platform to execute the lidar camera co-location perception method.

[0084] In an embodiment of the present invention, a policy gradient algorithm is applied to perform policy gradient learning based on the light distortion behavior correction data. The specific steps include: first, a policy network is established, which accepts the light distortion behavior correction data as input and processes it through a multi-layer neural network. The policy gradient method is used to optimize the network parameters, and the gradient is calculated and the network weights are adjusted by comparing the difference between the actual correction effect and the expected effect. This process will gradually improve the strategy in multiple iterations, so that the light correction effect is continuously optimized. The obtained light correction policy gradient data contains the best correction strategy and adjustment parameters. Based on these data, a light co-location perception model is constructed. A comprehensive light co-location perception model is constructed using the correction strategy gradient data. The model integrates the measurement data of the laser radar and the camera, and uses the correction strategy for real-time data fusion. The constructed light co-location perception model is uploaded to the data cloud platform, and the cloud platform is used to execute the laser radar camera co-location perception method to achieve real-time environmental perception and analysis.

[0085] Preferably, step S1 comprises the following steps:

[0086] Step S11: collecting real-time environmental data through an onboard camera lens to obtain real-time environmental data;

[0087] Step S12: extracting real-time ambient light data from the real-time ambient data to obtain real-time ambient light data;

[0088] Step S13: Obstacle real-time ranging is performed based on real-time environmental data by using a laser radar to obtain obstacle real-time ranging data;

[0089] Step S14: construct a mobile collaborative perspective three-dimensional model for the real-time environmental data according to the real-time environmental light data and the real-time obstacle ranging data to obtain a mobile collaborative perspective three-dimensional model.

[0090] In the embodiment of the present invention, the ambient light data is captured in real time by the onboard camera lens on the sweeping robot, and a high-resolution image sensor and a real-time image processing module are configured. The lens is stabilized by an automated control system to ensure the stability and clarity of image capture. The image sensor regularly collects environmental image data, which includes information such as objects, obstacles, and light conditions in the environment. The collected image data is processed in real time to remove noise and correct colors to improve data quality. Finally, the obtained real-time environmental data includes multiple high-resolution image sequences, and real-time environmental light data is extracted from the real-time environmental data. First, an image frame is extracted from the real-time environmental data obtained in step S11. High dynamic range image processing technology is applied to analyze the light information in the image in detail, identify and measure the intensity and direction of the light. The image is analyzed using a ray tracing algorithm to extract the light reflection information of each pixel. Subsequently, the light data is processed for spectral resolution by a spectral analysis method to extract the light intensity data of each band in the environment. Finally, the obtained real-time environmental light data includes the light intensity and distribution information of each spectral band, which can reflect the lighting conditions and changes of the current environment. The laser radar emits laser pulses and receives laser signals reflected from obstacles to calculate the distance from the obstacle to the laser radar. In order to improve the ranging accuracy, the laser radar performs multiple measurements at different angles and positions, and uses sensor fusion technology to weighted average the multiple measurement results. Combined with the image information captured in the real-time environmental data, the obstacle real-time ranging data provides the accurate location and distance information of obstacles in the environment. The real-time environmental data is constructed based on the real-time environmental light data and the obstacle real-time ranging data. First, the real-time environmental light data and the obstacle real-time ranging data are fused, and the depth information in the environment is processed by the stereo vision algorithm. Using image matching technology, the multi-view images captured by the camera lens are integrated with the laser radar ranging data to generate a preliminary three-dimensional point cloud model. Then, the three-dimensional reconstruction algorithm is applied to process the point cloud data to form a complete three-dimensional environmental model. Furthermore, the three-dimensional model is simulated by ray tracing technology, and the lighting effect of the model is adjusted in combination with the light data to ensure that the model reflects the light conditions and obstacle positions in the real environment. Ultimately, the resulting mobile collaborative perspective 3D model can accurately represent the 3D structure of the environment and its lighting characteristics, providing basic data for subsequent analysis.

[0091] Preferably, step S2 comprises the following steps:

[0092] Step S21: performing light change fluctuation analysis on the real-time ambient light data based on the mobile collaborative perspective three-dimensional model to obtain light change fluctuation data;

[0093] Step S22: evaluating the light reflection intensity of the light change fluctuation data to obtain light reflection intensity evaluation data;

[0094] Step S23: performing structural interference effect analysis on the light reflection intensity evaluation data to obtain light reflection structural interference effect data;

[0095] Step S24: Calculate the probability of the environmental space distortion behavior according to the light reflection structure interference effect data and the light reflection intensity evaluation data to obtain the environmental space distortion behavior probability data.

[0096] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:

[0097] Step S21: performing light change fluctuation analysis on the real-time ambient light data based on the mobile collaborative perspective three-dimensional model to obtain light change fluctuation data;

[0098] In an embodiment of the present invention, a light change fluctuation analysis is performed on real-time ambient light data based on a mobile collaborative perspective three-dimensional model. First, the mobile collaborative perspective three-dimensional model obtained in step S14 and the real-time ambient light data are aligned to ensure the spatial consistency of the three-dimensional model and the light data. Using a light propagation simulation algorithm, a light change fluctuation analysis is performed on each surface point in the model to measure the intensity changes of light at different time points and different positions. During the analysis process, a local light fluctuation calculation method is used to perform a Fourier transform on the time series data of light intensity to extract the frequency components and amplitude changes of light fluctuations. Finally, light change fluctuation data is obtained, which contains the fluctuation characteristic information of light intensity in space and time for subsequent evaluation and analysis.

[0099] Step S22: evaluating the light reflection intensity of the light change fluctuation data to obtain light reflection intensity evaluation data;

[0100] In an embodiment of the present invention, the light reflection intensity is evaluated for the light change fluctuation data. First, the intensity fluctuation characteristics of each light are extracted from the light change fluctuation data obtained in step S21. Using the reflection intensity evaluation algorithm, the reflection intensity of the light at different angles and surfaces is calculated through statistical analysis of the light fluctuation data. The specific process includes: calculating the local mean and standard deviation of the light fluctuation data to evaluate the intensity stability of the reflected light. Then, combined with the material properties of the surface in the three-dimensional model, the reflection intensity of the light on each surface is calculated, and the corresponding reflection intensity data is generated. Finally, the light reflection intensity evaluation data is obtained, which reflects the reflection characteristics and change laws of the light under different environmental conditions.

[0101] Step S23: performing structural interference effect analysis on the light reflection intensity evaluation data to obtain light reflection structural interference effect data;

[0102] In an embodiment of the present invention, a structural interference effect analysis is performed on the light reflection intensity evaluation data. First, based on the light reflection intensity evaluation data obtained in step S22, an interference effect analysis algorithm is applied to evaluate the light reflection pattern. The specific steps include: using an interference fringe detection method to identify the pattern of light interference fringes by analyzing the periodic changes in the reflection intensity data. Then, in combination with the spatial frequency analysis technology, the spatial distribution of the interference fringes is analyzed in detail to calculate the intensity and phase information of the interference effect. Further, the data is modeled through an interference effect model to quantify the interference effect in light reflection, including structural effects such as light spots and light bands. Finally, light reflection structural interference effect data is generated to describe the characteristics and distribution of the interference effect that occurs during the light reflection process.

[0103] Step S24: Calculate the probability of the environmental space distortion behavior according to the light reflection structure interference effect data and the light reflection intensity evaluation data to obtain the environmental space distortion behavior probability data.

[0104] In an embodiment of the present invention, the probability of environmental space distortion behavior is calculated based on the light reflection structure interference effect data and the light reflection intensity evaluation data. First, the light reflection structure interference effect data obtained in step S23 is fused with the light reflection intensity evaluation data in step S22. Using the environmental space distortion calculation model, the light reflection intensity and the interference effect are used as input to simulate the spatial distortion caused by the interference effect when the light propagates in the environment. The specific operation includes: constructing a light propagation model of the environmental space based on the light reflection data, applying the spatial distortion probability calculation algorithm, and quantifying the distortion probability in the light propagation path. This algorithm combines statistical methods to conduct a comprehensive analysis of the interference effect and light intensity changes, and calculates the probability of spatial distortion of light in the environment. Finally, the environmental space distortion behavior probability data is generated, which provides the possibility and distribution of light distortion in the environment.

[0105] Preferably, step S23 includes the following steps:

[0106] Step S231: marking the light reflection path of the light reflection intensity evaluation data to obtain light reflection path marking data;

[0107] Step S232: performing light reflection interlaced structure analysis according to the light reflection path mark data to obtain light reflection interlaced structure data;

[0108] Step S233: evaluating the color difference between different reflected lights on the light reflection interlaced structure data according to the light reflection intensity evaluation data, to obtain the color difference data of the reflected light;

[0109] Step S234: calculating the light wave phase difference between different reflected light rays for the light reflection interlaced structure data according to the reflected light chromatic aberration data and the light reflection intensity evaluation data, to obtain the reflected interlaced structure light wave phase difference data;

[0110] Step S235: Perform structural interference effect analysis based on the phase difference data of the reflected interlaced structure light wave to obtain light reflection structure interference effect data.

[0111] In an embodiment of the present invention, the light reflection path is marked for the light reflection intensity evaluation data. First, the path information of the light is extracted from the light reflection intensity evaluation data obtained in step S22. The propagation path of the light in the three-dimensional environment model is marked by a ray tracing algorithm, and the incident point, reflection point and propagation path of each light are recorded. The path tracing technology is applied to each light to ensure the accuracy of the path marking. The marking data is stored in a data structure, including information such as the reflection point, path length and reflection angle of the light. Finally, the light reflection path marking data is generated, which describes in detail the propagation path of the light and its interaction with the surface of the environment. The light reflection interlaced structure analysis is performed according to the light reflection path marking data. The light interlaced pattern analysis method is used to perform an interlaced structure analysis on the light path in the marking data. First, the interlaced area in the path marking data is identified by the light interlaced detection algorithm. The interlaced structure analysis includes high-resolution imaging of the interlaced points of the reflection path, and extracting the spatial features of the interlaced area using image processing technology. Combined with the reflection angle and path length of the light, the interaction pattern of the light in the interlaced structure is analyzed. The light reflection interlaced structure data is generated to describe the structural characteristics and distribution of the light in the interlaced area. The light reflection interlaced structure data is evaluated for the color difference between different reflected lights according to the light reflection intensity evaluation data. First, the color difference calculation algorithm is used to analyze the reflection intensity data of each light in the light reflection interlaced structure data. The color space conversion technology is applied to convert the light intensity data into the color difference value in the color space. The spectral resolution analysis method is used to evaluate the color difference between different lights, and the absolute and relative values ​​of the color difference are calculated. Finally, the obtained color difference data of the reflected light provides the color difference of different lights in the interlaced area. The light wave phase difference between different reflected lights is calculated for the light reflection interlaced structure data according to the color difference data of the reflected light and the light reflection intensity evaluation data. First, the light wave phase measurement technology is used to process the light reflection intensity data and calculate the light wave phase information of the light. The interference measurement technology is used to calculate the phase difference between the reflected lights by analyzing the color difference data. The phase calculation algorithm is used to quantitatively analyze the phase difference of the light in the interlaced structure, and the phase difference data of different reflected lights are integrated. Finally, the generated reflected interlaced structure light wave phase difference data reflects the phase difference of the light in the interlaced area. Structural interference effect analysis is performed based on the reflected interlaced structure light wave phase difference data. First, the phase difference information is extracted from the light wave phase difference data obtained in step S234. Using the interference effect model, the phase difference data is subjected to interference fringe analysis to determine the interference pattern of the light in the interlaced area. The intensity and distribution of the interference fringes are calculated by the light wave interference algorithm. The interference effect quantification method is applied to analyze the interference effect intensity and spatial distribution of the light, and generate the light reflection structure interference effect data.The final data describes the characteristics of the interference effect caused by the phase difference of light waves in the staggered structure.

[0112] Preferably, step S24 comprises the following steps:

[0113] Step S241: calculating the optical path difference of different light reflections on the light reflection structure interference effect data to obtain the light reflection optical path difference data;

[0114] Step S242: performing medium reflection interference simulation on the light reflection structure interference effect data according to the light reflection optical path difference data to obtain medium reflection interference simulation data;

[0115] Step S243: performing polarization light interaction analysis on the light reflection structure interference effect data according to the medium reflection interference simulation data to obtain reflection structure polarization light interaction data;

[0116] Step S244: performing light distortion critical state analysis based on the reflection structure polarization light interaction data, the medium reflection interference simulation data, and the light reflection optical path difference data to obtain light distortion critical state data;

[0117] Step S245: performing regression interval estimation on the light distortion critical state data according to the light reflection intensity evaluation data to obtain light reflection distortion interval estimation data;

[0118] Step S246: Calculate the probability of environmental space distortion behavior based on the light reflection distortion interval estimation data to obtain environmental space distortion behavior probability data.

[0119] In an embodiment of the present invention, the optical path difference of different light reflections is calculated for the light reflection structure interference effect data. First, the interference fringe information of the reflected light is extracted from the light reflection structure interference effect data obtained in step S235. The reflection paths of different light rays are compared using an optical path difference calculation algorithm. The specific operation includes: calculating the optical path difference of each light ray, that is, the optical path difference from the incident point to the reflection point, according to the reflection point and path recorded in the light reflection path mark data. By comparing the optical path differences of different reflected light rays, light reflection optical path difference data is generated, which describes the optical path difference caused by the path difference of the light during the interference process. According to the light reflection optical path difference data, medium reflection interference simulation is performed on the light reflection structure interference effect data. First, the optical path difference information is extracted from the light reflection optical path difference data obtained in step S241. Using the medium reflection interference simulation model, the light reflection optical path difference data is input into the model to simulate the interference effect of light passing through different media. Using the interference simulation tool, the reflection and transmission of light on different medium interfaces are calculated to simulate its interference pattern and intensity distribution. Finally, medium reflection interference simulation data is generated to describe the reflection interference of light on various medium interfaces. Polarized light interaction analysis is performed on the light reflection structure interference effect data according to the medium reflection interference simulation data. First, the interaction information of polarized light is extracted from the medium reflection interference simulation data obtained in step S242. Polarized light analysis tools are used to analyze the change of polarization state of light on the medium interface. The specific operation includes: measuring the polarization degree and polarization angle of light, and using a polarization interferometer to record the polarization characteristics of light under the interaction of different media. These data are comprehensively analyzed with the light reflection structure interference effect data to generate reflection structure polarized light interaction data, which shows the polarization interaction characteristics of light in different media. Based on the reflection structure polarized light interaction data, medium reflection interference simulation data and light reflection optical path difference data, the critical state of light distortion is analyzed. First, the reflection structure polarized light interaction data of step S243, the medium reflection interference simulation data of step S242, and the light reflection optical path difference data of step S241 are integrated. The light distortion analysis model is used to comprehensively analyze these data and calculate the critical state of distortion of light under the influence of interference and polarization effects. The specific operation includes: according to the reflection and distortion characteristics of light, the critical conditions of the distortion state are analyzed using optical simulation tools. Finally, the light distortion critical state data is obtained to describe the distortion of light under different interference and polarization conditions. The light distortion critical state data is regressed and estimated based on the light reflection intensity evaluation data. First, the light intensity information is extracted from the light reflection intensity evaluation data obtained in step S22. Combined with the light distortion critical state data obtained in step S244, the distortion state is estimated using a regression analysis method.The specific operations include: performing regression modeling on the distortion critical state data, calculating the influence of light intensity on the distortion state, and estimating the range of light distortion. Generate light reflection distortion interval estimation data, which describes the degree of distortion under different light intensity conditions. Perform environmental space distortion behavior probability calculation based on the light reflection distortion interval estimation data. First, extract the distortion interval information from the light reflection distortion interval estimation data obtained in step S245. Combined with the light reflection intensity evaluation data and the light distortion critical state data in step S24, the probability calculation model is used to calculate the probability of light distortion in the environment. The specific operations include: performing probability distribution analysis on the light distortion interval, and calculating the probability distribution of light distortion in combination with the light propagation characteristics in the environmental model. Finally, environmental space distortion behavior probability data is generated, providing probability information of light distortion in the environment.

[0120] Preferably, performing regression interval estimation on the light distortion critical state data according to the light reflection intensity evaluation data comprises the following steps:

[0121] According to the light reflection intensity evaluation data, the light distortion critical state data is subjected to light distortion increasing law identification to obtain light distortion increasing law data;

[0122] Perform incremental stepwise regression analysis on the data of increasing law of light distortion to obtain the incremental stepwise regression data of distortion;

[0123] According to the distortion increasing stepwise regression data, multicollinearity constraint processing is performed on the light distortion increasing law data to obtain the collinear constraint data of the light distortion increasing law;

[0124] According to the collinear constraint data of the increasing law of light distortion, regression analysis is performed on the data of the increasing law of light distortion to obtain the collinear regression data of the increasing law of light distortion;

[0125] The regression interval is estimated based on the light distortion incremental collinear regression data to obtain the light reflection distortion interval estimation data.

[0126] In an embodiment of the present invention, the light distortion critical state data is identified according to the light reflection intensity evaluation data. First, the change records of light intensity and distortion degree are extracted from the light reflection intensity evaluation data. The curve fitting technology is used to model the change trend of light distortion degree with light intensity. In the specific implementation, the light intensity is used as the independent variable and the light distortion degree is used as the dependent variable. The increasing trend analysis algorithm is applied to identify the increasing law of distortion, and the increasing law data of light distortion is obtained. The data describes in detail the increasing trend of the degree of distortion under different light intensities. Incremental stepwise regression analysis is performed on the increasing law data of light distortion. The obtained increasing law data of light distortion is imported into the regression analysis tool for stepwise regression modeling. In the stepwise regression process, starting from the simplest model, new independent variables are gradually added to evaluate the contribution of each step to the model fitting. By adjusting and optimizing the regression model, the influence degree of each variable on the increasing law of light distortion is determined, and the increasing stepwise regression data of distortion is obtained, which describes the specific influence of light intensity on the increasing distortion. The increasing law data of light distortion is subjected to multicollinearity constraint processing according to the increasing stepwise regression data of distortion. First, the collinearity diagnostic tool is used to evaluate the correlation between the independent variables in the regression model and calculate the variance inflation factor (VIF). The independent variables with serious collinearity are screened and adjusted. The combination of independent variables in the model is adjusted through the multicollinearity constraint algorithm to reduce the correlation between the independent variables. After the processing is completed, the collinear constraint data of the light distortion increase law is obtained, which reflects the distortion increase law after collinearity constraint. The light distortion increase law data is regressed according to the collinear constraint data of the light distortion increase law. The obtained collinear constraint data is applied to the regression model for standard regression analysis. A new regression model is established according to the collinear constraint data to evaluate the relationship between light intensity and distortion degree. The light distortion increase collinear regression data is generated, which describes the specific regression results of light intensity on distortion increase in the model after collinearity processing. The regression interval is estimated according to the light distortion increase collinear regression data. The regression analysis results obtained in step S245.4 are used in the interval estimation model to calculate the confidence interval of the light distortion degree. The specific operation includes: calculating the regression coefficient confidence interval of each interval through the coefficient and standard error in the regression model. Using the interval estimation tool, the estimation results are organized into light reflection distortion interval estimation data. This data provides the confidence interval of the degree of light distortion under different light intensity conditions, showing the range of distortion.

[0127] Preferably, step S3 comprises the following steps:

[0128] Step S31: performing spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data;

[0129] Step S32: extracting features from the spatial distortion structured data to obtain spatial distortion structure feature data;

[0130] Step S33: performing a distortion structure multi-dimensional analysis on the spatial distortion structure data according to the spatial distortion structure feature data to obtain the distortion structure multi-dimensional feature data;

[0131] Step S34: performing light distortion behavior correction processing according to the distortion structure multi-dimensional feature data to obtain light distortion behavior correction data.

[0132] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:

[0133] Step S31: performing spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data;

[0134] In an embodiment of the present invention, spatial distortion structured identification is performed on the probability data of environmental spatial distortion behavior. First, the probability data of environmental spatial distortion behavior is partitioned according to a preset spatial grid, and each grid represents a spatial unit. A spatial data analysis tool is used to identify and classify the distortion behavior of each spatial unit, and the distortion pattern in the data is marked and structured by setting thresholds and classification rules. Through a spatial mapping algorithm, regions with similar distortion characteristics are identified, and these regions are grouped and labeled to generate spatial distortion structured data. This data describes the regional distribution and structural characteristics of the distortion behavior in space.

[0135] Step S32: extracting features from the spatial distortion structured data to obtain spatial distortion structure feature data;

[0136] In an embodiment of the present invention, feature extraction is performed on the spatial distortion structured data. A feature extraction algorithm is used to extract geometric features of each marked spatial unit, including distortion degree, shape features, boundary information, etc. First, the geometric properties of each spatial unit are calculated, such as distortion angle, area change, edge curvature, etc. Then, these geometric properties are quantified into feature vectors to form spatial distortion structure feature data. By using feature engineering tools, the extracted data will describe the specific geometric features of the spatial distortion, which is helpful for subsequent analysis.

[0137] Step S33: performing a distortion structure multi-dimensional analysis on the spatial distortion structure data according to the spatial distortion structure feature data to obtain the distortion structure multi-dimensional feature data;

[0138] In the embodiment of the present invention, a multidimensional analysis of the distortion structure is performed on the spatial distortion structured data according to the spatial distortion structure feature data. A multidimensional data analysis method is used to perform an in-depth analysis on the extracted feature data. The feature data is reduced in dimension using principal component analysis (PCA) or factor analysis methods to convert the multidimensional features into main components or factors. By constructing a multidimensional space model, the main influencing factors and correlations of the spatial distortion are analyzed to obtain the multidimensional feature data of the distortion structure. This data reveals the complexity and main driving factors of the distortion behavior in the multidimensional space.

[0139] Step S34: performing light distortion behavior correction processing according to the distortion structure multi-dimensional feature data to obtain light distortion behavior correction data.

[0140] In the embodiment of the present invention, the light distortion behavior correction processing is performed according to the multi-dimensional feature data of the distortion structure. First, the light distortion behavior model is constructed using the multi-dimensional feature data of the distortion structure obtained in step S33 to describe the change law of the light in the distortion environment. Then, the light distortion behavior is adjusted by the correction algorithm. The correction algorithm applies the distortion parameters in the multi-dimensional feature data to the light model, and corrects the light in the distortion environment through calculation and adjustment. Finally, the light distortion behavior correction data is generated, which is used to correct the distortion effect of the original light in the actual environment, thereby improving the co-location perception accuracy of the laser radar and the camera.

[0141] Preferably, step S34 includes the following steps:

[0142] Step S341: calculating the curvature of the twisted structure multi-dimensional feature data to obtain the curvature data of the twisted structure;

[0143] Step S342: performing hierarchical feature processing on the multi-dimensional feature data of the twisted structure according to the twisted structure curvature data to obtain twisted structure curvature hierarchical data;

[0144] Step S343: calculating the layered light distortion index according to the layered data of the curvature of the distortion structure to obtain the layered light distortion index data;

[0145] Step S344: performing light dynamic parameter correction according to the layered light distortion index data to obtain layered light dynamic parameter correction data;

[0146] Step S345: performing light distortion behavior correction processing according to the layered light dynamic parameter correction data to obtain light distortion behavior correction data.

[0147] In an embodiment of the present invention, the multidimensional characteristic data of the distorted structure is subjected to the calculation of the distortion curvature. First, the multidimensional characteristic data of the distorted structure is input into the curvature analysis tool, and the curvature degree of the light distorted area is obtained by calculating the local curvature and global curvature of each data point. The curvature calculation uses the curvature formula in differential geometry, which includes solving the second-order derivative of each spatial point to determine its curvature in the multidimensional space. These calculation results are organized into the curvature data of the distorted structure, providing a specific quantitative index of the bending of the light in the distorted environment. The multidimensional characteristic data of the distorted structure is subjected to hierarchical feature processing according to the curvature data of the distorted structure. The curvature data is subjected to hierarchical processing using a hierarchical algorithm. First, the data is divided into multiple levels according to the size of the curvature value, such as mild distortion, moderate distortion and severe distortion. Then, the statistical characteristics of each level, such as the average curvature and the standard deviation, are calculated for each level. These hierarchical characteristics reflect the distribution of different distortion degrees, and the obtained hierarchical data of the curvature of the distorted structure are used for subsequent analysis. The hierarchical light distortion index is calculated according to the hierarchical data of the curvature of the distorted structure. For each level of data, its distortion index is calculated using the light distortion model. The specific method includes modeling each layer of light to determine the degree of distortion of light in different curvature levels. The calculation includes analyzing the optical behaviors of light such as refraction, reflection and scattering in a specific distortion level, so as to obtain the light distortion index data of each level, which describes the change of light under different distortion levels. The light dynamic parameters are corrected according to the layered light distortion index data. First, the layered light distortion index data is input into the light dynamic correction algorithm. The algorithm adjusts the light dynamic parameters, including light intensity, wavelength and phase, by comparing the actual measured light data with the light data predicted by the model. These parameters are adjusted by optimization technology so that the performance of light in the actual environment is consistent with expectations. The correction process includes linear and nonlinear optimization steps, and the obtained layered light dynamic parameter correction data is used for further light correction. The light distortion behavior correction processing is performed according to the layered light dynamic parameter correction data. The corrected light dynamic parameters are applied to the light distortion behavior model, and the distortion effect of light in the real environment is corrected by reverse simulation and adjustment of the correction data. This process involves applying the layered light dynamic parameters to the actual lidar and camera data, and using the correction algorithm to make detailed adjustments to the light distortion behavior. The resulting light distortion behavior correction data is used to improve the perception accuracy of lidar and camera systems, ensuring the accuracy and reliability of measurement results.

[0148] Preferably, the present invention further provides a laser radar camera co-location perception system, which is used to execute the laser radar camera co-location perception method as described above, and the laser radar camera co-location perception system comprises:

[0149] A perspective 3D model construction module is used to extract real-time ambient light data through an onboard camera lens to obtain real-time ambient light data; construct a mobile collaborative perspective 3D model based on the real-time ambient light data to obtain a mobile collaborative perspective 3D model;

[0150] The environment space distortion behavior probability calculation module is used to evaluate the light reflection intensity of the real-time environment light data based on the mobile collaborative perspective three-dimensional model to obtain the light reflection intensity evaluation data; perform structural interference effect analysis on the light reflection intensity evaluation data to obtain the light reflection structural interference effect data; perform environment space distortion behavior probability calculation based on the light reflection structural interference effect data to obtain the environment space distortion behavior probability data;

[0151] The correction processing module is used to perform spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data; perform light distortion behavior correction processing according to the spatial distortion structured data to obtain light distortion behavior correction data;

[0152] The co-location perception model building module is used to perform policy gradient learning on the light distortion behavior correction data based on the policy gradient algorithm to obtain the light correction policy gradient data; build the light co-location perception model according to the light correction policy gradient data to obtain the light co-location perception model, and send the light co-location perception model to the data cloud platform to execute the lidar camera co-location perception method.

[0153] The present invention also provides a computer program, which, when executed, implements the laser radar camera co-location perception method as described in any one of the above.

[0154] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0155] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A laser radar camera co-location perception method, characterized in that: The following steps are involved: Step S1: extracting real-time ambient light data through an onboard camera lens to obtain real-time ambient light data; The mobile collaborative perspective three-dimensional model is constructed according to the real-time ambient light data to obtain the mobile collaborative perspective three-dimensional model; wherein step S1 includes: Step S11: collecting real-time environmental data through an onboard camera lens to obtain real-time environmental data; Step S12: extracting real-time ambient light data from the real-time ambient data to obtain real-time ambient light data; Step S13: Obstacle real-time ranging is performed based on real-time environmental data by using a laser radar to obtain obstacle real-time ranging data; Step S14: constructing a mobile collaborative perspective three-dimensional model for the real-time environmental data according to the real-time environmental light data and the real-time obstacle ranging data to obtain a mobile collaborative perspective three-dimensional model; Step S2: Based on the mobile collaborative perspective three-dimensional model, the real-time environmental light data is evaluated for light reflection intensity to obtain light reflection intensity evaluation data; the light reflection intensity evaluation data is analyzed for structural interference effect to obtain light reflection structural interference effect data; the probability of environmental space distortion behavior is calculated based on the light reflection structural interference effect data to obtain environmental space distortion behavior probability data; wherein step S2 includes: Step S21: performing light change fluctuation analysis on the real-time ambient light data based on the mobile collaborative perspective three-dimensional model to obtain light change fluctuation data; Step S22: evaluating the light reflection intensity of the light change fluctuation data to obtain light reflection intensity evaluation data; Step S23: performing structural interference effect analysis on the light reflection intensity evaluation data to obtain light reflection structural interference effect data; wherein step S23 includes: Step S231: marking the light reflection path of the light reflection intensity evaluation data to obtain light reflection path marking data; Step S232: performing light reflection interlaced structure analysis according to the light reflection path mark data to obtain light reflection interlaced structure data; Step S233: evaluating the color difference between different reflected lights on the light reflection interlaced structure data according to the light reflection intensity evaluation data, to obtain the color difference data of the reflected light; Step S234: calculating the light wave phase difference between different reflected light rays for the light reflection interlaced structure data according to the reflected light chromatic aberration data and the light reflection intensity evaluation data, to obtain the reflected interlaced structure light wave phase difference data; Step S235: performing structural interference effect analysis based on the phase difference data of the reflected staggered structure light wave to obtain light reflection structure interference effect data; Step S24: Calculate the probability of the environmental space distortion behavior according to the light reflection structure interference effect data and the light reflection intensity evaluation data to obtain the environmental space distortion behavior probability data; wherein step S24 includes: Step S241: calculating the optical path difference of different light reflections on the light reflection structure interference effect data to obtain the light reflection optical path difference data; Step S242: performing medium reflection interference simulation on the light reflection structure interference effect data according to the light reflection optical path difference data to obtain medium reflection interference simulation data; Step S243: performing polarization light interaction analysis on the light reflection structure interference effect data according to the medium reflection interference simulation data to obtain reflection structure polarization light interaction data; Step S244: performing light distortion critical state analysis based on the reflection structure polarization light interaction data, the medium reflection interference simulation data, and the light reflection optical path difference data to obtain light distortion critical state data; Step S245: performing regression interval estimation on the light distortion critical state data according to the light reflection intensity evaluation data to obtain light reflection distortion interval estimation data; Step S246: Calculating the probability of environmental space distortion behavior based on the light reflection distortion interval estimation data to obtain environmental space distortion behavior probability data; Step S3: performing spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data; performing light distortion behavior correction processing according to the spatial distortion structured data to obtain light distortion behavior correction data; wherein step S3 includes: Step S31: performing spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data; Step S32: extracting features from the spatial distortion structured data to obtain spatial distortion structure feature data; Step S33: performing a distortion structure multi-dimensional analysis on the spatial distortion structure data according to the spatial distortion structure feature data to obtain the distortion structure multi-dimensional feature data; Step S34: performing light distortion behavior correction processing according to the multi-dimensional characteristic data of the distortion structure to obtain light distortion behavior correction data; wherein step S34 includes: Step S341: calculating the curvature of the twisted structure multi-dimensional feature data to obtain the curvature data of the twisted structure; Step S342: performing hierarchical feature processing on the multi-dimensional feature data of the twisted structure according to the twisted structure curvature data to obtain twisted structure curvature hierarchical data; Step S343: calculating the layered light distortion index according to the layered data of the curvature of the distortion structure to obtain the layered light distortion index data; Step S344: performing light dynamic parameter correction according to the layered light distortion index data to obtain layered light dynamic parameter correction data; Step S345: performing light distortion behavior correction processing according to the layered light dynamic parameter correction data to obtain light distortion behavior correction data; Step S4: Based on the policy gradient algorithm, policy gradient learning is performed on the light distortion behavior correction data to obtain light correction policy gradient data; a light co-location perception model is constructed according to the light correction policy gradient data to obtain a light co-location perception model, and the light co-location perception model is sent to the data cloud platform to execute the laser radar camera co-location perception method; wherein, the light co-location perception model is constructed including: based on the light distortion behavior correction data, a policy gradient algorithm is applied to perform policy gradient learning, including: firstly, a policy network is established, which accepts the light distortion behavior correction data as input, is processed through a multi-layer neural network, and the network parameters are optimized using the policy gradient algorithm, and the gradient is calculated and the network weight is adjusted by comparing the difference between the actual correction effect and the expected effect. This process will gradually improve the strategy in multiple iterations, so that the light correction effect is continuously optimized, and the obtained light correction policy gradient data contains the best correction strategy and adjustment parameters. Based on these data, the light co-location perception model is constructed.

2. The laser radar camera co-location perception method according to claim 1, characterized in that: The regression interval estimation of the light distortion critical state data based on the light reflection intensity evaluation data includes the following steps: According to the light reflection intensity evaluation data, the light distortion critical state data is subjected to light distortion increasing law identification to obtain light distortion increasing law data; Perform incremental stepwise regression analysis on the data of increasing law of light distortion to obtain the incremental stepwise regression data of distortion; According to the distortion increasing stepwise regression data, multicollinearity constraint processing is performed on the light distortion increasing law data to obtain the collinear constraint data of the light distortion increasing law; According to the collinear constraint data of the increasing law of light distortion, regression analysis is performed on the data of the increasing law of light distortion to obtain the collinear regression data of the increasing law of light distortion; The regression interval is estimated based on the light distortion incremental collinear regression data to obtain the light reflection distortion interval estimation data.

3. A laser radar camera co-location perception system, characterized in that: Used to execute the laser radar camera co-location perception method as claimed in claim 1, the laser radar camera co-location perception system comprises: A perspective 3D model construction module is used to extract real-time ambient light data through an onboard camera lens to obtain real-time ambient light data; construct a mobile collaborative perspective 3D model based on the real-time ambient light data to obtain a mobile collaborative perspective 3D model; The environment space distortion behavior probability calculation module is used to evaluate the light reflection intensity of the real-time environment light data based on the mobile collaborative perspective three-dimensional model to obtain the light reflection intensity evaluation data; perform structural interference effect analysis on the light reflection intensity evaluation data to obtain the light reflection structural interference effect data; perform environment space distortion behavior probability calculation based on the light reflection structural interference effect data to obtain the environment space distortion behavior probability data; The correction processing module is used to perform spatial distortion structured recognition on the environmental spatial distortion behavior probability data to obtain spatial distortion structured data; perform light distortion behavior correction processing according to the spatial distortion structured data to obtain light distortion behavior correction data; The co-location perception model building module is used to perform policy gradient learning on the light distortion behavior correction data based on the policy gradient algorithm to obtain the light correction policy gradient data; build the light co-location perception model according to the light correction policy gradient data to obtain the light co-location perception model, and send the light co-location perception model to the data cloud platform to execute the lidar camera co-location perception method.

4. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, the laser radar camera co-location perception method as described in any one of claims 1-2 is implemented.

Images