Multi-target identification and segmentation system for autonomous vehicle

Abstract

The invention relates to the technical field of autonomous vehicles, in particular to a multi-target recognition and segmentation system for an autonomous vehicle, which comprises a data alignment module, a boundary optimization module, a threshold adjustment module and a sensor fusion module. According to the method, deep fusion of data collected from the camera and the LiDAR is optimized through a stereo matching algorithm, high-precision space and time alignment is realized, richer and more accurate input data are provided for subsequent processing, the recognition capability of the system on the edges and details of multiple targets in a complex environment is enhanced through application of the graph convolutional network, and the recognition efficiency of the system on the edges and details of the multiple targets in the complex environment is improved. Especially, the performance under the conditions of low illumination and high light reflection, dynamic adjustment of a threshold value is adaptively optimized according to environment illumination and reflection changes, the stability of image analysis is improved, efficient integration of multi-sensor information is achieved through data fusion processing of a multi-task learning network, and the method is suitable for large-scale popularization and application. And the environment perception and decision-making capability of the automatic driving vehicle is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving vehicles, and in particular to a multi-target recognition and segmentation system for autonomous driving vehicles. Background Art

[0002] Autonomous driving vehicle technology mainly involves the use of computer vision, sensor technology, artificial intelligence and machine learning methods to achieve autonomous navigation and driving decisions of vehicles. This technology enables vehicles to drive automatically without human drivers, and monitor the surrounding environment such as other vehicles, pedestrians, road signs and traffic conditions through integrated sensors and cameras. Autonomous driving vehicles also need to parse data through algorithms to ensure safe and efficient driving. This technology has also promoted the development of vehicle-to-everything (V2X), allowing vehicles to not only perceive the environment around themselves, but also communicate with other vehicles and roadside infrastructure, further improving safety and driving efficiency.

[0003] Among them, the main purpose of the multi-target recognition and segmentation system of autonomous driving vehicles is to use cameras and other sensors to identify and segment multiple targets around the vehicle in real time, such as other vehicles, pedestrians, and bicycles. Its purpose is to accurately identify the position and outline of various objects, so that the autonomous driving system can more accurately understand its surrounding environment and make appropriate driving decisions. It is a key component for realizing advanced autonomous driving functions (such as L3 and above) and is of great significance to improving road safety and driving efficiency.

[0004] Existing technologies have difficulty achieving real-time synchronization and accurate processing of data in highly dynamic environments, resulting in delays or errors in environmental perception. For example, the performance of sensors in severe weather or low-light conditions is insufficient, limiting the operational safety of vehicles. Existing technologies fail to effectively handle multi-target tracking problems in high-density traffic scenarios. Insufficient recognition and segmentation accuracy makes the vehicle's response measures in complex traffic inflexible or accurate, limiting emergency avoidance and rapid response scenarios, leading to inappropriate driving decisions and increasing the risk of traffic accidents. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose a multi-target recognition and segmentation system for an autonomous driving vehicle.

[0006] In order to achieve the above-mentioned object, the present invention adopts the following technical solution: a multi-target recognition and segmentation system for an autonomous driving vehicle, the system comprising: The data alignment module performs timestamp alignment and spatial correction based on the data collected by the camera and LiDAR sensor, uses a stereo matching algorithm to merge the point cloud and image data, and then fuses the merged data and time to obtain a stereo fusion data set; The boundary optimization module uses a graph convolutional network to perform edge recognition based on the stereo fusion data set, identifies the edge correspondence between the image and the point cloud, and refines the boundaries of multiple targets to obtain a boundary refinement data set; The threshold adjustment module analyzes the illumination and reflection characteristics of the target based on the boundary refinement data set, adjusts the threshold parameters in the image analysis, optimizes the adjusted threshold parameters and target detection according to the differentiated illumination and reflection conditions, and obtains illumination adaptability detection information; The sensor fusion module performs data synchronization processing based on the illumination adaptability detection information and the ultrasonic sensor data, inputs the data into the multi-task learning network, and performs mixed input processing to obtain a multi-sensor collaborative output result.

[0007] The present invention is improved in that the steps of timestamp alignment and space correction are specifically as follows: Based on the data collected by the camera and LiDAR sensor, the data includes timestamp and spatial coordinates, and the minimum timestamp is determined to obtain the reference timestamp; Based on the reference timestamp, the timestamp is calibrated using the formula:

[0008] Get the calibrated timestamp set ,in, is the original timestamp, is the base timestamp; Correct the spatial coordinates using the transformation matrix and translation vector using the formula:

[0009] Convert the original space coordinates to obtain the calibrated space coordinate set ,in, is the original space coordinate, is the transformation matrix for rotation and scaling, is the translation vector of the space coordinates.

[0010] The present invention is improved in that the steps of acquiring the stereoscopic fusion data set are specifically as follows: Using the stereo matching algorithm, the spatial difference and angular deviation between points are calculated, and the spatial accuracy of the data set is optimized by minimizing the deviation, using the formula:

[0011] The optimized point cloud data is obtained, where Yes To point The Euclidean distance of Yes To point The angle difference between represents the standard deviation of the spatial distance, represents the standard deviation of the angle differences, is the total number of pairs; Based on the optimized point cloud data, time re-fusion is performed to optimize the time deviation between sensors, using the formula:

[0012] Get the time-adjusted dataset ,in, Represents the difference between consecutive timestamps, represents the second derivative of the time difference, and is the adjustment factor for linear and acceleration adjustment, is the timestamp of the data point.

[0013] The present invention is improved in that the step of identifying the edge correspondence is specifically as follows: Based on the stereo fusion data set, edge features are extracted from the image and point cloud, and the gradient of the image is calculated using the formula:

[0014] Get image edge features ,in, and is an image exist and Partial derivatives of direction; For point clouds, calculate the spatial gradient using the formula:

[0015] Get point cloud edge features ,in, , and It is a point cloud exist , and Partial derivatives of direction; Based on the image edge features and point cloud edge features, edge matching is performed to determine the matching degree between features using the formula:

[0016] Get edge correspondence ,in, is the image edge feature, is the edge feature of the point cloud, is a small constant that avoids division by zero.

[0017] The present invention is improved in that the step of acquiring the edge thinning data set is specifically as follows: Filter the edge data with the best matching degree and refine the boundaries of multiple targets using the formula:

[0018] Get the refined boundary ,in, is the optimal edge data, is the weight coefficient, assigned according to edge criticality, is the similarity score of a single edge; The refined boundaries are merged into the stereo fusion data set to obtain a boundary refinement data set.

[0019] The present invention is improved in that the step of adjusting the threshold parameter is specifically as follows: Based on the boundary refinement data set, the image data of the target is extracted, and the illumination intensity and reflectivity of each target under the differential illumination and reflection conditions are calculated using the formula: and

[0020] Get the average light intensity and the average reflectivity ,in, is the total number of data points, is the light intensity at a single data point, is the reflectivity of a single data point; Based on the average light intensity and average reflectance, the threshold parameter of image analysis is adjusted using the formula:

[0021] Get the adjusted threshold ,in, is the current threshold, is the average light intensity, is the average reflectivity, and is the adjustment factor.

[0022] The present invention is improved in that the optimization steps of the target detection are specifically as follows: Based on the differentiated illumination and reflectance conditions, the fluctuation of illumination and reflectance is evaluated and the adjusted threshold is optimized using the formula:

[0023] Get the optimized threshold ,in, is the adjusted threshold, and is a coefficient that adjusts for the effects of illumination and reflectance variability, and are the coefficients of variation of illumination and reflectance, respectively.

[0024] The present invention is improved in that the steps of obtaining the multi-sensor collaborative output result are specifically as follows: Based on the light adaptability detection information, capture ultrasonic sensor data, determine the latest timestamp, and perform data synchronization processing to obtain synchronized data; Based on the synchronized data, the original data is resampled or interpolated, and the missing data points are calculated using linear interpolation, using the formula:

[0025] Get the interpolated data value ,in, is the earlier data value among the adjacent data points before the timestamp synchronization, is the later data value among the adjacent data points before the timestamp synchronization, is the timestamp after synchronization, is the later timestamp among adjacent data points, is the earlier timestamp among adjacent data points; The interpolated data values ​​are analyzed to extract key features, and a multi-task learning network is applied for hybrid processing, using the formula:

[0026] Get multi-sensor collaborative output results ,in, is the interpolated data value, and Represents the current data values ​​of the light and ultrasonic sensors respectively. , and is the data source weight, which is used to adjust the influence of sensor input.

[0027] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, the deep fusion of data collected from the camera and LiDAR is optimized through the stereo matching algorithm, and high-precision spatial and temporal alignment is achieved, thereby providing richer and more accurate input data for subsequent processing. The application of graph convolutional networks enhances the system's ability to recognize the edges and details of multiple targets in complex environments, especially the performance under low light and high reflective conditions. The dynamic adjustment of the threshold is adaptively optimized according to the changes in ambient light and reflection, which improves the stability of image analysis. Through the data fusion processing of the multi-task learning network, the efficient integration of multi-sensor information is achieved, which improves the environmental perception and decision-making ability of autonomous driving vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a system flow chart of the present invention; Figure 2A flowchart of timestamp alignment and spatial correction in the present invention; Figure 3 This is a flow chart for obtaining a stereoscopic fusion data set in the present invention; Figure 4 This is a flow chart for identifying edge correspondence in the present invention; Figure 5 This is a flow chart for obtaining edge thinning data sets in the present invention; Figure 6 is a flow chart for adjusting the threshold parameters in the present invention; Figure 7 It is an optimization flow chart of target detection in the present invention; Figure 8 This is a flow chart for obtaining the collaborative output results of multiple sensors in the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0030] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating positions or positional relationships, are based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0031] See also Figure 1 The present invention provides a technical solution: a multi-target recognition and segmentation system for an autonomous driving vehicle comprises: The data alignment module performs timestamp alignment and spatial correction based on the data collected by the camera and LiDAR sensor, uses a stereo matching algorithm to merge the point cloud and image data, and then fuses the merged data and time to obtain a stereo fusion data set; The boundary optimization module uses a graph convolutional network to perform edge recognition based on a stereo fusion dataset, identifies the edge correspondence between the image and the point cloud, and refines the boundaries of multiple targets to obtain a boundary refinement dataset. The threshold adjustment module analyzes the illumination and reflection characteristics of the target based on the boundary refinement data set, adjusts the threshold parameters in the image analysis, optimizes the adjusted threshold parameters and target detection according to the differentiated illumination and reflection conditions, and obtains illumination adaptability detection information; The sensor fusion module performs data synchronization processing based on the light adaptability detection information and the ultrasonic sensor data, inputs the data into the multi-task learning network, and performs mixed input processing to obtain the multi-sensor collaborative output results.

[0032] The stereo fusion dataset includes image depth information, stereo alignment indicators, and fusion integrity evaluation results. The boundary refinement dataset includes boundary identification points, segmentation line types, and object distinction. The lighting adaptability detection information includes lighting level differences, reflectance characteristic index, and adjusted threshold range. The multi-sensor collaborative output results include perception data fusion degree, sensor feedback consistency, and output signal quality.

[0033] See also Figure 2 ,The steps of timestamp alignment and spatial correction are as follows: Based on the data collected by the camera and LiDAR sensor, the data includes timestamp and spatial coordinates, and the minimum timestamp is determined to obtain the reference timestamp; Based on the reference timestamp, other timestamps are calibrated using the formula:

[0034] Get the calibrated timestamp set , ensuring time data synchronization, where is the original timestamp, is the base timestamp; Correct the spatial coordinates using the transformation matrix and translation vector using the formula:

[0035] Convert the original space coordinates to obtain the calibrated space coordinate set ,in, is the original space coordinate, is the transformation matrix for rotation and scaling, is the translation vector of the space coordinates.

[0036] The collected timestamp data is [1625097600, 1625097602, 1625097605, 1625097601]; From the start of monitoring to the earliest recorded time point for ; Calibrate each timestamp: The calibrated timestamp set is , represents the time difference of other time points relative to the earliest acquisition time, which is used for synchronous analysis and processing.

[0037] set up

[0038] calculate

[0039]

[0040] Represents the calibrated spatial coordinates, ensuring the consistency of spatial data collected by different devices.

[0041] See also Figure 3 ,The specific steps for obtaining the stereo fusion data set are: Using the stereo matching algorithm, the spatial difference and angle deviation between points are calculated, and the spatial accuracy of the data set is optimized by minimizing the deviation. The formula is:

[0042] The optimized point cloud data is obtained, where Yes To point The Euclidean distance of Yes To point The angle difference between represents the standard deviation of the spatial distance, represents the standard deviation of the angle differences, is the total number of pairs; Based on the optimized point cloud data, time re-fusion is performed to optimize the time deviation between sensors using the formula:

[0043] Get the time-adjusted dataset , Represents the difference between consecutive timestamps, represents the second derivative of the time difference, and is the adjustment factor for linear and acceleration adjustment, is the timestamp of the data point.

[0044] Collect two points and , then: calculate :

[0045] Monitoring Angle is 30°, then Convert to radians:

[0046] set up and :

[0047] Reflects the point and The degree of match in space and angle, with higher scores indicating better match.

[0048] Collected timestamp sequence ,but:

[0049]

[0050] set up and :

[0051] Adjusted timestamps provide more accurate time estimates that take into account temporal trends.

[0052] See also Figure 4 , the identification steps of edge correspondence are as follows: Based on the stereo fusion data set, edge features are extracted from images and point clouds, and the gradient of the image is calculated using the formula:

[0053] Get image edge features ,in, and is an image exist and Partial derivatives of direction; For point clouds, calculate the spatial gradient using the formula:

[0054] Get point cloud edge features ,in, , and It is a point cloud exist , and Partial derivatives of direction; Based on the edge features of the image and the edge features of the point cloud, edge matching is performed to determine the matching degree between the features using the formula:

[0055] Get edge correspondence ,in, is the image edge feature, is the edge feature of the point cloud, is a small constant that avoids division by zero.

[0056] Image edge features : Suppose at some point, the image exist Directional gradient ,exist Directional gradient .

[0057] Calculate using the formula:

[0058] Indicates the edge strength of the image at this point.

[0059] Point cloud edge features : The same point in the point cloud middle, Directed Gradient , direction , direction ; Calculate using the formula:

[0060] Indicates the edge strength of the point cloud at this point.

[0061] For a pair and The value of :

[0062] Calculated using the formula:

[0063] Indicates the degree of edge matching between the image and the point cloud at the considered point.

[0064] See also Figure 5 , the steps for obtaining the edge thinning dataset are as follows: Filter the edge data with the best matching degree and refine the boundaries of multiple targets using the formula:

[0065] Get the refined boundary ,in, is the optimal edge data, is the weight coefficient, assigned according to edge criticality, is the similarity score of a single edge; The refined boundaries are merged into the stereo fusion dataset to obtain a boundary refinement dataset.

[0066] Let the weight of each edge be is 1.5, and the optimal edge data obtained from the data set are 0.8 and 0.9, and the boundary is refined:

[0067] It means that after weight adjustment, the average score of edge refinement is 1.275.

[0068] See also Figure 6 , the specific steps for adjusting the threshold parameters are: Based on the boundary refinement dataset, the image data of the target is extracted, and the illumination intensity and reflectivity of each target under the conditions of differential illumination and reflection are calculated using the formula: and

[0069] Get the average light intensity and the average reflectivity ,in, is the total number of data points, is the light intensity at a single data point is the reflectivity of a single data point; Based on the average light intensity and average reflectance, adjust the threshold parameters of image analysis using the formula:

[0070] Get the adjusted threshold ,in, is the current threshold, is the average light intensity, is the average reflectivity, and is the adjustment factor.

[0071] The following data points of light intensity and reflectance were collected: Light intensity:

[0072] Reflectivity:

[0073] Number of data points .

[0074] Calculate the average light intensity:

[0075] Calculate the average reflectivity:

[0076] The average light intensity is shown to be 187.5 and the average reflectance is 0.55.

[0077] The following data is collected: is 50; and Set to 0.1 and 5; Combining the above results, calculate:

[0078] Indicates a new image analysis threshold that is more adapted to the current lighting and reflection conditions.

[0079] See also Figure 7 , the optimization steps of target detection are as follows: Based on the differentiated lighting and reflection conditions, the fluctuation of lighting and reflectance is evaluated and the adjusted threshold is optimized using the formula:

[0080] Get the optimized threshold ,in, is the adjusted threshold, and is a coefficient that adjusts for the effects of illumination and reflectance variability, and are the coefficients of variation of illumination and reflectance, respectively.

[0081] The following data was collected: ; ; ; .

[0082] Coefficient of variation of light:

[0083] Reflectivity variation coefficient:

[0084] set up and , collect the adjusted threshold : ;

[0085] It shows that under given illumination and reflectivity variability, the threshold needs to be increased to 71.665 to better adapt to environmental conditions. The optimized threshold means that the sensitivity and accuracy of target detection have been improved, especially under complex illumination and reflection conditions. By using this new threshold, the algorithm can more effectively distinguish between targets and backgrounds, reducing false detections and missed detections.

[0086] See also Figure 8 ,The specific steps for obtaining the multi-sensor collaborative output results are: Based on the light adaptability detection information, the ultrasonic sensor data is captured, the latest timestamp is determined, and the data synchronization is performed to obtain the synchronized data; Based on the synchronized data, the original data is resampled or interpolated, and the missing data points are calculated using linear interpolation, using the formula:

[0087] Get the interpolated data value ,in, is the earlier data value among the adjacent data points before the timestamp synchronization, is the later data value among the adjacent data points before the timestamp synchronization, is the timestamp after synchronization, is the later timestamp among adjacent data points, is the earlier timestamp among adjacent data points; The interpolated data values ​​are analyzed to extract key features, and a multi-task learning network is applied for hybrid processing using the formula:

[0088] Get multi-sensor collaborative output results ,in, is the interpolated data value, and Represents the current data values ​​of the light and ultrasonic sensors respectively. , and is the data source weight, which is used to adjust the influence of sensor input.

[0089] In time The data value at ; In time The data value at ; Synchronize time , calculate the interpolated data points:

[0090] The interpolation results are shown in The estimated data value of time is used to fill the data gaps between time points, which determines the accuracy of data analysis at the synchronized time points, helps ensure the continuity and accuracy of data during the analysis process, and improves the adaptability to actual environmental changes and the accuracy of predictions.

[0091] Interpolated data value ; Current light sensor data ; Ultrasonic sensor data ; The weight is set to , , .

[0092] Compute the output of mixed input processing:

[0093] Represents the output of the multi-task learning network, which is a value synthesized based on the current sensor readings and the adjusted weights. The output value is used for further analysis and model training to optimize the performance of target detection.

[0094] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A multi-target recognition and segmentation system for autonomous driving vehicles, It is characterized in that The system comprises: The data alignment module performs timestamp alignment and spatial correction based on the data collected by the camera and LiDAR sensor, uses a stereo matching algorithm to merge the point cloud and image data, and then fuses the merged data and time to obtain a stereo fusion data set; The boundary optimization module uses a graph convolutional network to perform edge recognition based on the stereo fusion data set, identifies the edge correspondence between the image and the point cloud, and refines the boundaries of multiple targets to obtain a boundary refinement data set; The threshold adjustment module analyzes the illumination and reflection characteristics of the target based on the boundary refinement data set, adjusts the threshold parameters in the image analysis, optimizes the adjusted threshold parameters and target detection according to the differentiated illumination and reflection conditions, and obtains illumination adaptability detection information; The sensor fusion module performs data synchronization processing based on the illumination adaptability detection information and the ultrasonic sensor data, inputs the data into the multi-task learning network, and performs mixed input processing to obtain a multi-sensor collaborative output result.

2. The multi-target recognition and segmentation system for an autonomous driving vehicle according to claim 1, It is characterized in that The steps of timestamp alignment and spatial correction are specifically as follows: Based on the data collected by the camera and LiDAR sensor, the data includes timestamp and spatial coordinates, and the minimum timestamp is determined to obtain the reference timestamp; Based on the reference timestamp, the timestamp is calibrated using the formula: ； Get the calibrated timestamp set ,in, is the original timestamp, is the reference timestamp; the spatial coordinates are corrected using the transformation matrix and translation vector using the formula: ； Convert the original space coordinates to obtain the calibrated space coordinate set ,in, is the original space coordinate, is the transformation matrix for rotation and scaling, is the translation vector of the space coordinates.

3. The multi-target recognition and segmentation system for an autonomous driving vehicle according to claim 1, It is characterized in that The steps for obtaining the stereoscopic fusion data set are specifically as follows: Using the stereo matching algorithm, the spatial difference and angular deviation between points are calculated, and the spatial accuracy of the data set is optimized by minimizing the deviation, using the formula: ； The optimized point cloud data is obtained, where Yes To point The Euclidean distance of Yes To point The angle difference between represents the standard deviation of the spatial distance, represents the standard deviation of the angle differences, is the total number of pairs; Based on the optimized point cloud data, time re-fusion is performed to optimize the time deviation between sensors, using the formula: ; Get the time-adjusted dataset ,in, Represents the difference between consecutive timestamps, represents the second derivative of the time difference, and is the adjustment factor for linear and acceleration adjustment, is the timestamp of the data point.

4. The multi-target recognition and segmentation system for an autonomous driving vehicle according to claim 1, It is characterized in that The step of identifying the edge correspondence is specifically as follows: Based on the stereo fusion data set, edge features are extracted from the image and point cloud, and the gradient of the image is calculated using the formula: ; Get image edge features ,in, and is an image exist and The partial derivative of ; For point clouds, calculate the spatial gradient using the formula: ; Get point cloud edge features ,in, , and It is a point cloud exist , and Partial derivatives of direction; Based on the image edge features and point cloud edge features, edge matching is performed to determine the matching degree between features using the formula: ; Get edge correspondence ,in, is the image edge feature, is the edge feature of the point cloud, is a small constant that avoids division by zero.

5. The multi-target recognition and segmentation system for an autonomous driving vehicle according to claim 1, It is characterized in that The steps for obtaining the edge thinning data set are specifically as follows: Filter the edge data with the best matching degree and refine the boundaries of multiple targets using the formula: ; Get the refined boundary ,in, is the optimal edge data, Weight coefficients, assigned according to edge criticality, is the similarity score of a single edge; The refined boundaries are merged into the stereo fusion data set to obtain a boundary refinement data set.

6. The multi-target recognition and segmentation system for an autonomous driving vehicle according to claim 1, It is characterized in that The steps for adjusting the threshold parameters are specifically as follows: Based on the boundary refinement data set, the image data of the target is extracted, and the illumination intensity and reflectivity of each target under the differential illumination and reflection conditions are calculated using the formula: and ; Get the average light intensity and the average reflectivity ,in, is the total number of data points, is the light intensity at a single data point, is the reflectivity of a single data point; Based on the average light intensity and average reflectance, the threshold parameter of image analysis is adjusted using the formula: ; Get the adjusted threshold ,in, is the current threshold, is the average light intensity, is the average reflectivity, and is the adjustment factor.

7. The multi-target recognition and segmentation system for an autonomous driving vehicle according to claim 1, It is characterized in that The optimization steps of the target detection are specifically as follows: Based on the differentiated illumination and reflectance conditions, the fluctuation of illumination and reflectance is evaluated and the adjusted threshold is optimized using the formula: ; Get the optimized threshold ,in, is the adjusted threshold, and is a coefficient that adjusts for the effects of illumination and reflectance variability, and are the coefficients of variation of illumination and reflectance, respectively.

8. The multi-target recognition and segmentation system for an autonomous driving vehicle according to claim 1, It is characterized in that The steps for obtaining the multi-sensor collaborative output result are specifically as follows: Based on the light adaptability detection information, capture ultrasonic sensor data, determine the latest timestamp, and perform data synchronization processing to obtain synchronized data; Based on the synchronized data, the original data is resampled or interpolated, and the missing data points are calculated using linear interpolation, using the formula: ; Get the interpolated data value ,in, is the earlier data value among the adjacent data points before the timestamp synchronization, is the later data value among the adjacent data points before the timestamp synchronization, is the timestamp after synchronization, is the later timestamp among adjacent data points, is the earlier timestamp among adjacent data points; The interpolated data values ​​are analyzed to extract key features, and a multi-task learning network is applied for hybrid processing using the formula: ; Get multi-sensor collaborative output results ,in, is the interpolated data value, and Represents the current data values ​​of the light and ultrasonic sensors respectively. , and is the data source weight, which is used to adjust the influence of sensor input.

Images