Vehicle target classification method and electronic equipment

Abstract

The embodiment of the invention provides a target classification method of a vehicle and electronic equipment, and the method comprises the steps: controlling a plurality of sensors on the vehicle to sense the driving condition in front of the vehicle in a vehicle driving process, and obtaining a plurality of pieces of sensor data; controlling the plurality of sensors, and performing pre-classification processing on at least one target object in the plurality of sensor data to obtain pre-classification results corresponding to the plurality of sensor data; based on the multiple pieces of sensor data, scene condition information of the at least one target object is determined, and the scene condition information is used for representing the scene condition of the at least one target object; determining a target weight coefficient corresponding to the pre-classification result based on the scene condition information and / or the pre-classification result; and based on the target weight coefficient, performing weighted fusion on the pre-classification result to obtain a target classification result of the at least one target object. According to the invention, the technical problem of low accuracy of target classification of vehicles in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and more specifically, to a vehicle target classification method and electronic device. Background Technology

[0002] With the development of intelligent driving systems, target classification has become a crucial technology supporting the safe operation of vehicle decision-making modules. However, traditional target classification technologies face multiple challenges and limitations, primarily: multi-sensor fusion classification suffers from an efficiency-cost conflict; while data-level or feature-level fusion can compensate for the shortcomings of single sensors, the processing of large amounts of raw data or high-level features exacerbates computational latency; and the reliance on high-performance chips drives up hardware costs, making this technology difficult to widely apply in economical vehicles. Consequently, the accuracy of vehicle target classification in related technologies remains relatively low. Summary of the Invention

[0003] This application provides a vehicle target classification method and electronic device to at least solve the technical problem of low accuracy in vehicle target classification in related technologies.

[0004] According to one aspect of the embodiments of this application, a vehicle target classification method is provided, comprising: during vehicle operation, controlling multiple sensors on the vehicle to perceive the driving conditions in front of the vehicle and obtaining multiple sensor data; controlling the multiple sensors to perform pre-classification processing on at least one target object in the multiple sensor data to obtain pre-classification results corresponding to the multiple sensor data respectively; determining scene condition information of at least one target object based on the multiple sensor data, wherein the scene condition information is used to characterize the scene condition in which at least one target object is located; determining target weight coefficients corresponding to the pre-classification results based on the scene condition information and / or the pre-classification results; and performing weighted fusion on the pre-classification results based on the target weight coefficients to obtain a target classification result for at least one target object.

[0005] In the above embodiments of this application, controlling multiple sensors to perform pre-classification processing on at least one target object in the multiple sensor data to obtain pre-classification results corresponding to the multiple sensor data includes: controlling multiple sensors to perform pre-classification processing on at least one target object in the multiple sensor data based on preset pre-training algorithms corresponding to the multiple sensors to obtain pre-classification results, wherein the preset pre-training algorithms are determined in advance based on the sensor types of the multiple sensors, and each pre-classification result includes the pre-classification type of at least one target object and the confidence level of the pre-classification type.

[0006] In the above embodiments of this application, determining the target weight coefficient corresponding to the pre-classification result based on scene condition information and / or pre-classification result includes: adjusting the initial weight coefficients corresponding to multiple sensors based on scene condition information and / or pre-classification result to obtain the target weight coefficient, wherein the initial weight coefficient is determined in advance based on the sensor type of multiple sensors; preferably, the initial visual weight coefficient corresponding to the visual sensor among the multiple sensors is greater than the initial lidar weight coefficient corresponding to the lidar among the multiple sensors.

[0007] In the above embodiments of this application, the initial weight coefficients corresponding to multiple sensors are adjusted based on scene condition information to obtain target weight coefficients, including: when the scene condition information indicates that the visual observability of at least one target object is greater than a preset observability threshold, the initial visual weight coefficient in the initial weight coefficients is increased to obtain the target weight coefficient.

[0008] In the above embodiments of this application, the initial weight coefficients corresponding to multiple sensors are adjusted based on the pre-classification results to obtain the target weight coefficients. This includes: when the pre-classification result of the lidar in the pre-classification results indicates that at least one target object is a traffic participant, the initial lidar weight coefficient in the initial weight coefficients is increased to obtain the target weight coefficients.

[0009] In the above embodiments of this application, the method further includes: determining the moving speed, height information and location information of at least one target object based on lidar data from multiple sensor data, wherein the location information is used to indicate whether at least one target object is within the road area where the vehicle is traveling; and determining the lidar pre-classification result in the pre-classification result based on the moving speed, height information and location information.

[0010] In the above embodiments of this application, determining the LiDAR pre-classification result in the pre-classification result based on movement speed, altitude information, and location information includes: determining the initial LiDAR pre-classification result of at least one target object based on movement speed, altitude information, and location information; determining the number of clustered obstacles corresponding to at least one target object in the LiDAR data; adjusting the initial LiDAR pre-classification result based on the number of clustered obstacles to obtain the LiDAR pre-classification result; preferably, adjusting the initial LiDAR pre-classification result based on the number of clustered obstacles to obtain the LiDAR pre-classification result includes: determining the LiDAR pre-classification result as a non-traffic participant when the initial LiDAR pre-classification result is a traffic participant and the number of clustered obstacles is greater than a preset number.

[0011] In the above embodiments of this application, the method further includes: when the target classification result of at least one target object is a traffic participant, and at least one target object is not in the road area where the vehicle is traveling, adjusting the target classification result of at least one target object to a non-traffic participant.

[0012] In the above embodiments of this application, the method further includes: when the confidence level corresponding to the target classification result of at least one target object is less than a preset confidence level, determining the target classification result of at least one target object based on the historical target classification results of at least one target object.

[0013] According to one aspect of the embodiments of this application, a vehicle target classification device is provided, comprising: a first control module, configured to control multiple sensors on the vehicle to perceive the driving conditions in front of the vehicle during vehicle operation, and obtain multiple sensor data, wherein the multiple sensors are of different sensor types; a second control module, configured to control the multiple sensors to perform pre-classification processing on at least one target object in the multiple sensor data, and obtain pre-classification results corresponding to the multiple sensor data respectively; a first determination module, configured to determine scene condition information of at least one target object based on the multiple sensor data, wherein the scene condition information is used to characterize the scene condition in which at least one target object is located; a second determination module, configured to determine the target weight coefficient corresponding to the pre-classification result based on the scene condition information and / or the pre-classification result; and a fusion module, configured to perform weighted fusion of the pre-classification result based on the target weight coefficient, and obtain a target classification result for at least one target object.

[0014] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0016] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0017] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the methods in various embodiments of this application.

[0018] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0019] In this embodiment, during vehicle operation, firstly, multiple sensors on the vehicle are controlled to perceive the driving conditions ahead, obtaining multiple sensor data. These sensors are of different types. Next, the multiple sensors are controlled to pre-classify at least one target object from the sensor data, obtaining pre-classification results corresponding to each sensor data point. Then, based on the multiple sensor data, scene condition information for at least one target object is determined, characterizing the scene condition of the target object. Next, based on the scene condition information and / or the pre-classification results, target weight coefficients corresponding to the pre-classification results are determined. Finally, based on the target weight coefficients, the pre-classification results are weighted and fused to obtain a target classification result for at least one target object. This technical solution controls each sensor to operate independently during vehicle operation. Due to their different working principles and data characteristics, each sensor can capture information from different dimensions. Pre-classification processing allows for initial screening of sensor data before decision-level fusion, reducing the computational burden of subsequent fusion algorithms and minimizing the consumption of computing resources. Based on scene condition information and / or pre-classification results, the target weight coefficients corresponding to the pre-classification results are determined, and the weight coefficients of each sensor are intelligently adjusted, realizing a dynamic weight allocation mechanism for the scene. This allows for automatic adjustment of sensor data weights according to different scenes, thereby enhancing adaptability to various driving environments. Target classification results based on dynamic weights enable the intelligent driving system to rely on more reliable and comprehensive information when making decisions, improving driving safety and efficiency. The resulting target classification results are based on reliable information, helping intelligent driving vehicles make more accurate judgments in path planning, collision warning, etc., thereby improving driving safety, reducing unnecessary sudden braking or evasive maneuvers, enhancing passenger comfort and driving efficiency, and thus solving the technical problem of low accuracy in vehicle target classification in related technologies. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 This is a flowchart of a vehicle target classification method according to an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of a vehicle target classification device according to an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] According to an embodiment of this application, an embodiment of a vehicle target classification method is provided. The steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than that shown here.

[0026] This embodiment provides a vehicle target classification method. Figure 1 This is a flowchart of a vehicle target classification method according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0027] Step S102: During the vehicle's operation, control multiple sensors on the vehicle to perceive the driving conditions in front of the vehicle and obtain multiple sensor data.

[0028] Among them, several sensors are of different sensor types.

[0029] The aforementioned sensors refer to different types of sensors installed on a vehicle to perceive its surrounding environment, including but not limited to vision sensors, lidar, and millimeter-wave radar. Due to differences in their working principles and technical characteristics, the types of data acquired and their applicable scenarios may vary among these sensors. Vision sensors may include multiple high-definition cameras with different perspectives, capable of capturing static images and dynamic video information around the vehicle. Through computer vision technology, vision sensors can identify features such as the shape, color, and texture of objects in the image. Lidar uses laser beams to scan the surrounding environment, constructing a 3D point cloud model of the target by receiving reflected laser signals. Lidar provides high-precision distance and position information and can also measure the size and surface characteristics of targets, making it suitable for accurate target detection and classification in complex environments. Millimeter-wave radar uses millimeter-wave electromagnetic waves for detection, providing information on the presence and motion status of targets at long distances under various weather conditions. This includes the target's distance, relative velocity, and reflectivity. Millimeter-wave radar offers all-weather reliability and sensitivity to moving targets.

[0030] The aforementioned multiple sensor data can refer to various raw data collected by multiple sensors within a specific time window. This data can include: images or video streams provided by visual sensors; point cloud data generated by lidar; and distance, speed, and radar cross-section values ​​reported by millimeter-wave radar. These data together constitute the input of the vehicle perception system and can serve as the data basis for target classification.

[0031] As an optional implementation, the vehicle can be equipped with a variety of sensor suites, including but not limited to visual sensors, lidar, and millimeter-wave radar. These sensors can be deployed at the front, sides, and top of the vehicle to achieve omnidirectional perception coverage. Each sensor can operate independently, collecting real-time information about the road ahead based on specific perception principles and ranges. Visual sensors can capture continuous image streams, lidar can construct point cloud maps of the area ahead by emitting lasers and measuring echo times, and millimeter-wave radar can detect moving objects and static obstacles through the Doppler effect and the intensity of received signals.

[0032] As an alternative implementation, the raw sensor data collected by multiple sensors can be preprocessed to remove noise, fill in blind spots, and synchronized via timestamps. This ensures that data from different types of sensors can be compared and fused within the same timeframe. This preprocessing process may involve frame rate adjustment, data calibration, and time delay compensation between sensors. Each sensor can then generate specific perception results based on the data it collects, including information such as the target's position, velocity, size, and classification.

[0033] In the aforementioned process, the data redundancy and complementarity among different sensor types enhance the robustness of the perception system. For example, visual sensors provide high-resolution images under well-lit conditions, while lidar and millimeter-wave radar maintain stable detection performance at night or in adverse weather conditions, thus ensuring accurate acquisition of the required road information in various environments. Due to their physical characteristics, each sensor has different sensitivities and recognition advantages for specific types of targets. For instance, lidar's accurate capture of three-dimensional structures helps distinguish obstacles of different heights and shapes; this diverse information source improves the accuracy and reliability of target recognition. Through independent data streams from multiple sensors, the intelligent driving system can update its understanding of the road environment in real time and respond quickly to changes in the situation ahead.

[0034] Step S104: Control multiple sensors to perform pre-classification processing on at least one target object in the multiple sensor data to obtain the pre-classification results corresponding to the multiple sensor data respectively.

[0035] At least one target object mentioned above can refer to an entity located in front of or around the vehicle's driving path that may affect the vehicle's safe driving. This may include, but is not limited to, vehicles, pedestrians, cyclists, static obstacles, animals, or other non-standard targets. The target object can be a single individual or a collection of multiple targets in the same scene.

[0036] The aforementioned pre-classification results can refer to the preliminary classification results of the target object after each sensor performs independent processing, which may include the object's type and confidence level. For example, a visual haptic sensor may report a vehicle with a confidence level of 90%, a lidar may report a pedestrian with a confidence level of 85%, and a millimeter-wave radar may report a moving object but with an uncertain type.

[0037] As an alternative implementation, multiple sensor data can be pre-classified using their respective pre-classification algorithms. For example, a visual sensor can identify target boundaries in an image using a convolutional neural network; a lidar can use clustering algorithms to divide point clouds into individual targets; and a millimeter-wave radar can analyze target mobility based on Doppler frequency shift. Each sensor can have a dedicated pre-classification algorithm tailored to its data characteristics, such as a classifier based on image features, a classifier based on point cloud size and reflectivity, or a classifier based on radar signal features, thereby obtaining pre-classification results for each sensor data set.

[0038] In the above process, since a single sensor may encounter performance bottlenecks in certain scenarios, such as insufficient light affecting camera vision and rain and snow affecting the quality of LiDAR point clouds, independent pre-classification by each sensor allows for mutual verification and supplementation, enhancing classification stability and adaptability. Each sensor provides information from its own perspective, such as texture and color features from visual sensors, 3D point clouds and dimensions from LiDAR, and velocity and orientation data from millimeter-wave radar. Integrating this multi-dimensional information helps to understand the target object more comprehensively and accurately, improving classification accuracy. Pre-classification processing allows for preliminary screening of sensor data before decision-level fusion, reducing the consumption of computational resources.

[0039] Step S106: Based on data from multiple sensors, determine the scene status information of at least one target object.

[0040] Among them, scene condition information is used to characterize the scene condition in which at least one target object is located.

[0041] The aforementioned scene condition information refers to information derived from comprehensive analysis of sensor data, used to describe the state and characteristics of the target's environment. This information may include, but is not limited to, lighting conditions, weather conditions, the positional relationship between the target and the vehicle, and the target's motion state. Scene condition information can serve as a basis for adjusting sensor weights and improving classification strategies.

[0042] As an optional implementation, data from multiple sensors, including visual sensors, LiDAR, and millimeter-wave radar, can be calibrated and integrated temporally and spatially, unifying the observations from different sensors under a common reference framework. For example, coordinate transformation and time synchronization ensure that the target position in the LiDAR point cloud and camera image is consistent, enabling direct comparison. The fused data can be used to extract a series of information describing the target object and its environment, including but not limited to the target's movement state, its three-dimensional dimensions, its relative position to vehicles, environmental details near the target, and weather and lighting conditions. Then, based on the extracted features, scene information about the target can be constructed. For example, identifying the weather conditions, whether the target is at an intersection, inside a tunnel, or on an open road, or determining whether the target is partially obscured or near the roadside.

[0043] In the above process, combining surrounding scene information can significantly improve the accuracy of target object classification. For example, when a target is located at an intersection and there are pedestrians nearby, the classification results of LiDAR may be interfered with. In this case, visual information provided by visual sensors can play an auxiliary role, helping to make a more accurate judgment. By analyzing scene information, classification strategies and sensor weights can be dynamically adjusted to better adapt to the current driving environment, achieving more flexible and intelligent perception capabilities.

[0044] Step S108: Based on the scene status information and / or the pre-classification results, determine the target weight coefficient corresponding to the pre-classification results.

[0045] The aforementioned target weight coefficients refer to the weight coefficients assigned to the pre-classification results of each sensor, representing the reliability of each sensor under the current scene and target type. The level of the target weight coefficient affects the formation of the classification result; a higher weight indicates that the classification opinion of that sensor has a greater weight in the decision-making process.

[0046] As an optional implementation, target weight coefficients corresponding to the pre-classification results can be determined based on scene condition information. Scene analysis can be performed on the comprehensive data collected from visual sensors, LiDAR, and millimeter-wave radar, which may include identifying weather conditions, lighting levels, target types, and dynamic behaviors, including movement speed and direction. For example, it can be determined whether the current environment is a low-light nighttime environment or whether the target is moving rapidly. The performance of each sensor in the current environment can be evaluated based on scene condition information. For example, if the ambient light is insufficient, the camera's classification results may become unreliable, and the target weight coefficient of the visual sensor's pre-classification results can be reduced. Conversely, if LiDAR can still provide clear point cloud data in rainy weather, the weight coefficient can be increased accordingly. The calculation of weight coefficients can be based on complex mathematical models or machine learning algorithms, and factors considered may include, but are not limited to, historical classification accuracy, the performance of specific sensor types in specific scenes, and the distance between the target and the sensor, to ensure the rationality of weight allocation.

[0047] As an alternative implementation, target weight coefficients can be determined based on the pre-classification results. The pre-classification results of each sensor for the target object can be evaluated, considering the degree of consistency between the confidence level of the pre-classification results and the actual attributes of the target object. For example, for a visual camera, if the target is clear and conforms to the characteristics of a certain target category, and the confidence level is high, the visual camera will be assigned a higher weight. For millimeter-wave radar, if the echo intensity and Doppler frequency of the target indicate that it may be a stationary or slowly moving object, the weight of the millimeter-wave radar can be adjusted accordingly. Based on the above evaluation and fusion, the weight coefficients of each sensor for target classification can be dynamically adjusted to obtain the target weight coefficients.

[0048] As an alternative implementation, the target weight coefficients corresponding to the pre-classification results can be determined based on scene condition information and pre-classification results. Scene condition information, such as light intensity, weather conditions, target distance, and speed, can be extracted from sensor data. This scene condition information provides a basis for subsequent weight adjustments. Pre-classification confidence analysis can be performed to determine the pre-classification results. Each sensor's pre-classification result for the target can be accompanied by a confidence score, reflecting the sensor's level of certainty in classifying the target. Then, considering both scene condition information and pre-classification results, the target weight coefficients can be determined. Before the decision-level fusion processing begins, the pre-classification results of each sensor can be weighted according to the calculated target weight coefficients to ensure that the proportion of each sensor's contribution in the fusion result matches its classification performance in the current scene.

[0049] In the above process, even under adverse environmental conditions, such as severe weather and low light, high classification accuracy and stability can be maintained by assigning higher weights to robust sensor data. This allows for automatic adjustment of sensor data weights based on different scenarios, thereby enhancing adaptability to various driving environments. For example, on highways, more reliance is placed on millimeter-wave radar and lidar, while in complex urban environments, visual information from visual sensors can be given greater weight. Target classification results based on dynamic weights enable intelligent driving systems to rely on more reliable and comprehensive information when making decisions, improving driving safety and efficiency.

[0050] Step S110: Based on the target weight coefficient, the pre-classification results are weighted and fused to obtain the target classification result of at least one target object.

[0051] The target classification result mentioned above can refer to the target object type obtained by weighted fusion of pre-classification results from various sensors and taking into account scene condition information. The target classification result can serve as an important basis for intelligent driving systems to make decisions, such as planning driving routes and avoiding collisions.

[0052] As an optional implementation, target weight coefficients for each sensor's pre-classification results can be calculated based on the analyzed scene information and the pre-classification results from each sensor. For each target object, the pre-classification results obtained from different sensors can be multiplied by their corresponding weight coefficients, and then the weighted results can be summed. After weighted fusion of the pre-classification results, the classification with the higher weighted sum can be selected as the target classification result, thereby improving the consensus and accuracy of the classification.

[0053] In the above process, even if a particular sensor performs poorly in a specific environment, such as a vision sensor having limited field of view at night or in rainy weather, accurate classification can be achieved by relying on supplementary information from other sensors through weight adjustment, thus enhancing the overall anti-interference capability and adaptability. Weighting the classification results from multiple sensors according to their effectiveness in the current scenario allows for the comprehensive utilization of the advantages of various sensors, reducing classification errors. For example, combining the 3D information from LiDAR with the visual features of a vision sensor can more accurately distinguish between different types of vehicles and pedestrians. The resulting target classification results are based on reliable information-driven decision-making, helping intelligent driving vehicles make more accurate judgments in path planning, collision warning, and other aspects, thereby improving driving safety, reducing unnecessary sudden braking or evasive maneuvers, and enhancing passenger comfort and driving efficiency.

[0054] In this embodiment, during vehicle operation, firstly, multiple sensors on the vehicle are controlled to perceive the driving conditions ahead, obtaining multiple sensor data. These sensors are of different types. Next, the multiple sensors are controlled to pre-classify at least one target object from the sensor data, obtaining pre-classification results corresponding to each sensor data point. Then, based on the multiple sensor data, scene condition information for at least one target object is determined, characterizing the scene condition of the target object. Next, based on the scene condition information and / or the pre-classification results, target weight coefficients corresponding to the pre-classification results are determined. Finally, based on the target weight coefficients, the pre-classification results are weighted and fused to obtain a target classification result for at least one target object. This technical solution controls each sensor to operate independently during vehicle operation. Due to their different working principles and data characteristics, each sensor can capture information from different dimensions. Pre-classification processing allows for initial screening of sensor data before decision-level fusion, reducing the computational burden of subsequent fusion algorithms and minimizing the consumption of computing resources. Based on scene condition information and / or pre-classification results, the target weight coefficients corresponding to the pre-classification results are determined, and the weight coefficients of each sensor are intelligently adjusted, realizing a dynamic weight allocation mechanism for the scene. This allows for automatic adjustment of sensor data weights according to different scenes, thereby enhancing adaptability to various driving environments. Target classification results based on dynamic weights enable the intelligent driving system to rely on more reliable and comprehensive information when making decisions, improving driving safety and efficiency. The resulting target classification results are based on reliable information, helping intelligent driving vehicles make more accurate judgments in path planning, collision warning, etc., thereby improving driving safety, reducing unnecessary sudden braking or evasive maneuvers, enhancing passenger comfort and driving efficiency, and thus solving the technical problem of low accuracy in vehicle target classification in related technologies.

[0055] In the above embodiments of this application, controlling multiple sensors to perform pre-classification processing on at least one target object in the multiple sensor data to obtain pre-classification results corresponding to the multiple sensor data includes: controlling multiple sensors to perform pre-classification processing on at least one target object in the multiple sensor data based on preset pre-training algorithms corresponding to the multiple sensors to obtain pre-classification results, wherein the preset pre-training algorithms are determined in advance based on the sensor types of the multiple sensors, and each pre-classification result includes the pre-classification type of at least one target object and the confidence level of the pre-classification type.

[0056] The aforementioned pre-trained algorithms refer to algorithm models prepared and trained in advance for various sensor types, used to process sensor data and perform basic target classification. Pre-trained algorithms can be improved to suit the characteristics of specific sensors, thereby effectively classifying targets in different environments and scenarios. For visual sensors, pre-trained algorithms can employ convolutional neural network technology, trained on a large amount of image data, enabling the model to recognize and classify different target types, such as pedestrians, vehicles, and animals. It can extract features such as color, texture, and shape from images and predict target categories through machine learning. For LiDAR, pre-trained algorithms can be based on point cloud data, using features such as clustering analysis, 3D size comparison, and surface reflectivity for target classification. LiDAR pre-trained algorithms can involve point cloud processing, feature extraction, and classification models, such as segmenting the point cloud using Euclidean or density clustering, and then using a classifier to identify the target type. For millimeter-wave radar, pre-trained algorithms can utilize Doppler frequency shift and radar cross-section values ​​in the echo signal to perform preliminary target classification, such as distinguishing between moving vehicles and stationary objects. Millimeter-wave radar algorithms can consider the relative velocity and reflection characteristics of targets, and classify them through threshold settings or other machine learning models. These pre-trained algorithms can be trained on large datasets to achieve high classification accuracy and efficiency. After training, they can be used as fixed modules in real-time target classification processes.

[0057] The aforementioned pre-classification types refer to the initial classification labels given to target objects after each sensor independently processes the data. Each sensor has unique sensing capabilities and limitations, and under certain conditions, it is better at identifying certain types of objects. For example, visual sensor pre-classification labels pedestrians, vehicles, trees, etc., based on image recognition technology. LiDAR pre-classification focuses more on the three-dimensional shape and size of the target, such as labeling large vehicles, small vehicles, pedestrians, etc. Millimeter-wave radar pre-classification tends to identify motion states, such as moving vehicles and stationary vehicles, and may also label unknown targets due to its lower resolution.

[0058] The confidence level mentioned above can refer to the degree of certainty of the pre-classification algorithm regarding the pre-classification result, reflecting the probability that the algorithm believes the target belongs to a certain type. The higher the confidence level, the stronger the certainty of the classification.

[0059] As an optional implementation, suitable pre-training algorithms can be selected for different types of sensors. For example, for vision sensors, deep learning-based convolutional neural networks can be used, pre-trained on a large dataset of images with target category labels; for LiDAR, point cloud processing algorithms can be used, trained on point cloud data; for millimeter-wave radar, spectral analysis-based classification algorithms combined with machine learning models can be used. During the operation of an autonomous vehicle, vision sensors continuously capture video streams, LiDAR emits and receives laser pulses, and millimeter-wave radar emits and listens for high-frequency electromagnetic wave echoes to generate their respective sensor data. Then, the pre-trained algorithms can be used to process the sensor data in real time, identify target objects, and perform preliminary classification of each target. The output includes the pre-classified type and a confidence score for that classification, i.e., the algorithm's confidence in the correctness of the classification. The confidence score can be used for subsequent decision-level fusion to help determine how to weigh the inputs of different sensors in uncertain classification results. High-confidence results are given higher weights, while low-confidence results can be ignored or used only as a reference.

[0060] In the process described above, different types of sensors have different advantages and disadvantages under various environments. The design of the pre-training algorithm takes these differences into account, enabling it to provide at least one relatively accurate pre-classification result under various conditions, thus enhancing the stability of classification. The pre-classification process transforms the raw sensor data into abstract classification results and confidence levels, reducing the amount of data that needs to be processed during decision-level fusion, thereby reducing the demand for computing resources and processing time.

[0061] In the above embodiments of this application, determining the target weight coefficient corresponding to the pre-classification result based on scene condition information and / or pre-classification result includes: adjusting the initial weight coefficients corresponding to multiple sensors based on scene condition information and / or pre-classification result to obtain the target weight coefficient, wherein the initial weight coefficient is determined in advance based on the sensor type of multiple sensors; preferably, the initial visual weight coefficient corresponding to the visual sensor among the multiple sensors is greater than the initial lidar weight coefficient corresponding to the lidar among the multiple sensors.

[0062] The aforementioned initial weight coefficients may refer to the initial weights preset for each sensor in a multi-sensor fusion target classification method.

[0063] As an optional implementation, initial weight coefficients can be set for each sensor based on sensor type and expected performance during the initial design phase. For example, considering the good performance of visual sensors under sufficient lighting conditions, the initial weight coefficients can be set higher than those of LiDAR or other sensors. During vehicle operation, scene information, including weather, lighting conditions, target dynamics, and the surrounding environment, can be analyzed in real time. The initial weight coefficients of the sensors can be dynamically adjusted based on scene information. For example, the weight of the visual sensor can be increased during a clear day; the weights of LiDAR and millimeter-wave radar can be increased at night or in inclement weather conditions to compensate for insufficient visual information. The adjustment mechanism can include rule-based hard-coded logic or use machine learning models to predict the optimal weight configuration. Weight adjustment can be a closed-loop process, allowing real-time feedback adjustments to the weight adjustment strategy based on the current classification results to ensure long-term classification accuracy. For example, if the classification performance of LiDAR is found to be better than expected in a certain scene, the corresponding weight can be appropriately increased.

[0064] In the above process, dynamically adjusting the weighting coefficients can improve the sensor combination for specific scenarios. Even when sensor performance is limited, the advantages of other sensors can compensate, ensuring the accuracy and stability of target classification. With real-time input of scene information, it can automatically adapt to various driving environments, from sunny to rainy / foggy, from urban to wilderness, selecting appropriate sensor combinations for target classification, thus achieving intelligent and automated environmental perception. When resources are limited, the weighting adjustment mechanism can guide more efficient use of sensor resources. For example, in low-light environments, it can reduce the frequency of visual sensor usage and instead rely on LiDAR and millimeter-wave radar, achieving dynamic resource optimization.

[0065] In the above embodiments of this application, the initial weight coefficients corresponding to multiple sensors are adjusted based on scene condition information to obtain target weight coefficients, including: when the scene condition information indicates that the visual observability of at least one target object is greater than a preset observability threshold, the initial visual weight coefficient in the initial weight coefficients is increased to obtain the target weight coefficient.

[0066] The aforementioned visual observation capability refers to the effectiveness of a visual sensor in acquiring and analyzing relevant features of a target object under specific scene conditions. It can be affected by a variety of factors, including lighting conditions, weather conditions, target distance and angle, and occlusion.

[0067] The aforementioned preset observation threshold can refer to a pre-set quantitative standard for evaluating visual observation, which can be used to determine whether the visual sensor can reliably detect and identify target objects in the current scene.

[0068] As an optional implementation, onboard sensors can monitor environmental conditions in real time, including factors such as light intensity, visibility, and weather conditions, to quantify the visual observability of the target object. When analysis shows that the visual observability index in the current environment exceeds a preset observability threshold, the visual sensor can effectively capture the details of the target object. The preset observability threshold can be set based on experience, taking into account various environmental factors affecting the performance of the visual sensor. When good visual observability is confirmed, the initial weight coefficient of the visual sensor can be increased to ensure a greater weight in decision-level fusion. The adjustment range can be dynamically determined based on the difference between the actual visual observability value and the threshold. After adjusting the weights, the pre-classification results of the sensors are weighted and fused according to their respective weights to obtain the target classification. Increasing the weight of the visual sensor allows for greater emphasis on the information provided by the visual sensor during decision-making, including in scenes rich in visual features such as texture, color, and shape.

[0069] In the above process, under conditions of good visual observation, visual sensors can provide richer and more detailed image information, which helps improve the accuracy of target classification. Increasing the weighting coefficients ensures that this high-quality information can be fully utilized, reducing classification errors. Increasing the weighting coefficients of visual sensors allows for greater reliance on visual data when conditions permit, while reducing the processing of data from other types of sensors. Through dynamic weight adjustment, the effectiveness of visual sensors can be maximized when they are suitable for use, and a rapid shift to other sensors can be made when their performance is limited, maintaining the reliability of classification.

[0070] In the above embodiments of this application, the initial weight coefficients corresponding to multiple sensors are adjusted based on the pre-classification results to obtain the target weight coefficients. This includes: when the pre-classification result of the lidar in the pre-classification results indicates that at least one target object is a traffic participant, the initial lidar weight coefficient in the initial weight coefficients is increased to obtain the target weight coefficients.

[0071] The aforementioned traffic participants can refer to objects that can drive or move on the road and are of great significance to the decision-making of intelligent driving vehicles, including vehicles, pedestrians, and other moving targets.

[0072] As an optional implementation, LiDAR can emit laser pulses, receive reflected signals, and generate point cloud data. A pre-trained classification algorithm can perform preliminary identification of various targets in the point cloud, determining whether they belong to the category of traffic participants. Traffic participants can include vehicles, pedestrians, bicycles, etc. For each pre-classification result, a corresponding confidence score can be output, reflecting the LiDAR's confidence level in the classification result. If the LiDAR's pre-classification result confirms the target as a traffic participant, and the confidence score exceeds a preset threshold, the classification information can be considered relatively reliable. In this case, the initial LiDAR weight coefficient can be increased to reflect the LiDAR's significant value in the current context. The adjusted weight coefficient is stored and applied to the subsequent decision-level fusion process, ensuring that the LiDAR's classification result carries a greater weight in the final target classification decision.

[0073] In the above process, because lidar can provide the three-dimensional size and dynamic characteristics of the target, the weighting coefficient is increased when identifying traffic participants, which can more accurately determine the type of moving target. In complex traffic environments, the high resolution and all-weather capability of lidar make it a powerful tool for identifying traffic participants.

[0074] In the above embodiments of this application, the method further includes: determining the moving speed, height information and location information of at least one target object based on lidar data from multiple sensor data, wherein the location information is used to indicate whether at least one target object is within the road area where the vehicle is traveling; and determining the lidar pre-classification result in the pre-classification result based on the moving speed, height information and location information.

[0075] The aforementioned height information can refer to the target object's dimensions or position data in the vertical direction. LiDAR can accurately acquire the target's three-dimensional coordinates by emitting laser pulses and measuring the echo time.

[0076] The aforementioned location information can refer to the road area used to determine whether the target object is located in a road area directly related to vehicle travel, which may include specific areas such as main roads, intersections, and sidewalks.

[0077] As an optional implementation, LiDAR can continuously emit laser pulses, receive target reflections, and generate point cloud data. This point cloud data can be processed in real time, including but not limited to point cloud clustering, target tracking, and point cloud feature extraction. Through time-series analysis of the point cloud data, the target's movement speed can be calculated. Simultaneously, using the vertical distribution information of the point cloud, the target's height can be estimated, helping to distinguish between vehicles, pedestrians, and non-dynamic obstacles along the roadside. Based on the LiDAR's point cloud distribution, it can be determined whether the target is located within a road area, i.e., whether it is on the normal driving path of a vehicle, helping to exclude targets in irrelevant areas such as road shoulders and green belts, avoiding false alarms. Furthermore, by comprehensively considering movement speed, height information, and location information, the LiDAR's pre-classification results can be adjusted. For example, if a target is at a low height but moves quickly and is located within a road area, it may be predisposed to be classified as a pedestrian or cyclist.

[0078] In the aforementioned process, the addition of movement speed and altitude information provides the extra dimension needed for target type determination, helping to make more accurate classifications, especially in complex environments or scenarios with blurred boundaries, such as distinguishing between pedestrians and cyclists, or vehicles and other static obstacles. The determination of the vehicle's location information enables intelligent differentiation between dynamic targets and static background elements on the vehicle's path, avoiding warnings or interventions caused by misjudging roadside objects as targets, reducing false alarm rates, and improving the driving experience. Combining movement speed, altitude, and location information allows for faster identification of potential threats and timely adjustments to driving strategies, such as slowing down or changing lanes, thereby effectively improving the active safety performance of autonomous vehicles.

[0079] In the above embodiments of this application, determining the LiDAR pre-classification result in the pre-classification result based on movement speed, altitude information, and location information includes: determining the initial LiDAR pre-classification result of at least one target object based on movement speed, altitude information, and location information; determining the number of clustered obstacles corresponding to at least one target object in the LiDAR data; adjusting the initial LiDAR pre-classification result based on the number of clustered obstacles to obtain the LiDAR pre-classification result; preferably, adjusting the initial LiDAR pre-classification result based on the number of clustered obstacles to obtain the LiDAR pre-classification result includes: determining the LiDAR pre-classification result as a non-traffic participant when the initial LiDAR pre-classification result is a traffic participant and the number of clustered obstacles is greater than a preset number.

[0080] The aforementioned number of clustered obstacles can refer to the result obtained by separating and counting obstacles in point cloud data through specific algorithms in lidar data processing. When a laser pulse emitted by a lidar strikes an object's surface, it generates an echo; the collected echo data is represented as a point cloud in three-dimensional space. Point cloud data can contain a significant amount of information about obstacles in the environment, including their shape, size, and location.

[0081] The aforementioned preset quantity can refer to a threshold used to assess whether the number of obstacles identified from LiDAR data is abnormal, thereby determining the reliability of target classification. The preset quantity can be pre-set based on statistical analysis of point cloud clustering of traffic participants identified by LiDAR in typical scenarios.

[0082] As an optional implementation, the point cloud data generated by LiDAR can be processed using clustering algorithms, such as region growing, to segment independent obstacle clusters, thus identifying multiple possible target objects. For each obstacle cluster, the changes in the point cloud over time can be further analyzed to calculate the movement speed. Simultaneously, the target's height information can be estimated through the vertical distribution of the point cloud, providing a basis for subsequent type determination. The spatial distribution of the point cloud can determine whether the target is located inside the road or outside the curb. The number of point clouds in each obstacle cluster can be counted. If the number of point clouds in a cluster exceeds a preset number, and the initial classification identifies the target as a traffic participant, the classification can be reconsidered. A high number of point clouds may indicate complex object structures or non-traffic participant forms, such as dense flocks of birds or tall objects. These situations can lead to misclassification results; therefore, the LiDAR preclassification result is determined to be a non-traffic participant.

[0083] In the above process, by introducing point cloud data, it is possible to adjust for initial classification errors caused by large target volume or complex structure. When facing non-traffic participants, it avoids misclassifying large obstacles or densely packed non-traffic participants as traffic participants. The statistical analysis of point cloud data can assist in making more robust classification decisions in complex environments. When environmental factors such as rain, fog, and changes in lighting affect the performance of visual sensors, this additional judgment by LiDAR becomes even more important.

[0084] In the above embodiments of this application, the method further includes: when the target classification result of at least one target object is a traffic participant, and at least one target object is not in the road area where the vehicle is traveling, adjusting the target classification result of at least one target object to a non-traffic participant.

[0085] As an optional implementation, this method can update road boundary information around the vehicle in real time, identifying lane lines, curbs, etc., and defining the normal driving area for the vehicle. It can utilize multi-sensor fusion data and precise point cloud data from LiDAR to accurately locate the spatial coordinates of each target object and determine whether it is within a predefined road area. When a target is identified as a traffic participant, such as a pedestrian, vehicle, or animal, but its location is found to be significantly off-road or outside the curb, the target will be considered misclassified. An automatic classification adjustment mechanism can be triggered to re-label such targets as non-traffic participants, excluding them from immediate driving decisions. The adjusted classification results can be fed back to the learning module for subsequent algorithm training and improvement, ensuring more accurate classification in similar scenarios.

[0086] In the aforementioned process, cross-validation of location information effectively avoids misidentifying non-immediately threatening curb targets as traffic participants, thereby reducing false warnings and preventing unnecessary driver stress and interference. When planning routes or executing obstacle avoidance maneuvers, the focus shifts to the actual traffic participants on the road, improving the accuracy and efficiency of driving decisions. In complex and changing driving environments, such as construction zones and roadside parks, this mechanism ensures avoidance of the influence of curb objects, maintains a clear perception of road conditions, and enhances environmental adaptability and stability.

[0087] In the above embodiments of this application, the method further includes: when the confidence level corresponding to the target classification result of at least one target object is less than a preset confidence level, determining the target classification result of at least one target object based on the historical target classification results of at least one target object.

[0088] The aforementioned pre-set confidence level can refer to a threshold used to measure the reliability of the target classification result. Confidence level reflects the degree of certainty that an intelligent system has regarding a particular classification.

[0089] The aforementioned historical target classification results can refer to records of targets identified and classified by sensors at previous moments. Intelligent driving systems store classification information over a period of time, which may include the type of target, as well as the environmental conditions and classification confidence level at that time.

[0090] As an optional implementation, when classifying targets based on the results of multi-sensor fusion, a confidence level is assigned to each classification result. If the confidence level of a target's classification is lower than a preset confidence level, historical classification records for that target over a recent period can be queried, and classification results from the last few seconds can be considered to capture the target's dynamic changing trends. The consistency of historical classification results determines whether to adopt them. If past classification results consistently point to the same category and the confidence level is relatively high, then this historical classification is considered reliable. If the current classification has a low confidence level while historical classifications have high consistency and sufficient confidence, then the historical classification result can be used as the current target's classification result to ensure the continuity and stability of classification.

[0091] In the aforementioned process, by introducing a historical classification result correction mechanism, decisions can be made based on previous stable classifications even when the current classification uncertainty is high. This avoids rapid fluctuations in classification results and improves the continuity and stability of driving decisions. Under certain extreme conditions, such as transient sensor failures or a decrease in classification confidence due to sudden environmental changes, historical classification results can serve as a backup information source to help maintain basic classification functionality, enhancing the ability to respond to emergencies and robustness. This mechanism reduces classification errors or frequent alarms caused by instantaneous drops in confidence, increases user trust in the autonomous driving system, and improves comfort and safety during intelligent driving.

[0092] The technical solution proposed in this application is described below with reference to an optional embodiment. This application proposes a target classification method based on multi-sensor fusion, belonging to the field of intelligent driving perception technology, specifically involving multi-sensor target classification technology, including camera, LiDAR, and millimeter-wave radar fusion. The requirement of intelligent driving perception systems is to achieve high-precision classification of road targets, including vehicles, pedestrians, cyclists, and static obstacles, to support the safe operation of decision-making modules such as path planning. Target classification technology is mainly divided into two major technical paths: single-sensor independent classification and multi-sensor fusion classification. Multi-sensor fusion classification further includes data-level fusion classification, feature-level fusion classification, and decision-level fusion classification. This application, through decision-level fusion technology, assigns appropriate weights to different sensor characteristics under different scenario conditions, designing an efficient, robust, and easy-to-deploy multi-sensor fusion classification method. Its advantage lies in balancing cost while ensuring classification accuracy.

[0093] This application, based on pre-classification results, uses weighted voting and backoff rules to address at least the following technical issues: First, it addresses the insufficient robustness of single-sensor classification. Existing single-sensor classifications suffer from high misclassification rates in backlight, rain, fog, and occlusion scenarios. This application improves classification accuracy in complex scenes by fusing pre-classification results from multiple sensors, leveraging the complementary advantages of camera texture, LiDAR 3D, and millimeter-wave radar all-weather capabilities. Second, it addresses the low efficiency and high hardware cost of data-level or feature-level fusion. Data-level fusion suffers from latency, and feature-level fusion demands significant computing power. This application directly fuses sensor pre-classification results, eliminating the need to process raw data, making it compatible with automotive-grade microcontrollers, and reducing hardware costs. Third, it addresses the issues of fixed weights and lack of fallback rules in decision-level fusion. Decision-level fusion uses equal or fixed weights, lacking adjustment strategies for edge scenarios. This application designs dynamic scene weights to address the problem of coarse-grained determination of unknown targets.

[0094] This application is based on the pre-classification results independently output by each sensor, including camera, LiDAR, and millimeter-wave radar. It designs a dynamic scene weight allocation mechanism, fusing the pre-classification results from each sensor through a weighted voting algorithm to obtain a preliminary target classification result. A fallback rule is designed to reuse historical classification results or initiate supplementary judgment logic for edge scenarios such as partial occlusion or missing sensor data. This application solves the target misclassification problem of a single sensor in special scenarios by leveraging the complementary advantages of multiple sensors; it adopts a dynamic weight allocation mechanism to adapt to different scene changes, enhancing edge scene processing capabilities; and it eliminates the need to process raw data or high-level features, balancing efficiency and cost.

[0095] This application presents a multi-sensor fusion target classification method, which includes sensor pre-classification data acquisition, deployment of three types of sensors (camera, LiDAR, and millimeter-wave radar), independent pre-classification algorithm operation for each sensor, and output of target classification results and confidence scores. It filters effective measurement data, processing only measurement data within the last 0.5 seconds to ensure classification is based on the most recent sensor information. Weighted voting is performed according to sensor type, iterating through effective measurement data and calculating different weights based on sensor type, such as camera, LiDAR, etc. The calculation methods for different sensor weights are as follows: Camera measurement weight calculation rule: First, the base weight for the camera is designed to be 2; second, if the identified target type is a large vehicle or the trajectory is partially observed, the weight is multiplied by 2. LiDAR measurement weight calculation rule: First, the base weight for the LiDAR is designed to be 1; second, if the identified target type is a road participant, such as a pedestrian, cyclist, or vehicle, the weight is multiplied by 3.

[0096] LiDAR measurement type determination: If the type is pedestrian and the number of clustered obstacles identified is greater than 80, the type is set to static object; if the type is pedestrian or cyclist and the number of obstacles is greater than 300, the type is set to static object; if the type is static target or location target, further determination is made regarding its proximity to the roadside: if it is close to the roadside, the object type is set to static target; if there is a moving millimeter-wave radar measurement, the object type remains unknown. Initial type is static target or unknown target, further type determination: If the target is within the detection range: if the speed ≤ static threshold: 1, return to static object; if the height of the object measured by the LiDAR is < minimum movable height: 0.5m, return to static object; if the map manager is not empty and the object is not in the lane, return to static object; if none of the above conditions are met, return to unknown object. If the target is not within the detection range: if the speed > static threshold: 1, return to unknown object; if the speed ≤ static threshold: 1, return to static object.

[0097] The voting results are determined, and a weighted voting fusion calculation is performed. For the same target, the pre-classification results and normalized weights of the three types of sensors are extracted, and the pre-classification types of each sensor are voted on according to their weights. The type with the highest votes is selected as the target type from the voting results. Rapid type switching is avoided. If the target type is different from the current type and the type has been switched within 10 seconds, the type is not updated to prevent frequent fluctuations. Special scenario adjustments are made to the voting results to handle edge cases. If the identified trajectory target is judged as a fence, it is corrected to an obstacle; if the pedestrian trajectory is outside the curb and not on a zebra crossing, it is marked as a non-road area. A rollback rule is implemented: for trajectory types that are only partially observed, the previous classification result is used directly; adjacent lane vehicle type handling: if the distance between adjacent lanes is close (e.g., ≤7.5m) and it has historically been classified as a vehicle, the vehicle type is maintained.

[0098] According to another aspect of the present invention, a vehicle target classification device is also provided. This device can execute the vehicle target classification method of the above embodiments. The specific implementation method and preferred application scenarios are the same as those of the above embodiments, and will not be described in detail here.

[0099] Figure 2 This is a schematic diagram of a vehicle target classification device according to an embodiment of this application, such as... Figure 2 As shown, the device includes the following: a first control module 202, a second control module 204, a first determination module 206, a second determination module 208, and a fusion module 210.

[0100] The system comprises the following modules: a first control module 202, used to control multiple sensors on the vehicle to perceive the driving conditions ahead of the vehicle during vehicle operation and obtain multiple sensor data, wherein the multiple sensors are of different types; a second control module 204, used to control the multiple sensors to perform pre-classification processing on at least one target object in the multiple sensor data, and obtain pre-classification results corresponding to the multiple sensor data; a first determination module 206, used to determine scene condition information of at least one target object based on the multiple sensor data, wherein the scene condition information is used to characterize the scene condition in which at least one target object is located; a second determination module 208, used to determine the target weight coefficient corresponding to the pre-classification result based on the scene condition information and / or the pre-classification result; and a fusion module 210, used to perform weighted fusion of the pre-classification results based on the target weight coefficient to obtain the target classification result of at least one target object.

[0101] The second control module is used to control multiple sensors and perform pre-classification processing on at least one target object in the multiple sensor data based on the preset pre-training algorithms corresponding to the multiple sensors, so as to obtain pre-classification results. The preset pre-training algorithms are determined in advance based on the sensor types of the multiple sensors, and each pre-classification result includes the pre-classification type of at least one target object and the confidence level of the pre-classification type.

[0102] The second determining module is used to adjust the initial weight coefficients corresponding to multiple sensors based on scene condition information and / or pre-classification results to obtain target weight coefficients. The initial weight coefficients are determined in advance based on the sensor types of the multiple sensors. Preferably, the initial visual weight coefficient corresponding to the visual sensor among the multiple sensors is greater than the initial lidar weight coefficient corresponding to the lidar among the multiple sensors.

[0103] The second determining module is used to increase the initial visual weight coefficient in the initial weight coefficient to obtain the target weight coefficient when the scene condition information indicates that the visual observability of at least one target object is greater than a preset observability threshold.

[0104] The second determining module is used to increase the initial lidar weight coefficient in the initial weight coefficient when the lidar pre-classification result in the pre-classification result indicates that at least one target object is a traffic participant, so as to obtain the target weight coefficient.

[0105] The second control module is used to determine the moving speed, height information and location information of at least one target object based on lidar data from multiple sensor data, wherein the location information is used to indicate whether at least one target object is within the road area where the vehicle is traveling; and to determine the lidar pre-classification result in the pre-classification result based on the moving speed, height information and location information.

[0106] The second control module is used to determine the initial LiDAR pre-classification result of at least one target object based on the movement speed, height information, and location information; determine the number of clustered obstacles corresponding to at least one target object in the LiDAR data; adjust the initial LiDAR pre-classification result based on the number of clustered obstacles to obtain the LiDAR pre-classification result; preferably, adjusting the initial LiDAR pre-classification result based on the number of clustered obstacles to obtain the LiDAR pre-classification result includes: if the initial LiDAR pre-classification result is a traffic participant and the number of clustered obstacles is greater than a preset number, determine the LiDAR pre-classification result as a non-traffic participant.

[0107] The fusion module is used to adjust the target classification result of at least one target object to non-traffic participant when the target classification result of at least one target object is traffic participant and at least one target object is not in the road area where the vehicle is traveling.

[0108] The fusion module is used to determine the target classification result of at least one target object based on the historical target classification results of at least one target object when the confidence level corresponding to the target classification result of at least one target object is less than the preset confidence level.

[0109] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0110] Embodiments of this application also provide a vehicle, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods described in various embodiments of this application when it runs.

[0111] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0112] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0113] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0114] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this application.

[0115] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0117] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0118] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0119] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0120] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A vehicle target classification method, characterized in that, include: During the vehicle's operation, multiple sensors on the vehicle are controlled to perceive the driving conditions ahead of the vehicle and obtain multiple sensor data. Control the plurality of sensors to perform pre-classification processing on at least one target object in the data from the plurality of sensors, and obtain the pre-classification results corresponding to the data from the plurality of sensors respectively; Based on the data from the multiple sensors, scene condition information of the at least one target object is determined, wherein the scene condition information is used to characterize the scene condition in which the at least one target object is located; Based on the scenario situation information and / or the pre-classification result, determine the target weight coefficient corresponding to the pre-classification result; Based on the target weight coefficient, the pre-classification results are weighted and fused to obtain the target classification result of the at least one target object.

2. The vehicle target classification method according to claim 1, characterized in that, Controlling the plurality of sensors to perform pre-classification processing on at least one target object in the sensor data, obtaining pre-classification results corresponding to the sensor data respectively, including: Controlling the plurality of sensors, and based on preset pre-training algorithms corresponding to the plurality of sensors respectively, performing pre-classification processing on at least one target object in the data from the plurality of sensors to obtain the pre-classification result, wherein the preset pre-training algorithm is determined in advance based on the sensor type of the plurality of sensors, and each pre-classification result includes the pre-classification type of the at least one target object and the confidence level of the pre-classification type.

3. The vehicle target classification method according to claim 1, characterized in that, Based on the scenario information and / or the pre-classification result, determine the target weight coefficient corresponding to the pre-classification result, including: Based on the scene condition information and / or the pre-classification results, the initial weight coefficients corresponding to the multiple sensors are adjusted to obtain the target weight coefficients, wherein the initial weight coefficients are determined in advance based on the sensor types of the multiple sensors; Preferably, the initial visual weight coefficient corresponding to the visual sensor among the plurality of sensors is greater than the initial lidar weight coefficient corresponding to the lidar among the plurality of sensors.

4. The vehicle target classification method according to claim 3, characterized in that, Based on the scene condition information, the initial weight coefficients corresponding to the multiple sensors are adjusted to obtain the target weight coefficients, including: When the scene condition information indicates that the visual observability of at least one target object is greater than a preset observability threshold, the initial visual weight coefficient in the initial weight coefficient is increased to obtain the target weight coefficient.

5. The vehicle target classification method according to claim 3, characterized in that, Based on the pre-classification results, the initial weight coefficients corresponding to the multiple sensors are adjusted to obtain the target weight coefficients, including: If the lidar pre-classification result in the pre-classification result indicates that the at least one target object is a traffic participant, the initial lidar weight coefficient in the initial weight coefficient is increased to obtain the target weight coefficient.

6. The vehicle target classification method according to claim 1, characterized in that, The method further includes: Based on the lidar data from the multiple sensor data, the moving speed, height information, and location information of the at least one target object are determined, wherein the location information is used to indicate whether the at least one target object is within the road area where the vehicle is traveling; Based on the moving speed, the altitude information, and the location information, the lidar pre-classification result in the pre-classification result is determined.

7. The vehicle target classification method according to claim 6, characterized in that, Based on the movement speed, the altitude information, and the location information, the lidar pre-classification result in the pre-classification result is determined, including: Based on the moving speed, the altitude information, and the location information, determine the initial lidar pre-classification result of at least one target object; Determine the number of clustered obstacles corresponding to the at least one target object in the lidar data; Based on the number of clustered obstacles, the initial lidar pre-classification result is adjusted to obtain the lidar pre-classification result. Preferably, the initial lidar pre-classification result is adjusted based on the number of clustered obstacles to obtain the lidar pre-classification result, including: if the initial lidar pre-classification result is a traffic participant and the number of clustered obstacles is greater than a preset number, the lidar pre-classification result is determined to be a non-traffic participant.

8. The vehicle target classification method according to any one of claims 1 to 7, characterized in that, The method further includes: If the target classification result of at least one target object is a traffic participant, and the at least one target object is not within the road area where the vehicle is traveling, the target classification result of the at least one target object shall be adjusted to a non-traffic participant.

9. The vehicle target classification method according to any one of claims 1 to 7, characterized in that, The method further includes: If the confidence level corresponding to the target classification result of the at least one target object is less than the preset confidence level, the target classification result of the at least one target object is determined based on the historical target classification results of the at least one target object.

10. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, executes the vehicle target classification method according to any one of claims 1 to 9.

Images