Sensing method and unmanned vehicle
By setting up a dedicated mapping relationship and semantic perception model for lidar, the problem of noise misjudgment in lidar point cloud data in high-altitude and cold environments was solved, achieving accurate target recognition and reliable semantic perception in harsh environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-26
- Publication Date
- 2026-04-07
AI Technical Summary
In mining scenarios at high altitudes and cold regions, the perception performance of lidar is affected by extreme weather conditions such as low temperatures, strong winds, blizzards, dense fog, and freezing rain, resulting in abnormal noise in point cloud data, misjudging obstacles, and affecting the normal operation of unmanned vehicles.
By setting a unique mapping relationship for each type of LiDAR, the original intensity information in the point cloud data is mapped to a standard intensity range, and then processed through a semantic perception model to reduce noise interference and ensure the reliability of semantic perception.
It can accurately identify targets in harsh environments, reduce false detection rates, ensure the reliability and accuracy of semantic perception results, and adapt to complex and ever-changing mine and high-altitude environments.
Smart Images
Figure CN121805971A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of autonomous driving technology, and more particularly to a perception method and an autonomous vehicle. Background Technology
[0002] Mining environments are a crucial application area for autonomous driving technology, characterized by their unique and demanding working conditions. This is particularly true in high-altitude and frigid regions, where the harsh environment significantly impacts the perception performance of lidar. These areas frequently experience extreme weather conditions such as low temperatures, strong winds, blizzards, dense fog, and freezing rain. Low temperatures lead to attenuation of the lidar sensor's laser emission power and a decrease in detector sensitivity. Strong winds and blizzards directly interfere with the laser transmission path, while dense fog and freezing rain cause laser scattering and refraction. These factors result in abnormal noise in the point cloud data acquired by the lidar.
[0003] In related technologies, abnormal noise in point cloud data will cause serious false detections in the semantic perception process, misjudging dense fog, blizzards, ice accumulation, gravel, etc. as obstacles such as vehicles, cones, and retaining walls, thus affecting the normal operation of unmanned vehicles. Summary of the Invention
[0004] To overcome the problems existing in related technologies, this disclosure provides a perception method and an unmanned vehicle.
[0005] According to a first aspect of the present disclosure, a sensing method is provided, comprising: Acquire point cloud data collected by lidar, wherein the point cloud data includes: point cloud location information and original intensity information; According to the mapping relationship corresponding to the lidar, the original intensity information in the point cloud data corresponding to the same semantic type is mapped to the standard intensity range corresponding to the semantic type to obtain the standard intensity information. The mapping relationship is a pre-determined mapping relationship between the intensity acquisition range of the lidar and the standard intensity range. The semantic type includes at least the target obstacle type and the noise type. The point cloud location information and the standard intensity information are input into the semantic perception model to obtain the semantic perception result output by the semantic perception model.
[0006] In some embodiments, before mapping the original intensity information corresponding to the same semantic type in the point cloud data to the standard intensity range corresponding to that semantic type according to the mapping relationship corresponding to the lidar, and obtaining the standard intensity information, the method further includes: Obtain the original intensity range of the LiDAR for different semantic type objects; wherein, the original intensity range is the range of original intensity information of the point cloud data of the corresponding semantic type object; For each semantic type object, establish a mapping relationship between the original intensity range and the standard intensity range corresponding to that semantic type object; The step of mapping the original intensity information of the point cloud data corresponding to the same semantic type to the standard intensity range corresponding to the semantic type according to the mapping relationship of the lidar, to obtain the standard intensity information, includes: Based on the original intensity range in which the original intensity information is located, determine the target semantic type corresponding to the original intensity information; If the target semantic type can be determined, the original intensity information is mapped to the corresponding value in the standard intensity range based on the mapping relationship corresponding to the target semantic type, and used as the standard intensity information.
[0007] In some embodiments, before establishing the mapping relationship between the original intensity interval and the standard intensity interval corresponding to the semantic object, the method further includes: Determine the standard intensity range of the lidar for objects of different semantic types; Among them, the standard intensity range of different lidars used to collect point cloud data is the same for the same semantic type of object; In the case where there is partial overlap between the original intensity intervals corresponding to multiple semantic types, there is an equal proportion of overlap between the standard intensity intervals corresponding to the multiple semantic types.
[0008] In some embodiments, the method further includes: If the target semantic type cannot be determined based on the original intensity information, the original intensity information is mapped to a preset intensity value and used as standard intensity information. The preset intensity value is a value outside the standard intensity range.
[0009] In some embodiments, the method further includes: For each point in the point cloud data, if the semantic correction condition is met, the semantic type of the point is corrected to the semantic type corresponding to the original intensity information of the point. The semantic correction conditions include: The original intensity information at this point corresponds to a unique semantic type; and The semantic type of the point shown in the semantic perception result is inconsistent with the semantic type corresponding to the original intensity information of the point.
[0010] In some embodiments, the semantic awareness model is trained in the following manner: Acquire sample point cloud data collected by various lidar systems, wherein the sample point cloud data includes: sample location information and sample intensity information; The sample intensity information corresponding to the same semantic type collected by each lidar is mapped to the standard intensity range corresponding to that semantic type to obtain the standard sample intensity information. For each sample point cloud data, the sample location information and standard sample intensity information corresponding to the sample point cloud data are input into the semantic perception model to be trained, and the model parameters of the semantic perception model are adjusted based on the semantic perception results output by the semantic perception model to complete the training of the semantic perception model.
[0011] In some embodiments, the method further includes: Extract the first point cloud information within at least one foreground target bounding box from the sample point cloud data; The first point cloud information is inserted into a first preset position in any sample point cloud data to generate new sample point cloud data; wherein, the preset position is the location where noise points are concentrated in the sample point cloud data; The semantic perception model is trained based on new sample point cloud data.
[0012] In some embodiments, the method further includes: Extract second point cloud information from at least some of the noise points in the sample point cloud data; The second point cloud information is inserted into the second preset position in any sample point cloud data to generate new sample point cloud data; wherein, the second preset position is the position in the sample point cloud data where the distance between it and the vehicle is less than a preset distance threshold; The semantic perception model is trained based on new sample point cloud data.
[0013] In some embodiments, before inserting the second point cloud information into a second preset position in arbitrary sample point cloud data to generate new point cloud data, the method further includes: Adjust the point cloud density of the second point cloud information to generate new second point cloud information; The step of inserting the second point cloud information into the second preset position in any sample point cloud data to generate new sample point cloud data includes: Insert the new second point cloud information into the second preset position in any sample point cloud data to generate new sample point cloud data.
[0014] According to a second aspect of this disclosure, an unmanned vehicle is provided for performing the perception method described in the first aspect.
[0015] The solution provided in this disclosure can acquire point cloud data collected by a lidar, including point cloud location information and raw intensity information. According to the mapping relationship corresponding to the lidar, the raw intensity information in the point cloud data corresponding to the same semantic type is mapped to a standard intensity range corresponding to that semantic type, obtaining standard intensity information. The mapping relationship is a pre-determined mapping relationship between the intensity acquisition range of the lidar and the standard intensity range. The semantic type includes at least target obstacle type and noise type. The point cloud location information and standard intensity information are input into a semantic perception model to obtain the semantic perception result output by the semantic perception model. This disclosure uses intensity information standardized according to semantic type as input to the semantic perception model, which can reduce the interference of noise in the point cloud data on semantic perception, ensuring that the semantic perception model can still accurately identify targets in harsh environments and guaranteeing the reliability of the semantic perception result. Attached Figure Description
[0016] Figure 1 A flowchart illustrating a sensing method according to an embodiment of this disclosure is shown.
[0017] Figure 2 The diagram shows a flowchart of a semantic awareness model training method according to an embodiment of this disclosure.
[0018] Figure 3 The diagram shows a flowchart of a sample enhancement method according to an embodiment of this disclosure.
[0019] Figure 4 A flowchart illustrating another sample enhancement method in an embodiment of this disclosure is shown.
[0020] Figure 5 A schematic diagram of the structure of a sensing device according to an embodiment of the present disclosure is shown. Detailed Implementation
[0021] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0022] First, this disclosure provides a perception method that can be executed by any unmanned vehicle.
[0023] Figure 1 This diagram illustrates a flow chart of a sensing method according to an embodiment of the present disclosure, such as... Figure 1 As shown, the method includes the following steps S101 and S102.
[0024] S101 acquires point cloud data collected by the lidar.
[0025] The point cloud data includes point cloud location information and raw intensity information.
[0026] In some embodiments, at least one lidar can be deployed on the autonomous vehicle to collect point cloud data of the surrounding environment in real time. The point cloud data can contain two core pieces of information: first, point cloud location information, i.e., three-dimensional spatial coordinates (x, y, z), used to locate the spatial position of target objects; second, raw intensity information, which is generated by the interaction between the laser beam emitted by the lidar and the target object (such as vehicles, cones, barriers, and noise such as dust, fog, and blizzard scattering points). Its value is related to the material and surface friction coefficient of the target object, and will fluctuate due to factors such as low temperature, strong wind, and dense fog in high-altitude and cold environments.
[0027] Among them, at least one lidar can be a single lidar, multiple lidars of the same type, or multiple lidars of different types; this disclosure does not limit this.
[0028] S102, according to the mapping relationship corresponding to the lidar, the original intensity information in the point cloud data corresponding to the same semantic type is mapped to the standard intensity range corresponding to the semantic type to obtain the standard intensity information.
[0029] The mapping relationship is a pre-determined mapping relationship between the intensity acquisition range of the lidar and the standard intensity range, and the semantic type includes at least the target obstacle type and the noise type.
[0030] In some embodiments, the conversion from raw intensity information to standard intensity information can be completed according to a pre-configured mapping relationship for the lidar. The mapping relationship corresponds to the type of lidar, and each type of lidar has its own exclusive mapping relationship based on its intensity information characteristics. That is, for the intensity acquisition range of each type of lidar, a correspondence is established between it and the standard intensity range corresponding to each semantic type. Among them, the semantic types at least cover target obstacle types (such as vehicles, cones, retaining walls) and noise types (such as dust, fog, blizzard scattering points).
[0031] In some embodiments, the original intensity information of each point in the point cloud data can be traversed to identify its corresponding semantic type. The original intensity information of each semantic type can be mapped to the standard intensity range exclusive to that semantic type to obtain the standardized standard intensity information, thereby providing a unified representation of the intensity information according to the semantic type.
[0032] S103, input the point cloud location information and standard intensity information into the semantic perception model to obtain the semantic perception result output by the semantic perception model.
[0033] In some embodiments, the point cloud location information and the transformed standard intensity information can be used as joint inputs to a pre-trained semantic perception model (e.g., a model used to perform 3D detection or 3D segmentation tasks). The semantic perception model learns the correlation between standard intensity information and semantic type, combines location information to perform feature fusion and inference, and finally outputs the semantic perception result.
[0034] For example, for a 3D detection model, the semantic perception result may include information such as the bounding box information, geometric features, and semantic type of the target object in the point cloud data. For a 3D segmentation model, the semantic perception result may include the semantic type of each point in the point cloud data and semantic region division information.
[0035] This embodiment of the disclosure sets a unique mapping relationship for each type of LiDAR, which maps the raw intensity information in the point cloud data collected by the LiDAR to a unified standard intensity range. It also enables semantic perception of point cloud data collected by different types of LiDAR through the same semantic perception model, improving the versatility of the semantic perception method. Furthermore, using the intensity information standardized according to semantic type as input to the semantic perception model reduces the interference of noise in the point cloud data on semantic perception, ensuring that the semantic perception model can accurately identify targets even in harsh environments and guaranteeing the reliability of the semantic perception results.
[0036] To facilitate understanding, the method of establishing the mapping relationship will be explained in detail below.
[0037] In some embodiments, before executing S102, the mapping relationship can be established as follows: obtain the original intensity range of the LiDAR for different semantic type objects; for each semantic type object, establish a mapping relationship between the original intensity range and the standard intensity range corresponding to that semantic type object. Here, the original intensity range is the range of original intensity information of the point cloud data for the corresponding semantic type object.
[0038] For example, for each type of lidar, sufficient sample point cloud data can be collected in actual operating scenarios (e.g., mining operations in high-altitude and cold regions). Point sets corresponding to different semantic types can be selected through manual annotation or automatic classification tools, and the distribution range of the original intensity information of each point set can be statistically analyzed to determine the original intensity range of each semantic type.
[0039] For example, if the original intensity information for points with the semantic type "cone" in the point cloud data collected by a certain lidar is all in the range of 80 to 150, then the original intensity range for the lidar corresponding to the semantic type "cone" is 80 to 150. Similarly, if the original intensity information for points with the semantic type "fog" in the point cloud data collected by the lidar is all in the range of 10 to 40, then the original intensity range for the lidar corresponding to the semantic type "fog" is 10 to 40.
[0040] For each semantic type, a mapping relationship between its original intensity range and a preset standard intensity range can be established. The preset standard intensity range is independent of the type of LiDAR; for the same semantic type, all types of LiDARs correspond to the same standard intensity range, ensuring a consistent logic for intensity information conversion within the same semantic type.
[0041] Continuing with the previous example, if the standard intensity range of the "cone" is 40~70 and the standard intensity range of the "fog" is 5~20, then for the "cone", a mapping relationship of 80~150 to 40~70 can be established for the lidar, while for the "fog", a mapping relationship of 10~40 to 5~20 can be established for the lidar.
[0042] Therefore, when performing the standard intensity information mapping step described in S102 above, the target semantic type corresponding to the original intensity information can be determined based on the original intensity interval in which the original intensity information is located. If the target semantic type can be determined, the original intensity information is mapped to the corresponding value in the standard intensity interval based on the mapping relationship corresponding to the target semantic type, and this value is used as the standard intensity information.
[0043] In other words, for the raw intensity information in point cloud data, the raw intensity interval library corresponding to the LiDAR used to collect the point cloud data can be queried. If the raw intensity information falls within the raw intensity interval of a certain semantic type (for example, the raw intensity information of a point in the point cloud data is 120, which falls within the "cone" interval of 80~150), then its target semantic type is determined to be "cone". At this time, based on the corresponding mapping relationship, 120 can be converted into the corresponding value within the standard intensity interval of 40~70 (for example, 60), which is used as the standard intensity information of the point cloud.
[0044] This embodiment enhances the distinguishability of intensity features across different semantic types by establishing mapping relationships through classification, providing more reliable feature inputs for subsequent model inference and further reducing the probability of false detections. Simultaneously, since different types of LiDAR correspond to the same standard intensity range, a unified standard can be provided for downstream inputs, improving the versatility of intensity information.
[0045] In some embodiments, before establishing the mapping relationship between the original intensity range and the standard intensity range corresponding to a semantic object, the standard intensity range of the LiDAR for different semantic type objects can be determined. Specifically, different LiDARs used to collect point cloud data have the same standard intensity range for the same semantic type object. In cases where there is partial overlap between the original intensity ranges corresponding to multiple semantic types, there is also an equal proportion of overlap between the standard intensity ranges corresponding to multiple semantic types.
[0046] As described in the previous embodiment, the standard intensity range is uniformly set for various LiDARs by combining the intensity feature patterns of common semantic types in autonomous driving. For example, the standard intensity range is uniformly set as 50~80 for "vehicle", 60~90 for "wall", 10~30 for "dust", and 5~25 for "blizzard scattering point", ensuring that the original intensity information of the same semantic type has a consistent intensity representation standard in different types of LiDARs after mapping.
[0047] If the original intensity ranges of multiple semantic types of a LiDAR overlap, the overlap ratio of the standard intensity range can be maintained consistent with that of the original ranges when setting the standard intensity range. For example, if the original intensity range of 60-100 for "retaining wall" partially overlaps with the original intensity range of 80-140 for "vehicle," and the overlap range of the original ranges is 80-100, accounting for 50% of the original intensity range of "retaining wall" and 33% of the original intensity range of "vehicle," then the standard intensity range corresponding to "retaining wall" can be set to 50-90, and the standard intensity range corresponding to "vehicle" can be set to 70-130. That is, the same overlap ratio as the original intensity ranges is maintained, thereby reproducing the intensity characteristics of the LiDAR and ensuring the logical consistency of intensity mapping.
[0048] For example, in order to improve the types of lidar compatible with the sensing method of this disclosure, when the original intensity range of any lidar applied to the sensing method has the aforementioned partial overlap, the standard intensity range corresponding to various lidars can be uniformly set to include the partially overlapping range.
[0049] By ensuring the consistency of the intensity distribution pattern before and after mapping, the embodiments disclosed herein can improve the generalization ability of the sensing method and ensure that the characteristics of the lidar can be maintained after mapping, thereby adapting to the complex and ever-changing environments of mines and high-altitude and cold regions.
[0050] In some embodiments, if the target semantic type cannot be determined based on the original intensity information, the original intensity information is mapped to a preset intensity value and used as standard intensity information. The preset intensity value is a value outside the standard intensity range.
[0051] Understandably, if there is no significant regularity between the original intensity information distribution of the LiDAR and the semantic type, or if the original intensity information of the LiDAR does not fall within the original intensity range of any known semantic type, it can be determined that the target semantic type cannot be determined. In this case, the original intensity information for which the target semantic type cannot be determined can be uniformly mapped to a preset intensity value (e.g., -1). This preset intensity value does not fall within the standard intensity range of all preset semantic types, avoiding confusion with the intensity values of normal semantic types, and serves as a feature identifier for the model to identify abnormal point clouds.
[0052] This disclosure provides a clear processing solution for point cloud data with irregular intensity or abnormal intensity data, avoiding invalid intensity information from interfering with model inference, and improving the adaptability of the technical solution to harsh environments, especially suitable for scenarios where the performance of some equipment is degraded in high-altitude and cold environments.
[0053] In some embodiments, for 3D segmentation tasks, after outputting semantic perception results, semantic correction can be performed on the semantic types in the semantic perception results. That is, for each point in the point cloud data, if the semantic correction conditions are met, the semantic type of the point can be corrected to the semantic type corresponding to the original intensity information of the point.
[0054] The semantic correction conditions include: the original intensity information of the point corresponds to a unique semantic type, and the semantic type of the point shown in the semantic perception result is inconsistent with the semantic type corresponding to the original intensity information of the point.
[0055] For example, a point in point cloud data might have an initial intensity of 25, falling only within the initial intensity range of 10-30 for "dust," without any other semantic type ranges covered. When performing semantic perception on this point cloud data using the perception method provided in this disclosure, the semantic type corresponding to this point in the output semantic perception result is "vehicle." In this case, there is a conflict between the semantic type obtained based on the intensity pattern and the semantic type obtained through the semantic perception model. To resolve this conflict, the semantic type of this point can be corrected to "dust," and the semantic perception result can be updated, completing the post-processing correction of segmentation misdetection.
[0056] This embodiment establishes a verification relationship between the semantic type corresponding to the original intensity information and the semantic type corresponding to the semantic perception result. This directly corrects segmentation misjudgments caused by noise interference and intensity fluctuations in high-altitude and cold environments (such as misjudging snow scattering points or fog points as obstacles). This further reduces the false detection rate and improves the accuracy of the perception results from the post-processing stage. Since there is an inherent correlation between the original intensity information and the semantic type, it provides an objective verification basis for the segmentation results, unaffected by the degradation of point cloud data quality caused by harsh conditions such as low temperatures and strong winds, ensuring the reliability of semantic correction.
[0057] The perception method provided in this disclosure has been described in detail above. The training method for the semantic perception model in this disclosure will now be illustrated by example. Since the principle of the semantic perception model training method is similar to that of the perception method described above, repeated details will not be elaborated upon.
[0058] Please refer to Figure 2 , Figure 2 This diagram illustrates a flowchart of a semantic awareness model training method according to an embodiment of this disclosure. Figure 2 As shown, the method includes the following steps.
[0059] S201 acquires sample point cloud data collected by various lidar systems.
[0060] The sample point cloud data includes: sample location information and sample intensity information.
[0061] In some embodiments, sample point cloud data collected by various types and models of lidar in actual operating scenarios (e.g., mining operations in high-altitude and cold regions) can be collected, and the data can cover the full semantic types of target obstacles (vehicles, cones, retaining walls, etc.) and noise (dust, fog, snow scattering points, etc.) to ensure that the sample point cloud data can reflect the typical intensity fluctuation characteristics and noise distribution patterns in high-altitude and cold environments.
[0062] S202, the sample intensity information corresponding to the same semantic type collected by each lidar is mapped to the standard intensity range corresponding to that semantic type to obtain the standard sample intensity information.
[0063] In some embodiments, the same mapping rules as in S102 above can be used to map the sample intensity information of the same semantic type to a unified standard intensity range to obtain standard sample intensity information, thereby achieving intensity standardization of sample data collected by different lidars.
[0064] S203. For each sample point cloud data, the sample location information and standard sample intensity information corresponding to the sample point cloud data are input into the semantic perception model to be trained, and the model parameters of the semantic perception model are adjusted based on the semantic perception results output by the semantic perception model to complete the training of the semantic perception model.
[0065] In some embodiments, the sample location information (x, y, z) and standard sample intensity information of each batch of sample point cloud data can be used as joint inputs to the semantic perception model and fed into the semantic perception model to be trained. Manually labeled semantic type labels are used as supervision signals. The convolution kernel parameters and feature fusion weights of the model are adjusted through backpropagation to minimize the error between the semantic perception result output by the model and the real label. The model is iteratively trained until it converges.
[0066] This embodiment of the present disclosure standardizes the sample intensity information during the training of the semantic perception model, which can unify the distribution of sample intensity information of different types of LiDAR, solve the problem of data that cannot be shared between multiple types of LiDAR, improve the model's adaptability to the coupling environment of multiple types of LiDAR, and enable the trained semantic perception model to be applicable to various types of LiDAR without having to train a device-specific model separately for each type of LiDAR.
[0067] Furthermore, by including intensity features from harsh environments such as high altitude and cold weather in the sample data, the semantic perception model can fully learn the interference patterns of harsh environments during the training phase, thereby improving its noise resistance and generalization ability and reducing false detections in practical applications.
[0068] The training process of the semantic perception model has been explained in detail above. Next, two sample augmentation methods will be introduced to solve the problem of insufficient sample point cloud data collected in harsh environments.
[0069] First, please refer to Figure 3 , Figure 3 A schematic flowchart of a sample enhancement method according to an embodiment of this disclosure is shown. Figure 3 As shown, the method includes the following steps.
[0070] S301, Extract the first point cloud information within at least one foreground target bounding box in the sample point cloud data.
[0071] In some embodiments, samples containing bounding boxes of foreground targets (vehicles, pedestrians, cones, etc.) can be selected from the collected sample point cloud data, and all point cloud data within each bounding box can be extracted as the first point cloud information, including the location, intensity, and semantic label information of the foreground target.
[0072] S302, insert the first point cloud information into the first preset position in any sample point cloud data to generate new sample point cloud data.
[0073] The preset location is the location where noise is concentrated in the sample point cloud data.
[0074] In some embodiments, the coordinate system of the first point cloud information can be transformed to the bounding box coordinate system with the center of the annotation box as the origin, and then the transformed first point cloud information can be inserted into the first preset position in any sample point cloud data. Subsequently, the point cloud coordinate system of the first preset position can be obtained, and all point cloud information in the bounding box can be transformed to the point cloud coordinate system in combination with the bounding box coordinate system, thereby completing the insertion of the first point cloud information.
[0075] For example, the noise concentration location can be selected as the first preset location based on the noise distribution characteristics of the mine and high-altitude / cold environment. For instance, the first preset location can be any location within a radius of 5 to 20 meters centered on the vehicle, where a large amount of noise is easily generated by harsh environments such as high altitude and cold.
[0076] S303 trains the semantic perception model based on new sample point cloud data.
[0077] In some embodiments, after generating new sample point cloud data containing a noisy environment and foreground targets, the new sample point cloud data can be mixed with the original sample point cloud data and used together for training the semantic perception model to improve the model's ability to recognize foreground targets under noise interference.
[0078] This embodiment of the present disclosure can extract the point cloud information (i.e., the first point cloud information) of the foreground target in the sample point cloud data that is not affected by harsh environments such as high altitude and cold weather, and insert it into the noise generated by the harsh environment, thereby supplementing the number of foreground target samples in harsh environments, solving the problem of insufficient model recognition ability caused by the scarcity of samples in such scenarios, and especially improving the detection rate of foreground targets under fog and snow interference in high altitude and cold weather environments.
[0079] Next, please refer to Figure 4 , Figure 4 A flowchart illustrating another sample enhancement method according to an embodiment of this disclosure is shown. Figure 4 As shown, the method includes the following steps.
[0080] S401, Extract second point cloud information of at least some noise points in the sample point cloud data.
[0081] In some embodiments, noisy sample point cloud data collected in high-altitude and cold environments, such as point cloud data affected by fog, blizzards, or dust, are selected from the sample point cloud data. Then, at least a portion of this noisy data is extracted as second point cloud information. The noisy portion can be determined manually or through a classification algorithm from the sample point cloud data.
[0082] For example, point cloud data that is less than a preset distance threshold away from the vehicle in a harsh environment can also be extracted as second point cloud information. The preset distance threshold can be 3 meters, which corresponds to the high noise area near the vehicle.
[0083] S402, insert the second point cloud information into the second preset position in any sample point cloud data to generate new sample point cloud data.
[0084] The second preset position is the position in the sample point cloud data where the distance between the vehicle and the sample point cloud is less than a preset distance threshold.
[0085] In some embodiments, for any sample point cloud data (including samples without severe weather), second point cloud information can be inserted into a second preset position to ensure that the inserted noise points conform to the spatial distribution pattern of the actual interference scene, thereby generating new sample point cloud data.
[0086] S403 trains the semantic perception model based on new sample point cloud data.
[0087] In some embodiments, new sample point cloud data containing near-field noise interference can be added to the training set and used together with the original sample point cloud data to train the semantic perception model, thereby enhancing the model's ability to resist interference from near-field noise.
[0088] The embodiments disclosed herein enable new sample point cloud data to accurately simulate noise interference scenarios near vehicles in high-altitude and cold environments, supplement noise features of samples in severe weather, improve the model's ability to distinguish near-distance noise, reduce false detections caused by near-distance noise (such as misjudging fog points as retaining walls or cones), and ensure the driving efficiency and safety of unmanned vehicles in mines.
[0089] In some embodiments, before inserting the second point cloud information into the second preset position in any sample point cloud data to generate new point cloud data, the point cloud density of the second point cloud information can be adjusted to generate new second point cloud information. Subsequently, when executing S402 above, the new second point cloud information can be inserted into the second preset position in any sample point cloud data to generate new sample point cloud data.
[0090] For example, the extracted second point cloud information can be thinned or encrypted to generate new second point cloud information. For instance, thinning can be achieved by randomly deleting part of the point cloud, or encryption can be achieved by supplementing the point cloud with an interpolation algorithm, thereby simulating noise interference of different intensities (such as the difference in noise density between light fog and heavy fog).
[0091] This embodiment of the disclosure can further enrich the diversity of noise samples by adjusting the point cloud density, covering the differences in noise density between light snow and blizzard, and between light fog and dense fog in severe weather interference of different intensities in high-altitude and cold environments, so that the generalization ability of the semantic perception model obtained after training is stronger.
[0092] Based on the same inventive concept, this disclosure also provides a sensing device, as shown in the following embodiment. Since the principle of this sensing device embodiment in solving the problem is the same as that described above... Figure 1 The method embodiments shown are similar, therefore the implementation of this sensing device embodiment can be found in the above description. Figure 1 The implementation of the method embodiments shown will not be repeated here.
[0093] Figure 5 A schematic diagram of the structure of a sensing device according to an embodiment of this disclosure is shown. Figure 5 As shown, the sensing device 500 includes: an acquisition module 501, a mapping module 502, and a sensing module 503.
[0094] The acquisition module 501 is used to acquire point cloud data collected by the lidar. The point cloud data includes point cloud location information and original intensity information.
[0095] The mapping module 502 is used to map the original intensity information of the same semantic type in the point cloud data to the standard intensity range corresponding to the semantic type according to the mapping relationship of the lidar, so as to obtain the standard intensity information. The mapping relationship is the pre-determined mapping relationship between the intensity acquisition range of the lidar and the standard intensity range. The semantic type includes at least the target obstacle type and the noise type.
[0096] The perception module 503 is used to input point cloud location information and standard intensity information into the semantic perception model to obtain the semantic perception result output by the semantic perception model.
[0097] In some embodiments, the mapping module 502 is used to obtain the original intensity range of the lidar for different semantic type objects; wherein, the original intensity range is the range of the original intensity information of the point cloud data of the corresponding semantic type object; and for each type of semantic type object, a mapping relationship between the original intensity range and the standard intensity range corresponding to that type of semantic object is established.
[0098] In some embodiments, the mapping module 502 is used to determine the target semantic type corresponding to the original intensity information based on the original intensity interval in which the original intensity information is located; when the target semantic type can be determined, the original intensity information is mapped to the corresponding value in the standard intensity interval based on the mapping relationship corresponding to the target semantic type, and used as the standard intensity information.
[0099] In some embodiments, the mapping module 502 is used to determine the standard intensity range of the lidar for objects of different semantic types; wherein, the standard intensity range of different lidars used to collect point cloud data for the same semantic type object is the same; and in the case where there is a partial overlap between the original intensity ranges corresponding to multiple semantic types, there is an equal proportion of overlap between the standard intensity ranges corresponding to multiple semantic types.
[0100] In some embodiments, the mapping module 502 is used to map the original intensity information to a preset intensity value and use it as standard intensity information when the target semantic type cannot be determined based on the original intensity information. The preset intensity value is a value outside the standard intensity range.
[0101] In some embodiments, the sensing device 500 further includes a correction module (not shown in the figure), which is used to correct the semantic type of each point in the point cloud data to the semantic type corresponding to the original intensity information of the point, provided that the point meets the semantic correction conditions; wherein the semantic correction conditions include: the original intensity information of the point corresponds to a unique semantic type; and the semantic type of the point shown in the semantic perception result is inconsistent with the semantic type corresponding to the original intensity information of the point.
[0102] In some embodiments, the sensing device 500 further includes a training module (not shown in the figure) for acquiring sample point cloud data collected by various lidars, the sample point cloud data including sample location information and sample intensity information; mapping the sample intensity information collected by each lidar corresponding to the same semantic type to a standard intensity range corresponding to that semantic type to obtain standard sample intensity information; for each sample point cloud data, inputting the sample location information and standard sample intensity information corresponding to the sample point cloud data into the semantic perception model to be trained, and adjusting the model parameters of the semantic perception model based on the semantic perception results output by the semantic perception model to complete the training of the semantic perception model.
[0103] In some embodiments, the training module is used to extract first point cloud information within at least one foreground target bounding box in the sample point cloud data; insert the first point cloud information into a first preset position in any sample point cloud data to generate new sample point cloud data; wherein, the preset position is the location where noise is concentrated in the sample point cloud data; and train the semantic perception model based on the new sample point cloud data.
[0104] In some embodiments, the training module is used to extract second point cloud information of at least some noise points in the sample point cloud data; insert the second point cloud information into a second preset position in any sample point cloud data to generate new sample point cloud data; wherein, the second preset position is a position in the sample point cloud data where the distance between the vehicle and the sample point cloud data is less than a preset distance threshold; and train the semantic perception model based on the new sample point cloud data.
[0105] In some embodiments, the training module is used to adjust the point cloud density of the second point cloud information to generate new second point cloud information; and to insert the new second point cloud information into a second preset position in any sample point cloud data to generate new sample point cloud data.
[0106] Based on the same inventive concept, this disclosure also provides an unmanned vehicle for performing... Figure 1 The perception method shown.
[0107] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.
Claims
1. A sensing method, characterized in that, include: Acquire point cloud data collected by lidar, wherein the point cloud data includes: point cloud location information and original intensity information; According to the mapping relationship corresponding to the lidar, the original intensity information in the point cloud data corresponding to the same semantic type is mapped to the standard intensity range corresponding to the semantic type to obtain the standard intensity information. The mapping relationship is a pre-determined mapping relationship between the intensity acquisition range of the lidar and the standard intensity range. The semantic type includes at least the target obstacle type and the noise type. The point cloud location information and the standard intensity information are input into the semantic perception model to obtain the semantic perception result output by the semantic perception model.
2. The method according to claim 1, characterized in that, Before mapping the original intensity information corresponding to the same semantic type in the point cloud data to the standard intensity range corresponding to that semantic type according to the mapping relationship corresponding to the lidar, and obtaining the standard intensity information, the method further includes: Obtain the original intensity range of the LiDAR for different semantic type objects; wherein, the original intensity range is the range of original intensity information of the point cloud data of the corresponding semantic type object; For each semantic type object, establish a mapping relationship between the original intensity range and the standard intensity range corresponding to that semantic type object; The step of mapping the original intensity information of the point cloud data corresponding to the same semantic type to the standard intensity range corresponding to the semantic type according to the mapping relationship of the lidar, to obtain the standard intensity information, includes: Based on the original intensity range in which the original intensity information is located, determine the target semantic type corresponding to the original intensity information; If the target semantic type can be determined, the original intensity information is mapped to the corresponding value in the standard intensity range based on the mapping relationship corresponding to the target semantic type, and used as the standard intensity information.
3. The method according to claim 2, characterized in that, Before establishing the mapping relationship between the original intensity interval and the standard intensity interval corresponding to this type of semantic object, the method further includes: Determine the standard intensity range of the lidar for objects of different semantic types; Among them, the standard intensity range of different lidars used to collect point cloud data is the same for the same semantic type of object; In the case where there is partial overlap between the original intensity intervals corresponding to multiple semantic types, there is an equal proportion of overlap between the standard intensity intervals corresponding to the multiple semantic types.
4. The method according to claim 2 or 3, characterized in that, The method further includes: If the target semantic type cannot be determined based on the original intensity information, the original intensity information is mapped to a preset intensity value and used as standard intensity information. The preset intensity value is a value outside the standard intensity range.
5. The method according to claim 1, characterized in that, The method further includes: For each point in the point cloud data, if the semantic correction condition is met, the semantic type of the point is corrected to the semantic type corresponding to the original intensity information of the point. The semantic correction conditions include: The original intensity information at this point corresponds to a unique semantic type; and The semantic type of the point shown in the semantic perception result is inconsistent with the semantic type corresponding to the original intensity information of the point.
6. The method according to claim 1, characterized in that, The semantic awareness model is trained in the following manner: Acquire sample point cloud data collected by various lidar systems, wherein the sample point cloud data includes: sample location information and sample intensity information; The sample intensity information corresponding to the same semantic type collected by each lidar is mapped to the standard intensity range corresponding to that semantic type to obtain the standard sample intensity information. For each sample point cloud data, the sample location information and standard sample intensity information corresponding to the sample point cloud data are input into the semantic perception model to be trained, and the model parameters of the semantic perception model are adjusted based on the semantic perception results output by the semantic perception model to complete the training of the semantic perception model.
7. The method according to claim 6, characterized in that, The method further includes: Extract the first point cloud information within at least one foreground target bounding box from the sample point cloud data; The first point cloud information is inserted into a first preset position in any sample point cloud data to generate new sample point cloud data; wherein, the preset position is the location where noise points are concentrated in the sample point cloud data; The semantic perception model is trained based on new sample point cloud data.
8. The method according to claim 6, characterized in that, The method further includes: Extract second point cloud information from at least some of the noise points in the sample point cloud data; The second point cloud information is inserted into the second preset position in any sample point cloud data to generate new sample point cloud data; wherein, the second preset position is the position in the sample point cloud data where the distance between it and the vehicle is less than a preset distance threshold; The semantic perception model is trained based on new sample point cloud data.
9. The method according to claim 8, characterized in that, Before inserting the second point cloud information into the second preset position in the arbitrary sample point cloud data to generate new point cloud data, the method further includes: Adjust the point cloud density of the second point cloud information to generate new second point cloud information; The step of inserting the second point cloud information into the second preset position in any sample point cloud data to generate new sample point cloud data includes: Insert the new second point cloud information into the second preset position in any sample point cloud data to generate new sample point cloud data.
10. An unmanned vehicle, characterized in that, Used to perform the sensing method according to any one of claims 1 to 9.