Feature division method and device, storage medium and computer equipment
By employing multimodal feature fusion and adaptive threshold adjustment, the problem of misjudgment in the decoupling of static and dynamic environmental data is solved, enabling accurate identification and high-precision feature segmentation of dynamic objects of unknown categories.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-05
- Publication Date
- 2026-03-10
AI Technical Summary
Existing technologies are susceptible to occlusion by static objects and changes in lighting when decoupling static and dynamic environmental data, leading to misjudgments, and cannot handle dynamic objects of unknown categories.
By acquiring target point cloud data and target image data from multiple sensors at the same time and in the same space, the dynamic probability value and adaptive threshold of each feature point are determined. Dynamic probability estimation is performed using multimodal features, and normalization is performed by combining geometric inconsistency and feature stability. The threshold is adaptively adjusted to divide the static and dynamic feature sets.
It improves the accuracy of dynamic detection, suppresses false detections in texture repetition areas, accurately identifies the features of unknown categories of dynamic objects, and enhances the robustness and adaptability of the system.
Smart Images
Figure CN121637003A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of industrial automation technology, and in particular to a feature segmentation method, apparatus, storage medium, and computer equipment. Background Technology
[0002] Currently, in the fields of industrial automation and intelligent operation and maintenance, robots and drones are gradually replacing manual labor in performing high-risk, high-frequency, or high-precision tasks. For example, in substation inspection scenarios, quadruped robots equipped with infrared cameras and LiDAR autonomously navigate and detect equipment temperature and abnormal noises; in warehousing and logistics scenarios, drone swarms collaboratively inventory high-bay racks, and AGV (Automated Guided Vehicle) robots sort goods; in disaster relief scenarios, drones search for survivors, and ground robots enter collapsed buildings to deliver supplies. These scenarios rely on the autonomous perception, real-time decision-making, and precise control of robots or drones.
[0003] In existing technologies, when performing high-precision tasks, it is necessary to first acquire environmental data and then build a map based on the environmental data in order to generate a navigation path. When the environment is complex, the types of environmental data acquired are diverse. In this case, the environmental data can be decoupled from static and dynamic elements, and a corresponding map can be built using static features. However, traditional methods rely solely on the geometric consistency of the environmental data for decoupling. This process is susceptible to interference from static objects occlusion and changes in lighting, leading to misjudgments (such as misidentifying a reflective glass door as a dynamic object). Alternatively, they may use semantic segmentation for dynamic-static decoupling, which relies on the generalization ability of pre-trained models and cannot handle unknown categories of dynamic objects (such as suddenly appearing irregularly shaped obstacles). Summary of the Invention
[0004] The purpose of this application is to at least solve one of the aforementioned technical defects, particularly the technical defects in the prior art where the dynamic and static decoupling of environmental data is easily affected by static objects blocking the data and changes in lighting, leading to misjudgment, and the inability to handle dynamic objects of unknown categories.
[0005] This application provides a feature segmentation method, the method comprising:
[0006] Acquire target point cloud data and target image data collected simultaneously in the same space by multiple pre-integrated sensors;
[0007] Determine the dynamic probability value of each feature point in the target point cloud data and the target image data, as well as the adaptive threshold of the current environment;
[0008] Based on the dynamic probability value of each feature point and the adaptive threshold, each feature point is divided into a static feature set and a dynamic feature set.
[0009] Optionally, determining the dynamic probability value of each feature point in the target point cloud data and the target image data includes:
[0010] For each feature point in the target point cloud data and the target image data:
[0011] Obtain the 3D coordinates, feature descriptor, and timestamp of the feature point, as well as the geometric and semantic weights of the current environment;
[0012] The geometric factors of the feature point are determined based on its 3D coordinates, feature descriptor, and timestamp, and the semantic factors of the feature point are determined based on its feature descriptor.
[0013] The dynamic probability value of a feature point is determined based on its geometric factors, geometric weights, semantic factors, and semantic weights.
[0014] Optionally, obtaining the geometric weights and semantic weights of the current environment includes:
[0015] Obtain the total number of historical features and the number of historical static features of the current environment;
[0016] The geometric and semantic weights of the current environment are determined based on the total number of historical features and the number of historical static features.
[0017] Optionally, determining the semantic factor of the feature point based on its feature descriptor includes:
[0018] Determine whether the feature point satisfies the semantic factor constraint based on the feature descriptor of the feature point;
[0019] If satisfied, the semantic factor of the feature point is determined according to the preset entropy range.
[0020] Optionally, determining the dynamic probability value of the feature point based on its geometric factors, geometric weights, semantic factors, and semantic weights includes:
[0021] Based on the 3D coordinates, feature descriptor, and timestamp of the feature point, determine whether the feature point meets the reliability assessment conditions;
[0022] If satisfied, the depth confidence and environmental change quantification index of the feature point are determined, and the dynamic probability value of the feature point is determined based on the geometric factor, geometric weight, semantic factor, semantic weight, depth confidence and environmental change quantification index of the feature point.
[0023] Optionally, the quantitative indicators for determining the environmental change of the feature point include:
[0024] Determine the voxel corresponding to the feature point, the voxel gradient, the voxel's occupancy state at the previous time step, the voxel's occupancy state at the current time step, and all sensor observation data from the initial time step to the current time step;
[0025] Based on the voxel corresponding to the feature point, the voxel gradient, the voxel's occupancy state at the previous time step, the voxel's occupancy state at the current time step, and all sensor observation data from the initial time step to the current time step, calculate the environmental change quantification index of the feature point.
[0026] Optionally, determining the adaptive threshold for the current environment includes:
[0027] Obtain the total number of historical features and the number of historical dynamic features of the current environment;
[0028] The adaptive threshold for the current environment is determined based on the total number of historical features and the number of historical dynamic features.
[0029] This application also provides a feature segmentation apparatus, including:
[0030] The data acquisition module is used to acquire target point cloud data and target image data collected by multiple pre-integrated sensors at the same time and in the same space.
[0031] The data processing module is used to determine the dynamic probability value of each feature point in the target point cloud data and the target image data, as well as the adaptive threshold of the current environment;
[0032] The feature segmentation module is used to divide each feature point into a static feature set and a dynamic feature set based on the dynamic probability value of each feature point and the adaptive threshold.
[0033] This application also provides a computer-readable storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the feature partitioning method as described in any of the above embodiments.
[0034] This application also provides a computer device, including: one or more processors, and memory;
[0035] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the feature segmentation method as described in any of the above embodiments.
[0036] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:
[0037] The feature segmentation method, apparatus, storage medium, and computer equipment provided in this application, when acquiring target point cloud data and target image data collected simultaneously by multiple pre-integrated sensors in the same time and space, can capture kinematic anomalies and quantify feature stability and suppress false detections of texture repetition regions by using multimodal features, such as geometric features in the target point cloud data and semantic features in the target image data, thereby improving dynamic detection accuracy. In this process, this application can first determine the dynamic probability value of each feature point in the target point cloud data and target image data, as well as the adaptive threshold of the current environment, and then divide each feature point into a static feature set and a dynamic feature set according to the dynamic probability value and adaptive threshold of each feature point. The dynamic probability value of this application is obtained by normalizing the geometric inconsistency of the target point cloud data and the target image data with the feature stability. The adaptive threshold of this application can be adaptively adjusted according to the current environment. Therefore, dividing the static feature set and the dynamic feature set by the dynamic probability value and the adaptive threshold can effectively improve the accuracy of feature division and is not easily affected by static object occlusion or lighting changes. It can also accurately identify the features of dynamic objects of unknown category. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 A flowchart illustrating a feature segmentation method provided in an embodiment of this application;
[0040] Figure 2 A schematic diagram illustrating the process of determining the dynamic probability value of each feature point as provided in the embodiments of this application;
[0041] Figure 3 This is a schematic diagram of the structure of a feature division device provided in an embodiment of this application;
[0042] Figure 4 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0043] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0044] In one embodiment, such as Figure 1 As shown, Figure 1 This is a flowchart illustrating a feature segmentation method provided in an embodiment of this application; this application provides a feature segmentation method, which may include:
[0045] S110: Acquire target point cloud data and target image data collected by multiple pre-integrated sensors at the same time and in the same space.
[0046] In this step, when classifying dynamic and static features, multiple pre-integrated sensors can be used to acquire target point cloud data and target image data collected at the same time and in the same space. This allows the current location, device distribution, and surrounding environment information to be determined based on the target point cloud data and target image data, and further classify dynamic and static features.
[0047] Specifically, when this application performs high-precision tasks, it can acquire target point cloud data and target image data collected simultaneously by multiple pre-integrated sensors in the same time and space. Among them, the sensors pre-integrated in this application include, but are not limited to, LiDAR, vision sensors, inertial measurement units (IMU), millimeter-wave radar, BeiDou navigation system, etc., and can be configured according to the actual application environment, without limitation.
[0048] Among the aforementioned sensors, lidar can provide high-precision 3D point cloud data for environmental modeling and localization; visual sensors can include monocular, binocular, or multi-view cameras for image feature extraction, target recognition, and depth estimation; inertial measurement units can measure acceleration and angular velocity and provide high-frequency motion state information; millimeter-wave radar is used for long-distance obstacle detection and velocity measurement, and performs stably, especially in adverse weather conditions; and the BeiDou Navigation Satellite System can provide absolute position information to assist in global positioning.
[0049] Furthermore, in multi-sensor systems, due to the inconsistent data acquisition times of different sensors, if each sensor uses its own local clock, factors such as crystal oscillator errors and temperature variations will cause different clock frequencies among the sensors, leading to gradual time desynchronization and initial time discrepancies between sensors. This application addresses this by synchronizing the data acquisition times of each sensor to ensure timestamp alignment across multiple data sources. Additionally, to eliminate spatial positional biases, this application can align the coordinate systems of each sensor to the same reference frame. This improves data fusion accuracy, avoids feature misalignment caused by spatiotemporal deviations (such as matching errors between laser point clouds and image pixels), enhances system robustness (allowing for continued functionality through synchronized redundant data even when some sensors fail, such as short-term localization relying solely on IMU and vision), and optimizes real-time performance by reducing data processing latency through synchronization, thus meeting the real-time control requirements of dynamic scenarios (such as robot obstacle avoidance and path planning).
[0050] For example, when a quadruped robot needs to traverse equipment areas and avoid energized structures and oil pipelines, the lidar on the quadruped robot can scan the three-dimensional outline of the equipment (such as scanning to generate a point cloud skeleton of substation equipment); the binocular camera can identify equipment nameplates (such as "#3 main transformer 101 switch"), instrument readings, insulator cracks and other defects; after collecting these raw data, they can be spatiotemporally synchronized to obtain target point cloud data and target image data.
[0051] S120: Determine the dynamic probability value of each feature point in the target point cloud data and target image data, as well as the adaptive threshold for the current environment.
[0052] S130: Based on the dynamic probability value and adaptive threshold of each feature point, divide each feature point into a static feature set and a dynamic feature set.
[0053] In the above steps, after obtaining the target point cloud data and target image data collected by each sensor at the same time and in the same space through S110, the dynamic probability value of each feature point in the target point cloud data and target image data, as well as the adaptive threshold of the current environment, are determined. In this way, each feature point can be divided into a static feature set and a dynamic feature set according to the dynamic probability value and the adaptive threshold of each feature point.
[0054] Specifically, after acquiring target point cloud data and target image data, this application utilizes multimodal features, such as geometric features in the target point cloud data and semantic features in the target image data, to determine the target through feature extraction and target recognition. This allows for the capture of kinematic anomalies and the quantification of feature stability through multimodal features, suppressing false detections in textured regions and improving dynamic detection accuracy.
[0055] In the process of decoupling the static and dynamic features of target point cloud data and target image data, traditional methods rely solely on geometric consistency (such as optical flow residuals), which is easily affected by static object occlusion and lighting changes, leading to misjudgment (such as misidentifying a reflective glass door as a dynamic object). Semantic segmentation alone relies on the generalization ability of pre-trained models and cannot handle dynamic objects of unknown categories (such as suddenly appearing irregular obstacles).
[0056] Based on this, when this application uses a dynamic probability estimation model to evaluate the dynamic probability value of each feature point, the dynamic probability estimation model can generate a dynamic probability value by normalizing a linear combination of geometric inconsistency and feature stability. For example, if the geometric difference is large and the feature is unstable, the output is close to 1, and it is determined to be a dynamic feature; if the geometry is consistent and the feature is stable, the output is close to 0, and it is determined to be a static feature, thereby effectively improving the recognition accuracy of static and dynamic features.
[0057] After obtaining the dynamic probability value of each feature point, this application can divide each feature point into a static feature set and a dynamic feature set based on the dynamic probability value of each feature point and an adaptive threshold, as follows:
[0058] Static feature set ;
[0059] Dynamic feature set .
[0060] In the above formula, Represents a static feature set. Represents a dynamic feature set. Represents the i-th feature point object. This represents the dynamic probability value of the i-th feature point, and the static feature set and dynamic feature set. This refers to the adaptive threshold, a key parameter for achieving high accuracy and robustness in robot dynamic localization and alignment technology. However, traditional algorithms use fixed adaptive thresholds, which cannot distinguish between dynamic and static features and may lead to false matches. Furthermore, excessively high thresholds can cause missed detections of slow-moving objects. This application determines the adaptive threshold based on the current environment. This threshold automatically increases (up to a maximum of 0.25) in highly dynamic scenes (such as dense crowds) to avoid oversensitivity, and automatically decreases to 0.15 in predominantly static scenes (such as empty corridors) to improve detection sensitivity.
[0061] In the above embodiments, after acquiring target point cloud data and target image data collected by multiple pre-integrated sensors at the same time and in the same space, since the target point cloud data and target image data contain multimodal features, such as geometric features in the target point cloud data and semantic features in the target image data, this application can capture kinematic anomalies through multimodal features, quantify feature stability, suppress false detections of texture repetition regions, and improve dynamic detection accuracy. In this process, this application can first determine the dynamic probability value of each feature point in the target point cloud data and target image data, as well as the adaptive threshold of the current environment, and then divide each feature point into a static feature set and a dynamic feature set according to the dynamic probability value and adaptive threshold of each feature point. The dynamic probability value of this application is obtained by normalizing the geometric inconsistency of the target point cloud data and the target image data with the feature stability. The adaptive threshold of this application can be adaptively adjusted according to the current environment. Therefore, dividing the static feature set and the dynamic feature set by the dynamic probability value and the adaptive threshold can effectively improve the accuracy of feature division and is not easily affected by static object occlusion or lighting changes. It can also accurately identify the features of dynamic objects of unknown category.
[0062] In one embodiment, such as Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the process of determining the dynamic probability value of each feature point provided in the embodiments of this application; S120, determining the dynamic probability value of each feature point in the target point cloud data and the target image data, may include:
[0063] S121: For each feature point in the target point cloud data and target image data: obtain the 3D coordinates, feature descriptor, and timestamp of the feature point, as well as the geometric weight and semantic weight of the current environment.
[0064] S122: Determine the geometric factors of the feature point based on its 3D coordinates, feature descriptor, and timestamp, and determine the semantic factors of the feature point based on its feature descriptor.
[0065] S123: Determine the dynamic probability value of the feature point based on its geometric factors, geometric weights, semantic factors, and semantic weights.
[0066] In this embodiment, when determining the dynamic probability value of each feature point in the target point cloud data and target image data, for each feature point in the target point cloud data and target image data, this application can first obtain the 3D coordinates, feature descriptor, and timestamp of the feature point, as well as the geometric weight and semantic weight of the current environment. Then, the geometric factor of the feature point is determined based on the 3D coordinates, feature descriptor, and timestamp, and the semantic factor of the feature point is determined based on the feature descriptor. Finally, the obtained parameters are input into the dynamic probability estimation model to obtain the dynamic probability value of the feature point.
[0067] The formula for the dynamic probability estimation model in this application is as follows:
[0068]
[0069] in, ∈(0,1) represents the dynamic probability value of the i-th feature point. Let i be the i-th feature point object. This feature point object includes the 3D coordinates, feature descriptor, and timestamp of the i-th feature point, where i is the index number of the feature point. The sigmoid function maps any real number to the interval (0, 1), and its output can be interpreted as a dynamic probability. This represents the actual displacement between adjacent frames (calculated using optical flow). The expected projected displacement based on camera motion. Geometric factor Let i be the 3D coordinates of the i-th feature point. To preset the maximum dynamic object speed, The weights are time-varying and determined based on timestamps; α is the geometric weight and β is the semantic weight. Let the entropy value of the feature descriptor be the semantic factor, and let the feature descriptor of the i-th feature point be... For a 256-dimensional vector (such as ORB or DeepLabv3+ output), its entropy is calculated as follows:
[0070]
[0071] in, The entropy value is the probability value obtained after Softmax normalization of the k-th dimension of the feature descriptor. The larger the entropy value of the feature descriptor, the more uniform the distribution of each dimension of the descriptor. At this time, the feature points lack significant distinguishability (may belong to the repetitive texture of the surface of a dynamic object). The smaller the entropy value, the stronger the response of some dimensions of the descriptor. At this time, the feature points have uniqueness (more likely to belong to a static object). Let k be the feature descriptor value of the i-th feature point. The dimension index number of the feature descriptor. Let j be the feature descriptor value of the j-th dimension of the feature descriptor for the i-th feature point. This is the dimension index number for the feature descriptor.
[0072] In one embodiment, obtaining the geometric weights and semantic weights of the current environment in S121 may include:
[0073] S1211: Obtain the total number of historical features and the number of historical static features of the current environment.
[0074] S1212: Determine the geometric weight and semantic weight of the current environment based on the total number of historical features and the number of historical static features.
[0075] In this embodiment, since geometric factors can capture kinematic anomalies and semantic factors can quantify feature stability and suppress false detections of texture repetition regions, this application can use an adaptive mechanism to dynamically adjust geometric and semantic weights. This allows for increasing geometric weights in structured environments (such as corridors) to enhance sensitivity to moving objects, and increasing semantic weights in complex textured environments (such as forests) to suppress false detections of static features.
[0076] Based on this, when obtaining the geometric and semantic weights of the current environment, this application can first obtain the total number of historical features and the number of historical static features of the current environment, and then determine the geometric and semantic weights of the current environment based on the total number of historical features and the number of historical static features. The specific formula is as follows:
[0077]
[0078] in, The number of historical static features. This represents the total number of historical characteristics.
[0079] After dynamically adjusting the geometric and semantic weights through the above adaptive mechanism, in a warehouse environment, α=0.6 and β=0.4 can be set to balance the interference between the forklift movement trajectory and the shelf texture; in a field environment, α=0.4 and β=0.6 can be set to prioritize the feature stability to resist the influence of grass and trees swaying.
[0080] In one embodiment, determining the semantic factor of the feature point based on its feature descriptor in step S122 may include:
[0081] S1221: Determine whether the feature point satisfies the semantic factor constraint condition based on the feature descriptor of the feature point.
[0082] S1222: If satisfied, the semantic factor of the feature point is determined according to the preset entropy range.
[0083] In this embodiment, when determining the semantic factor of a feature point, it is possible to first determine whether the feature point satisfies the semantic factor constraint condition based on the feature descriptor of the feature point. If it does, the semantic factor of the feature point is determined based on the preset entropy value range.
[0084] It is understood that the feature descriptor of each feature point in this application represents an abstract representation of the local region surrounding a keypoint in an image or point cloud, used for efficient matching of corresponding features in different viewpoints or time series. For example, the feature descriptor corresponding to a feature point in the target point cloud data in this application can describe the pixel intensity or color statistics of the keypoint's neighborhood, or it can capture texture features through filter responses (such as Gabor filters) or Local Binary Patterns (LBP), or it can use Histogram of Gradient Orientation (HOG) to express edge and corner structures, etc. Therefore, this application can determine whether a feature point satisfies a semantic factor constraint based on its feature descriptor. This semantic factor constraint can be a highly dynamic object or other constraints. For example, for highly dynamic objects (such as birds), this application can constrain the entropy range. The entropy range [1.2, 3.0] is used to filter transient noise. If the entropy value of the feature descriptor falls within this range, it is classified as a high-dynamic feature and assigned a lower semantic weight. Conversely, static features with entropy values below this range (such as wall markings) are assigned a higher semantic weight. This differentiated processing mechanism effectively improves the robustness of feature classification in dynamic scenarios. For example, in a substation inspection scenario, when a feature point with an entropy value of 2.5 is detected, the system can automatically classify it as a temporary dynamic obstacle such as a bird and filter it out during map updates; while when an insulator feature point with an entropy value of 0.8 is detected, it is included in the permanent map as a stable static feature. This mechanism achieves intelligent differentiation between dynamic and static objects by quantifying the information entropy of the feature descriptor, providing a reliable data foundation for subsequent spatiotemporal joint optimization.
[0085] In one embodiment, determining the dynamic probability value of the feature point in step S123 based on its geometric factor, geometric weight, semantic factor, and semantic weight may include:
[0086] S1231: Based on the 3D coordinates, feature descriptor, and timestamp of the feature point, determine whether the feature point meets the reliability assessment conditions.
[0087] S1232: If satisfied, determine the depth confidence and environmental change quantification index of the feature point, and determine the dynamic probability value of the feature point based on the geometric factor, geometric weight, semantic factor, semantic weight, depth confidence and environmental change quantification index of the feature point.
[0088] In this embodiment, when determining the dynamic probability value of the feature point, this application can first determine whether the feature point meets the reliability assessment conditions based on the 3D coordinates, feature descriptor, and timestamp of the feature point. If it meets the conditions, the depth confidence and environmental change quantification index of the feature point are determined, and the dynamic probability value of the feature point is determined based on the geometric factor, geometric weight, semantic factor, semantic weight, depth confidence, and environmental change quantification index of the feature point. If it does not meet the conditions, the dynamic probability value of the feature point is determined directly based on the geometric factor, geometric weight, semantic factor, and semantic weight of the feature point.
[0089] For example, this application can capture kinematic anomalies and texture features based on the 3D coordinates, feature descriptor, and timestamp of the feature point. When the feature point is determined to be a feature point of a static but low-texture object, relying solely on a single feature cannot accurately evaluate the dynamic probability value of the feature point. Therefore, this application can regard feature points that satisfy the above features as satisfying the reliability assessment conditions, introduce depth confidence as a third constraint, and introduce environmental change quantification index as a regulating valve for dynamically balancing multi-sensor constraints. The specific formula is as follows:
[0090]
[0091] in, For depth confidence, Quantify indicators for environmental change.
[0092] This application enables SLAM systems to improve accuracy by utilizing depth confidence and environmental change quantification indicators in synergistic ways, while also suppressing errors caused by cross-modal interference in complex scenarios such as substations.
[0093] In one embodiment, determining the environmental change quantification index of the feature point in S1232 may include:
[0094] S321: Determine the voxel corresponding to the feature point, the voxel gradient, the voxel's occupancy state at the previous time step, the voxel's occupancy state at the current time step, and all sensor observation data from the initial time step to the current time step.
[0095] S322: Calculate the environmental change quantification index of the feature point based on the voxel corresponding to the feature point, the voxel gradient, the voxel's occupancy state at the previous time, the voxel's occupancy state at the current time, and all sensor observation data from the initial time to the current time.
[0096] In this embodiment, when determining the environmental change quantification index of a feature point, the voxel, voxel gradient, voxel occupancy state at the previous moment, voxel occupancy state at the current moment, and all sensor observation data from the initial moment to the current moment can be determined first. Then, the environmental change quantification index of the feature point can be calculated based on the relevant information confirmed in the previous step.
[0097] For example, the calculation formula for the environmental change quantification index of this application is as follows:
[0098]
[0099] in, For the i-th voxel, Let be the total number of voxels, and i be the voxel index number. Each feature point can correspond to one or more voxels. voxels The TSDF gradient change represents the degree of change in the distance field of the voxel between two consecutive time points. For indicator functions (when voxel) The value is 1 when the occupation state changes and 0 when it remains unchanged. This represents the state of the voxel at the previous moment. This represents the current state of voxels. This refers to all sensor observation data (such as lidar point clouds and depth camera data) from the initial time to time t-1. This refers to all sensor observation data from the initial time to time t. For sensor observation data at time t, when voxel When occupied, For the actual observation of the sensor The probability, For the sensor to observe The probability, According to Infer the probability that a voxel is occupied. This is the normalization constant for the observed data, ensuring that the sum of probabilities is 1.
[0100] This application can calculate the quantitative index of environmental change at the feature point using the above formula. Furthermore, to obtain more accurate calculation results, after obtaining multiple voxels of the feature point, this application can also filter voxels based on the voxel confidence level of each voxel to further optimize the calculation results.
[0101] In one embodiment, determining the adaptive threshold for the current environment in S124 may include:
[0102] S1241: Obtain the total number of historical features and the number of historical dynamic features of the current environment.
[0103] S1242: Determine the adaptive threshold of the current environment based on the total number of historical features and the number of historical dynamic features.
[0104] In this embodiment, since the adaptive threshold of this application is strongly correlated with the current environment, when determining the adaptive threshold of the current environment, this application can first obtain the total number of historical features and the number of historical dynamic features of the current environment, and then determine the adaptive threshold of the current environment based on the total number of historical features and the number of historical dynamic features. The specific formula is as follows:
[0105]
[0106] in, This represents the number of historical dynamic features. The adaptive threshold for the current environment can be quickly determined using the formula described above.
[0107] The feature division apparatus provided in the embodiments of this application is described below. The feature division apparatus described below and the feature division method described above can be referred to each other.
[0108] In one embodiment, such as Figure 3 As shown, Figure 3 This is a schematic diagram of a feature segmentation device provided in an embodiment of this application; this application also provides a feature segmentation device, which may include a data acquisition module 210, a data processing module 220, and a feature segmentation module 230, specifically including the following:
[0109] The data acquisition module 210 is used to acquire target point cloud data and target image data collected by multiple pre-integrated sensors at the same time and in the same space.
[0110] The data processing module 220 is used to determine the dynamic probability value of each feature point in the target point cloud data and the target image data, as well as the adaptive threshold of the current environment.
[0111] The feature segmentation module 230 is used to divide each feature point into a static feature set and a dynamic feature set based on the dynamic probability value of each feature point and the adaptive threshold.
[0112] In the above embodiments, after acquiring target point cloud data and target image data collected by multiple pre-integrated sensors at the same time and in the same space, since the target point cloud data and target image data contain multimodal features, such as geometric features in the target point cloud data and semantic features in the target image data, this application can capture kinematic anomalies through multimodal features, quantify feature stability, suppress false detections of texture repetition regions, and improve dynamic detection accuracy. In this process, this application can first determine the dynamic probability value of each feature point in the target point cloud data and target image data, as well as the adaptive threshold of the current environment, and then divide each feature point into a static feature set and a dynamic feature set according to the dynamic probability value and adaptive threshold of each feature point. The dynamic probability value of this application is obtained by normalizing the geometric inconsistency of the target point cloud data and the target image data with the feature stability. The adaptive threshold of this application can be adaptively adjusted according to the current environment. Therefore, dividing the static feature set and the dynamic feature set by the dynamic probability value and the adaptive threshold can effectively improve the accuracy of feature division and is not easily affected by static object occlusion or lighting changes. It can also accurately identify the features of dynamic objects of unknown category.
[0113] In one embodiment, this application also provides a computer-readable storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the feature partitioning method as described in any of the above embodiments.
[0114] In one embodiment, this application also provides a computer device, including: one or more processors, and memory.
[0115] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the feature segmentation method as described in any of the above embodiments.
[0116] Indicatively, such as Figure 4 As shown, Figure 4 This is a schematic diagram of the internal structure of a computer device 300 provided in an embodiment of this application. The computer device 300 can be provided as a server. (Refer to...) Figure 4 The computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by memory 301 for storing instructions, such as application programs, that can be executed by the processing component 302. The application programs stored in memory 301 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 302 is configured to execute instructions to perform the feature partitioning method of any of the above embodiments.
[0117] The computer device 300 may also include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate on an operating system stored in memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.
[0118] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0119] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0120] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.
[0121] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A feature partitioning method, comprising: The method comprises: acquiring target point cloud data and target image data collected by a plurality of pre-integrated sensors at the same time and in the same space; determining a dynamic probability value of each feature point in the target point cloud data and the target image data, and an adaptive threshold value of a current environment; dividing each feature point into a static feature set and a dynamic feature set according to the dynamic probability value of each feature point and the adaptive threshold value.
2. The feature partitioning method of claim 1, wherein, The determination of the dynamic probability value of each feature point in the target point cloud data and the target image data comprises: for each feature point in the target point cloud data and the target image data: acquiring a 3D coordinate, a feature descriptor and a timestamp of the feature point, and a geometric weight and a semantic weight of the current environment; determining a geometric factor of the feature point according to the 3D coordinate, the feature descriptor and the timestamp of the feature point, and determining a semantic factor of the feature point according to the feature descriptor of the feature point; determining the dynamic probability value of the feature point according to the geometric factor, the geometric weight, the semantic factor and the semantic weight of the feature point.
3. The feature partitioning method of claim 2, wherein, The acquisition of the geometric weight and the semantic weight of the current environment comprises: acquiring a total number of historical features and a number of historical static features of the current environment; determining the geometric weight and the semantic weight of the current environment according to the total number of historical features and the number of historical static features.
4. The feature partitioning method of claim 2, wherein, The determination of the semantic factor of the feature point according to the feature descriptor of the feature point comprises: determining whether the feature point satisfies a semantic factor restriction condition according to the feature descriptor of the feature point; if yes, determining the semantic factor of the feature point according to a preset entropy value range.
5. The feature partitioning method of claim 2, wherein, The determination of the dynamic probability value of the feature point according to the geometric factor, the geometric weight, the semantic factor and the semantic weight of the feature point comprises: determining whether the feature point satisfies a reliability evaluation condition according to the 3D coordinate, the feature descriptor and the timestamp of the feature point; if yes, determining a depth confidence and an environmental change quantitative index of the feature point, and determining the dynamic probability value of the feature point according to the geometric factor, the geometric weight, the semantic factor, the semantic weight, the depth confidence and the environmental change quantitative index of the feature point.
6. The feature partitioning method of claim 5, wherein, The determination of the environmental change quantitative index of the feature point comprises: determining a voxel corresponding to the feature point, a voxel gradient, an occupancy state of the voxel at a previous time, an occupancy state of the voxel at a current time, and all sensor observation data from an initial time to the current time; calculating the environmental change quantitative index of the feature point according to the voxel corresponding to the feature point, the voxel gradient, the occupancy state of the voxel at the previous time, the occupancy state of the voxel at the current time, and all sensor observation data from the initial time to the current time.
7. The feature partitioning method of any one of claims 1-6, wherein, The determination of the adaptive threshold value of the current environment comprises: acquiring a total number of historical features and a number of historical dynamic features of the current environment; determining the adaptive threshold value of the current environment according to the total number of historical features and the number of historical dynamic features.
8. An apparatus for feature partitioning, the apparatus comprising: comprises: a data acquisition module configured to acquire target point cloud data and target image data collected by a plurality of pre-integrated sensors at the same time and in the same space; a data processing module configured to determine a dynamic probability value of each feature point in the target point cloud data and the target image data, and an adaptive threshold value of a current environment; a feature division module configured to divide each feature point into a static feature set and a dynamic feature set according to the dynamic probability value of each feature point and the adaptive threshold value.
9. A computer-readable storage medium, characterized in that: The computer readable storage medium stores computer readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the feature division method according to any one of claims 1 to 7.
10. A computer device, comprising: comprise: one or more processors, and a memory; The memory stores computer readable instructions, which, when executed by the one or more processors, perform the steps of the feature division method according to any one of claims 1 to 7.