Automatic Inspection and Sensing Method and System Based on 3D Point Cloud Data

Abstract

This application provides an automatic inspection and perception method and system based on 3D point cloud data, relating to the field of automatic inspection and perception technology. First, a spatial association network is constructed for the 3D point cloud data set of the inspection target. Network nodes correspond to point cloud units, and association edges correspond to spatial association parameters. Next, anomaly propagation simulation processing is performed on the network to locate potential abnormal point cloud regions. Then, a pre-trained inspection feature enhancement model is called to perform feature interaction enhancement processing on the point cloud data of the potential abnormal regions, generating an enhanced point cloud feature set. The enhanced point cloud feature set is then associated with an anomaly feature sample library through feature mapping processing to determine the inspection focus area units. Finally, based on relevant region information and the enhanced feature set, an automatic inspection and perception result containing a regional spatial association description and an anomaly feature association expression is generated. This invention improves the perception efficiency and practicality of automatic inspection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic inspection perception, in particular to an automatic inspection perception method and system based on three-dimensional point cloud data. BACKGROUND

[0002] In many fields such as industrial production, building monitoring and traffic management, automatic inspection is crucial to ensure facility safety and improve operational efficiency. Traditional automatic inspection methods mostly rely on single-dimensional data collected by sensors, such as temperature and pressure, which are difficult to comprehensively and accurately perceive the state of complex structure inspection targets. With the development of three-dimensional point cloud data collection technology, it can provide rich spatial information, bringing new opportunities for automatic inspection.

[0003] However, the existing inspection methods based on three-dimensional point cloud data have certain limitations. Some methods only perform simple geometric feature analysis on point cloud data, ignoring the spatial correlation between point cloud units, and cannot effectively identify abnormal situations caused by spatial correlation. Some other methods lack in-depth analysis and feature enhancement of potential abnormal areas during anomaly detection, resulting in insufficient accuracy and reliability of anomaly identification. In addition, the existing methods have a single correlation method when correlating the detected features with the abnormal feature sample library, which makes it difficult to accurately determine the inspection focus area unit, affecting the perception effect and practicality of automatic inspection. SUMMARY

[0004] Therefore, the purpose of the present application is to provide an automatic inspection perception method and system based on three-dimensional point cloud data.

[0005] According to the first aspect of the present application, an automatic inspection perception method based on three-dimensional point cloud data is provided, which comprises:

[0006] performing spatial correlation network construction processing on the three-dimensional point cloud data set corresponding to the inspection target to form a point cloud spatial correlation network containing the spatial correlation relationship of the point cloud units, each node in the point cloud spatial correlation network corresponding to a point cloud unit in the three-dimensional point cloud data set, and the correlation edge between nodes corresponding to the spatial correlation parameters between point cloud units, the spatial correlation parameters including the spatial direction parameters between point cloud units and the normal vector correlation parameters of adjacent point cloud units;

[0007] performing abnormal propagation simulation processing on the point cloud spatial correlation network, and locating a potential abnormal point cloud area with abnormal propagation characteristics from the point cloud spatial correlation network based on a preset abnormal propagation rule, the potential abnormal point cloud area composed of nodes and correlation edges corresponding to a plurality of continuous point cloud units;

[0008] The pre-trained inspection feature enhancement model is invoked to perform feature interaction enhancement processing on the point cloud data corresponding to the potential abnormal point cloud region, generating an enhanced point cloud feature set with anomaly feature-specific representation. The enhanced point cloud feature set includes the correlation representation of the spatial structure features and point cloud unit attribute features of the potential abnormal point cloud region.

[0009] The enhanced point cloud feature set is associated with the inspection target anomaly feature sample library through feature mapping, and inspection interest area units with anomaly feature matching degree are determined from the potential anomaly point cloud region.

[0010] Based on the potential abnormal point cloud region information and enhanced point cloud feature set corresponding to the inspection focus area unit, an automatic inspection perception result containing regional spatial association description and abnormal feature association expression is generated.

[0011] According to a second aspect of this application, an automatic inspection and perception system based on three-dimensional point cloud data is provided. The automatic inspection and perception system based on three-dimensional point cloud data includes a processor and a readable storage medium. The readable storage medium stores a program that, when executed by the processor, implements the aforementioned automatic inspection and perception method based on three-dimensional point cloud data.

[0012] Based on any of the above aspects, a spatial association network is constructed from the 3D point cloud data set corresponding to the inspection target, forming a point cloud spatial association network that includes the spatial relationships between point cloud units. This fully considers spatial association parameters such as spatial direction parameters and normal vector association parameters between point cloud units, clearly presenting the inherent connections between point cloud data. Then, anomaly propagation simulation processing is performed on the point cloud spatial association network, accurately locating potential anomalous point cloud regions with anomalous propagation characteristics based on preset anomaly propagation rules. This effectively captures anomalies caused by spatial association, improving the targeting and accuracy of anomaly detection. A pre-trained inspection feature enhancement model is called to perform feature interaction enhancement processing on the point cloud data corresponding to the potential anomalous point cloud regions, generating an enhanced point cloud feature set with specific representations of anomalous features. This enhances the anomaly-related feature information in the point cloud data, further improving the identifiability of anomalous features. By performing feature mapping association processing between the enhanced point cloud feature set and the inspection target anomalous feature sample library, inspection interest area units with anomalous feature matching degree can be accurately determined from the potential anomalous point cloud regions, enabling automatic inspection to focus on key areas. Finally, based on the potential abnormal point cloud region information and enhanced point cloud feature set corresponding to the inspection focus area unit, an automatic inspection perception result containing regional spatial association description and abnormal feature association expression is generated, which significantly improves the perception efficiency and practicality of automatic inspection. Attached Figure Description

[0013] Figure 1A flowchart illustrating the automatic inspection and perception method based on 3D point cloud data provided in an embodiment of this application is shown.

[0014] Figure 2 This paper illustrates a schematic diagram of the component structure of an automatic inspection and perception system based on three-dimensional point cloud data provided in an embodiment of this application. Detailed Implementation

[0015] Figure 1 The diagram illustrates an automatic inspection and perception method based on 3D point cloud data provided in an embodiment of this application. It should be understood that in other embodiments, the order of some steps in the automatic inspection and perception method based on 3D point cloud data can be interchanged according to actual needs, or some steps can be omitted or deleted. The detailed steps of this automatic inspection and perception method based on 3D point cloud data are described below.

[0016] Step S110: Perform spatial association network construction processing on the three-dimensional point cloud data set corresponding to the inspection target to form a point cloud spatial association network containing the spatial association relationship of point cloud units. Each node in the point cloud spatial association network corresponds to a point cloud unit in the three-dimensional point cloud data set, and the association edge between nodes corresponds to the spatial association parameter between point cloud units. The spatial association parameter includes the spatial direction parameter between point cloud units and the normal vector association parameter of adjacent point cloud units.

[0017] In this embodiment, the 3D point cloud dataset is acquired using LiDAR or structured light equipment. Each point cloud unit is a discrete spatial point output by the acquisition device, containing 3D coordinates and attribute information such as reflection intensity and color. The core of the spatial association network construction process is to establish the topological relationship between point cloud units. Specifically, each point cloud unit in the 3D point cloud dataset is first traversed, and its unique identifier is used as the node identifier of the point cloud spatial association network. The node attribute field stores the spatial coordinate information and normal vector information of the point cloud unit. Then, for each node, its spatial distance to other nodes is calculated, and nodes with distances within a preset threshold are selected as adjacent nodes, and association edges are constructed between adjacent nodes. The calculation of the spatial association parameters of the association edges is divided into two parts: the spatial direction parameter is obtained by normalizing the spatial coordinate difference vector of the two nodes, and the normal vector association parameter is obtained by calculating the dot product of the normal vectors of the two nodes. The dot product result reflects the similarity of the normal vectors. After all the adjacent nodes of all nodes have been traversed, all nodes, association edges, and association edge attribute fields are integrated to form the point cloud spatial association network. In this point cloud spatial association network, each node corresponds to a discrete spatial point output by the acquisition device. The spatial direction parameter of the associated edge reflects the relative orientation of adjacent discrete spatial points, and the normal vector association parameter reflects the similarity of the surface normals of adjacent discrete spatial points.

[0018] Step S111: Extract the basic attribute information of each point cloud unit in the three-dimensional point cloud data set. The basic attribute information includes the spatial coordinate information and normal vector information of the point cloud unit. The spatial coordinate information represents the position of the point cloud unit in the three-dimensional coordinate system, and the normal vector information represents the spatial orientation of the surface of the point cloud unit.

[0019] In this embodiment, the 3D point cloud dataset is stored in a local database. Each point cloud unit is stored in structured data format, containing a unique identifier, spatial coordinate information, normal vector information, and other attribute information. When extracting basic attribute information, all point cloud unit records are first traversed using a database query. For each point cloud unit, the spatial coordinate information is read from the corresponding record. This spatial coordinate information is stored as 3D coordinate values, containing the values ​​of the X, Y, and Z axes, representing the position of the point cloud unit in the three directions of the 3D coordinate system. Then, the normal vector information is read. This normal vector information is stored as a 3D vector, containing components in the X, Y, and Z directions, representing the orientation of the surface normal of the point cloud unit in the three directions. After extraction, the basic attribute information of each point cloud unit is stored in a temporary data structure indexed by the unique identifier of the point cloud unit for easy access in subsequent steps.

[0020] Step S112: Select the first point cloud unit in the three-dimensional point cloud data set as the initial target point cloud unit. Based on the spatial coordinate information of the initial target point cloud unit, set the spatial search range. The size of the spatial search range is determined based on the structural feature parameters of the inspection target and is used to filter adjacent point cloud units that have spatial correlation with the initial target point cloud unit.

[0021] In this embodiment, the 3D point cloud dataset is stored in the order of acquisition, with the first point cloud unit being the first discrete spatial point output by the acquisition device. When selecting the initial target point cloud unit, the first element of the dataset is directly read as the initial target point cloud unit. When setting the spatial search range, the structural feature parameters of the inspection target are first obtained. These structural feature parameters are read from the inspection task configuration file and include information such as the target's size and shape. Based on the structural feature parameters, the size of the spatial search range is calculated. This size is expressed as a spatial distance threshold and is used to limit the spatial range of adjacent point cloud units. Then, with the spatial coordinate information of the initial target point cloud unit as the center, a spatial search range is constructed. This spatial search range is a spherical region with the center coordinates as the center and the spatial distance threshold as the radius, used for subsequent filtering of adjacent point cloud units.

[0022] Step S113: Traverse other point cloud units in the three-dimensional point cloud data set within the spatial search range, and determine all point cloud units within the spatial search range as the set of adjacent point cloud units of the initial target point cloud unit.

[0023] In this embodiment, when traversing other point cloud units, the initial target point cloud unit itself is first excluded. Then, for each remaining point cloud unit, its spatial distance to the initial target point cloud unit is calculated. The spatial distance is calculated by taking the square root of the sum of the squares of the differences in three-dimensional coordinates. This result reflects the straight-line distance between the two point cloud units in three-dimensional space. The calculated spatial distance is compared with the size of the spatial search range. If the spatial distance is less than or equal to the size of the spatial search range, the point cloud unit is determined to be within the spatial search range. All point cloud units that meet the conditions are collected to form a set of adjacent point cloud units of the initial target point cloud unit. This set of adjacent point cloud units is stored in the form of a list of unique identifiers for point cloud units, facilitating access in subsequent steps.

[0024] Step S114: Calculate the spatial orientation parameters between the initial target point cloud unit and each adjacent point cloud unit in the adjacent point cloud unit set. Subtract the spatial coordinate information of the adjacent point cloud units from the spatial coordinate information of the initial target point cloud unit to obtain the spatial coordinate difference vector. Normalize the spatial coordinate difference vector to generate spatial orientation parameters that represent the relative directions of the two.

[0025] In this embodiment, when calculating the spatial orientation parameter, firstly, for each adjacent point cloud unit, the spatial coordinate information of the initial target point cloud unit and the adjacent point cloud unit is read and denoted as the first coordinate and the second coordinate, respectively. Then, the spatial coordinate difference vector is calculated, where each component of the spatial coordinate difference vector is the result of subtracting the corresponding component of the second coordinate from the component corresponding to the first coordinate. Next, the magnitude of the spatial coordinate difference vector is calculated, which is the square root of the sum of the squares of the vector's components. Dividing each component of the spatial coordinate difference vector by the magnitude yields a normalized vector, which is the spatial orientation parameter. The sum of the squares of each component is 1, and the direction points from the adjacent point cloud unit to the initial target point cloud unit. The spatial orientation parameter corresponding to each adjacent point cloud unit is stored in a temporary data structure, which is indexed by the unique identifier of the adjacent point cloud unit.

[0026] Step S115: Calculate the normal vector association parameter between the normal vector information of the initial target point cloud unit and the normal vector information of the adjacent point cloud unit. By calculating the cosine value of the angle between the two normal vectors, the cosine value of the angle is used as the normal vector association parameter to characterize the similarity between the two normal vectors.

[0027] In this embodiment, when calculating the normal vector association parameter, firstly, for each adjacent point cloud unit, the normal vector information of the initial target point cloud unit and the adjacent point cloud unit is read and denoted as the first normal vector and the second normal vector, respectively. Then, the dot product of the two normal vectors is calculated, which is the sum of the products of the corresponding components of the two vectors. Next, the magnitudes of the first and second normal vectors are calculated, which are the square roots of the sum of the squares of the components of the two vectors, respectively. The dot product result is divided by the product of the magnitudes of the two normal vectors to obtain the cosine value of the included angle. This cosine value is the normal vector association parameter, and its value range is [-1, 1]. The closer the value is to 1, the more similar the two normal vectors are; the closer the value is to -1, the more opposite the two normal vectors are. The normal vector association parameter corresponding to each adjacent point cloud unit is stored in a temporary data structure, which is indexed by the unique identifier of the adjacent point cloud unit.

[0028] Step S116: Take the initial target point cloud unit as a node in the point cloud spatial association network, take each adjacent point cloud unit in the adjacent point cloud unit set as the associated node corresponding to that node, and construct an association edge between the node corresponding to the initial target point cloud unit and the associated node.

[0029] In this embodiment, when constructing the point cloud spatial association network, an empty network structure is first created, which includes a set of nodes and a set of associated edges. Then, the initial target point cloud unit is added to the node set as a node, and its attribute field stores the basic attribute information of that point cloud unit. Next, for each adjacent point cloud unit in the adjacent point cloud unit set, if the adjacent point cloud unit is not yet in the node set, it is added to the node set as an associated node. Then, an associated edge is created between the node corresponding to the initial target point cloud unit and each associated node, and the attribute field of the associated edge reserves space for storing spatial association parameters. All created associated edges are added to the associated edge set, completing the construction of the nodes and associated edges of the initial target point cloud unit.

[0030] Step S117: The calculated spatial orientation parameters and normal vector association parameters are used as the spatial association parameters corresponding to the associated edge and stored in the attribute field of the associated edge.

[0031] In this embodiment, when storing spatial association parameters, firstly, for each associated edge, the unique identifier of the initial target point cloud unit and the associated node corresponding to that edge is read. Then, the spatial orientation parameter and normal vector association parameter corresponding to that associated edge are obtained from a temporary data structure indexed by the unique identifier of the associated node. Next, the spatial orientation parameter and normal vector association parameter are stored in the attribute field of the associated edge. The attribute field is stored in key-value pairs, with the keys being "spatial orientation parameter" and "normal vector association parameter," and the values ​​being the corresponding parameter values. After storage is completed, the corresponding associated edges in the associated edge set are updated to ensure a one-to-one correspondence between the spatial association parameters and the associated edges.

[0032] Step S118: Select the next unprocessed point cloud unit in the three-dimensional point cloud data set as the new target point cloud unit, and repeat the above operations of setting the spatial search range, determining the set of adjacent point cloud units, calculating spatial association parameters, constructing association edges and storing attribute fields.

[0033] In this embodiment, when selecting the next unprocessed point cloud unit, a set of processed point cloud units is first maintained. This set stores the unique identifiers of point cloud units that have completed node construction and edge processing. Then, all point cloud units in the 3D point cloud dataset are traversed, and the first point cloud unit not in the set of processed point cloud units is found as the new target point cloud unit. This new target point cloud unit is added to the set of processed point cloud units, and then steps S112 to S117 are repeated, including setting the spatial search range, determining the set of adjacent point cloud units, calculating spatial association parameters, constructing association edges, and storing attribute fields. The above process is repeated until all point cloud units have been processed.

[0034] Step S119: After all point cloud units in the 3D point cloud dataset have completed node creation and associated edge construction, integrate all nodes, associated edges, and associated edge attribute fields to form a point cloud spatial association network that includes the spatial association relationships of point cloud units.

[0035] In this embodiment, when integrating the point cloud spatial association network, it is first confirmed that the processed point cloud unit set contains all point cloud units in the 3D point cloud dataset. Then, all nodes in the node set, all associated edges in the associated edge set, and their attribute fields are integrated into a single network data structure. This data structure supports fast access and traversal of nodes and associated edges. After integration, the point cloud spatial association network is verified to check if the number of associated edges for each node is reasonable and if the spatial association parameters of the associated edges are complete. Once verification is successful, the point cloud spatial association network is stored in a local file or database for use in subsequent steps.

[0036] Step S120: Perform anomaly propagation simulation processing on the point cloud spatial association network. Based on the preset anomaly propagation rules, locate potential abnormal point cloud regions with anomaly propagation characteristics from the point cloud spatial association network. The potential abnormal point cloud regions are composed of nodes and associated edges corresponding to multiple consecutive point cloud units.

[0037] Step S121: Obtain the spatial correlation change pattern of the point cloud unit of the inspection target under abnormal state, and set the abnormal propagation rule based on the change pattern. The abnormal propagation rule includes the calculation method of the propagation weight of the associated edge, the triggering condition of the abnormal state of the node, and the termination condition of the propagation path.

[0038] In this embodiment, when obtaining the spatial association change pattern of point cloud units, the three-dimensional point cloud data of the inspection target under abnormal conditions is first read from the historical abnormal data set. This data set contains point cloud data with different abnormal types and degrees. Then, spatial association network construction processing is performed on the point cloud data under each abnormal state to obtain the corresponding abnormal spatial association network. The abnormal spatial association network is compared with the normal spatial association network to analyze the change pattern of spatial association parameters between point cloud units, including the change magnitude of spatial direction parameters, the change magnitude of normal vector association parameters, and the change of the number of associated edges. Based on the analysis of the change pattern, anomaly propagation rules are set, wherein the calculation method of the propagation weight of the associated edges reflects the degree of influence of spatial association parameters on anomaly propagation, the node abnormal state triggering condition is the threshold of the anomaly propagation intensity value, and the propagation path termination condition is that the anomaly propagation intensity value is lower than the preset threshold or the number of propagation nodes reaches the preset upper limit.

[0039] Step S122: Randomly select a node from the point cloud spatial association network as the initial propagation node, and assign an initial anomaly propagation intensity value to the initial propagation node. The initial anomaly propagation intensity value is set based on the characteristic change amplitude of the point cloud unit under the abnormal state.

[0040] In this embodiment, when selecting the initial propagation node, a list of all nodes in the point cloud spatial association network is first obtained. Then, a random number generator is used to randomly select a node from the list as the initial propagation node. When assigning an initial anomaly propagation intensity value to this initial propagation node, the characteristic change amplitude of the point cloud unit under anomaly conditions is first statistically analyzed from the historical anomaly data set. This characteristic change amplitude is the absolute value of the difference between the attribute information of the point cloud unit under anomaly conditions and that under normal conditions. Based on the statistically obtained change amplitude, an initial anomaly propagation intensity value is set, which reflects the degree of anomaly of the initial propagation node. The initial anomaly propagation intensity value is stored in the attribute field of the initial propagation node for use in subsequent steps.

[0041] Step S123: Calculate the propagation weight of all associated edges of the initial propagation node according to the calculation method of the associated edge propagation weight in the anomaly propagation rule. The propagation weight is related to the spatial association parameter corresponding to the associated edge. The larger the value of the spatial direction parameter's direction consistency evaluation result, the larger the value of the propagation weight. The larger the value of the normal vector association parameter, the larger the value of the propagation weight.

[0042] For example, step S1231: Extract the spatial association parameters corresponding to all associated edges of the initial propagation node, and separate the spatial direction parameters and normal vector association parameters of each associated edge. In this embodiment, firstly, all associated edges of the initial propagation node are read from the point cloud spatial association network, and the spatial association parameters are stored in the attribute field of each associated edge. For each associated edge, the spatial direction parameters and normal vector association parameters are separated and stored in a temporary data structure respectively.

[0043] Step S1232: Perform a directional consistency assessment on the spatial direction parameter of each associated edge. Calculate the average angle between this spatial direction parameter and the spatial direction parameters of other associated edges adjacent to the initial propagation node. The smaller the value of the average angle, the larger the value of the directional consistency assessment result. In this embodiment, for the spatial direction parameter of each associated edge, calculate the angle between this parameter and the spatial direction parameters of all other associated edges of the initial propagation node. The angle is calculated using the vector dot product formula. Sum all the angles and divide by the number of angles to obtain the average angle. The directional consistency assessment result is 1 minus the ratio of the average angle to π (π is pi). Therefore, the smaller the average angle, the larger the directional consistency assessment result.

[0044] Step S1233: Convert the directional consistency assessment result into a directional consistency coefficient. The value range of the directional consistency coefficient is a preset interval. The larger the value of the directional consistency assessment result, the larger the value of the directional consistency coefficient. In this embodiment, the preset interval is set to [0, 1]. The directional consistency coefficient is equal to the directional consistency assessment result. Therefore, the larger the directional consistency assessment result, the larger the directional consistency coefficient.

[0045] Step S1234: Extract the normal vector association parameter for each associated edge. The normal vector association parameter is the cosine value of the angle between the normal vectors of the initial propagation node and the associated node. The larger the value of the cosine value, the more significant the representation result corresponding to the normal vector similarity. In this embodiment, the normal vector association parameter is read from the attribute field of the associated edge. This normal vector association parameter is the cosine value of the angle between the normal vectors of the initial propagation node and the associated node. The larger the cosine value, the smaller the angle between the two normal vectors, and the higher the normal vector similarity.

[0046] Step S1235: Use the normal vector association parameter as the normal vector similarity coefficient. The value range of the normal vector similarity coefficient is consistent with the value range of the direction consistency coefficient. In this embodiment, the normal vector similarity coefficient is directly equal to the normal vector association parameter. Since the value range of the normal vector association parameter is [-1, 1], while the value range of the direction consistency coefficient is [0, 1], the normal vector association parameter is adjusted to convert it into a value in the interval [0, 1], for example, (normal vector association parameter + 1) / 2.

[0047] Step S1236: Set the weight ratio of the directional consistency coefficient and the normal vector similarity coefficient. The weight ratio is set based on the statistical analysis results of the influence of the two parameters on anomaly propagation. In this embodiment, based on the statistical analysis of historical anomaly propagation data, the weight of the directional consistency coefficient is set to 0.4, the weight of the normal vector similarity coefficient is set to 0.6, and the total weight ratio is 1.

[0048] Step S1237: Calculate the propagation weight of each associated edge. The propagation weight is the directional consistency coefficient multiplied by its weight plus the normal vector similarity coefficient multiplied by its weight. In this embodiment, for each associated edge, the directional consistency coefficient is multiplied by its weight, and the normal vector similarity coefficient is multiplied by its weight. Then, the two products are added together to obtain the propagation weight.

[0049] Step S1238: Compare the calculated propagation weight with the preset effective weight range. If the propagation weight is greater than the upper weight limit, adjust it to the upper weight limit; if the propagation weight is less than the lower weight limit, adjust it to the lower weight limit. In this embodiment, the preset effective weight range is [0.1, 0.9], the upper weight limit is 0.9, and the lower weight limit is 0.1. If the calculated propagation weight is greater than 0.9, adjust it to 0.9; if it is less than 0.1, adjust it to 0.1.

[0050] Step S1239: Determine the adjusted propagation weight as the final propagation weight of the associated edge, and perform the above calculation operation sequentially on all associated edges of the initial propagation node to obtain the propagation weight corresponding to each associated edge. In this embodiment, the adjusted propagation weight is used as the final propagation weight of the associated edge, and all associated edges of the initial propagation node are processed sequentially in the same way to obtain the final propagation weight of each associated edge.

[0051] Step S124: Based on the propagation weight and the initial abnormal propagation strength value, calculate the abnormal propagation strength value of each associated node of the initial propagation node. The abnormal propagation strength value of the associated node is the initial abnormal propagation strength value multiplied by the propagation weight of the corresponding associated edge.

[0052] In this embodiment, when calculating the anomaly propagation strength value of associated nodes, firstly, for each associated edge of the initial propagation node, the propagation weight corresponding to that edge and the initial anomaly propagation strength value of the initial propagation node are read. Then, the initial anomaly propagation strength value is multiplied by the propagation weight to obtain the anomaly propagation strength value of the associated node corresponding to that edge. The anomaly propagation strength value of each associated node is stored in a temporary data structure, which is indexed by the unique identifier of the associated node. If an associated node already has an anomaly propagation strength value, the maximum value is taken as the final anomaly propagation strength value.

[0053] Step S125: Compare the abnormal propagation intensity value of each associated node with the node abnormal state triggering condition in the abnormal propagation rule. If the abnormal propagation intensity value of the associated node meets the node abnormal state triggering condition, then mark the associated node as an abnormal state node and record the abnormal propagation path of the node.

[0054] In this embodiment, when comparing the anomaly propagation strength value, the node anomaly state triggering condition in the anomaly propagation rule is first read. This node anomaly state triggering condition is a threshold value for the anomaly propagation strength value. Then, for each associated node, its anomaly propagation strength value is read and compared with the threshold. If the anomaly propagation strength value is greater than or equal to the threshold, the associated node is determined to meet the triggering condition and is marked as an anomaly state node. The mark is stored in the node's attribute field. At the same time, the anomaly propagation path of the node is recorded. This anomaly propagation path is the sequence of associated edges from the initial propagation node to the associated node and is stored in the node's attribute field. If an associated node has already been marked as an anomaly state node, its anomaly propagation path is updated, and the path with the shorter propagation path length is taken as the final path.

[0055] Step S126: Using the associated node marked as abnormal as the new propagation node, repeat the above operations of calculating the propagation weight of the associated edge, calculating the abnormal propagation strength value of the new associated node, comparing the triggering conditions, and marking the abnormal state node.

[0056] In this embodiment, when using an associated node marked as an abnormal state as a new propagation node, firstly, a node that has not yet been used as a propagation node is selected from the already marked abnormal state nodes and designated as the new propagation node. This new propagation node is added to the set of processed propagation nodes, and then steps S123 to S125 are repeated, including calculating the propagation weight of the associated edge, calculating the abnormal propagation strength value of the new associated node, comparing trigger conditions, and marking abnormal state nodes. This process is repeated until all associated nodes of the new propagation node have been processed, or the propagation process meets the propagation path termination condition.

[0057] Step S127: Compare the current state of the propagation process with the propagation path termination conditions in the abnormal propagation rules. If the propagation process meets the propagation path termination conditions, stop the propagation process of that propagation path. The propagation path termination conditions include the abnormal propagation intensity value being lower than a preset propagation threshold and the number of propagation nodes reaching a preset upper limit.

[0058] In this embodiment, when comparing the propagation path termination conditions, the propagation path termination conditions in the abnormal propagation rules are first read, including a preset propagation threshold and a preset upper limit. Then, for the current propagation path, the current state of the propagation process is read, including the abnormal propagation intensity value of the current propagation node and the number of processed propagation nodes. If the abnormal propagation intensity value of the current propagation node is lower than the preset propagation threshold, or the number of processed propagation nodes reaches the preset upper limit, the propagation process is determined to meet the termination conditions, and the propagation process of that propagation path is stopped. The termination state is stored in the attribute field of the propagation path for use in subsequent steps.

[0059] Step S128: Select the next node in the point cloud spatial association network that is not the initial propagation node as the new initial propagation node, and repeat the above operations of assigning an initial abnormal propagation strength value, calculating the propagation weight, the propagation abnormality strength, and terminating propagation.

[0060] In this embodiment, when selecting a new initial propagation node, a set of nodes that have already served as initial propagation nodes is first maintained. This set stores the unique identifiers of nodes that have completed the initial propagation node processing. Then, all nodes in the point cloud spatial association network are traversed, and the first node not in the set of nodes that have already served as initial propagation nodes is found and designated as the new initial propagation node. This new initial propagation node is added to the set of nodes that have already served as initial propagation nodes, and then steps S122 to S127 are repeated, including assigning an initial anomaly propagation strength value, calculating propagation weights, propagation anomaly strength, and terminating propagation. This process is repeated until all nodes have completed the initial propagation node processing.

[0061] Step S129: Collect the set of all nodes marked as abnormal state nodes, and determine each continuously connected cluster of abnormal state nodes and its corresponding associated edge as a potential abnormal point cloud region with abnormal propagation characteristics.

[0062] In this embodiment, when collecting the set of anomalous state nodes, all nodes in the point cloud spatial association network are first traversed, and nodes marked as anomalous are selected to form a set of anomalous state nodes. Then, cluster analysis is performed on the set of anomalous state nodes. A connectivity clustering algorithm is used to cluster consecutively connected anomalous state nodes into a cluster, with the criterion for consecutive connection being the existence of an associated edge between nodes. For each cluster, all nodes and their corresponding associated edges are collected to form a potential anomalous point cloud region. All potential anomalous point cloud regions are stored in a temporary data structure, indexed by a unique region identifier.

[0063] Step S1210: Perform deduplication on all identified potential anomalous point cloud regions, removing duplicate or contained regions to form the final set of potential anomalous point cloud regions.

[0064] In this embodiment, during deduplication, the number of nodes and associated edges contained in each potential anomalous point cloud region are first calculated as its feature information. Then, the feature information of all potential anomalous point cloud regions is compared. If the node sets of two regions are completely identical, they are determined to be duplicate regions; one region is retained, and the other is deleted. If the node set of one region contains the node set of another region, an inclusion relationship is determined; the region with more nodes is retained, and the region with fewer nodes is deleted. This comparison process is repeated until all potential anomalous point cloud regions have been deduplicated. The deduplicated potential anomalous point cloud regions are stored in the final potential anomalous point cloud region set for use in subsequent steps.

[0065] Step S130: Call the pre-trained inspection feature enhancement model to perform feature interaction enhancement processing on the point cloud data corresponding to the potential abnormal point cloud region, and generate an enhanced point cloud feature set with anomaly feature-specific representation. The enhanced point cloud feature set includes the correlation representation of the spatial structure features and point cloud unit attribute features of the potential abnormal point cloud region.

[0066] Step S131: Divide the point cloud data corresponding to the potential abnormal point cloud region into spatial structure data and attribute data. The spatial structure data includes the spatial coordinate relationship of the point cloud units in the region, and the attribute data includes the normal vector information and basic attribute information of the point cloud units in the region.

[0067] In this embodiment, when segmenting point cloud data, firstly, all point cloud units within the potentially abnormal point cloud region are acquired, and the spatial coordinate information, normal vector information, and basic attribute information of each point cloud unit are read. Then, spatial structure data is constructed, which includes the spatial coordinate relationships between point cloud units within the region, specifically a spatial distance matrix and a spatial direction parameter matrix between point cloud units. Each element of the spatial distance matrix represents the spatial distance between two point cloud units, and each element of the spatial direction parameter matrix represents the spatial direction parameter between two point cloud units. Attribute data is then constructed, which includes the normal vector information and basic attribute information of each point cloud unit within the region, specifically a normal vector matrix and a basic attribute matrix. Each element of the normal vector matrix represents the normal vector component of the point cloud unit, and each element of the basic attribute matrix represents the basic attribute value of the point cloud unit. The spatial structure data and attribute data are stored in different temporary data structures for use in subsequent steps.

[0068] Step S132: Input the spatial structure data into the spatial structure feature extraction layer of the inspection feature enhancement model. The spatial structure feature extraction layer contains three-level feature transformation units. The first-level transformation unit performs coordinate association encoding on the spatial structure data and converts the spatial coordinate association relationship into an initial structural feature vector.

[0069] In this embodiment, the three-level feature transformation units of the spatial structure feature extraction layer are connected sequentially. The first-level transformation unit is a coordinate association encoding layer, implemented using a fully connected neural network. When inputting spatial structure data, the spatial distance matrix and spatial orientation parameter matrix are first flattened into a one-dimensional vector, which serves as the input to the first-level transformation unit. The first-level transformation unit encodes the input vector and converts it into an initial structural feature vector through a multilayer perceptron. This initial structural feature vector contains a compressed representation of the spatial coordinate association relationship. The dimension of the initial structural feature vector is set by the model configuration file and includes multiple feature dimensions, each representing one aspect of the spatial coordinate association relationship. The initial structural feature vector is then output to the second-level transformation unit.

[0070] Step S133: The second-level transformation unit performs dimensional expansion processing on the initial structural feature vector. By adding feature dimensions derived from coordinate relationships to the initial structural feature vector, an extended structural feature vector is generated. The derived feature dimensions include the coordinate difference statistical features and spatial distribution density features of adjacent point cloud units.

[0071] In this embodiment, the second-level transformation unit is a dimension expansion layer, comprising a feature derivation module and a feature concatenation module. The feature derivation module first analyzes the spatial coordinate relationships in the initial structural feature vector, calculates the statistical features of coordinate differences between adjacent point cloud units, including the average, variance, and maximum value of the coordinate differences; and calculates the spatial distribution density features, including the local and global densities of the point cloud units. Then, it concatenates the derived feature dimensions with the initial structural feature vector to generate an extended structural feature vector, the dimension of which is the dimension of the initial structural feature vector plus the number of derived feature dimensions. The extended structural feature vector is then output to the third-level transformation unit.

[0072] Step S134: The third-level transformation unit performs feature recombination processing on the extended structure feature vector, groups and fuses the similar feature dimensions in the extended structure feature vector to generate a spatial structure feature vector that is representative of the spatial structure.

[0073] In this embodiment, the third-level transformation unit is a feature reorganization layer, which includes a feature similarity calculation module and a feature fusion module. The feature similarity calculation module first calculates the similarity between each feature dimension in the extended structural feature vector using a cosine similarity algorithm to obtain a similarity matrix. Then, based on the similarity matrix, the feature dimensions are grouped, with feature dimensions having a similarity greater than a threshold grouped together. For each group, the feature fusion module fuses the feature dimensions within that group using a weighted average method. The weights are set based on the importance of the feature dimensions, which is obtained through statistical analysis of historical data. The fusion results of all groups are concatenated to generate a spatial structure feature vector. The dimension of this spatial structure feature vector is the number of groups, with each dimension representing a spatial structure feature. The spatial structure feature vector is then output to the feature interaction layer.

[0074] Step S135: Input the attribute data into the attribute feature extraction layer of the inspection feature enhancement model. The attribute feature extraction layer includes an attribute mapping unit and a feature aggregation unit. The attribute mapping unit converts the normal vector information and basic attribute information of each point cloud unit into a single point cloud unit attribute feature vector.

[0075] In this embodiment, the attribute mapping unit and the feature aggregation unit of the attribute feature extraction layer are connected sequentially. The attribute mapping unit is a fully connected neural network that processes the normal vector information and basic attribute information of each point cloud unit. When inputting attribute data, the normal vector information and basic attribute information of each point cloud unit are first flattened into a one-dimensional vector, which is then used as the input to the attribute mapping unit. The attribute mapping unit encodes each input vector and converts it into a single point cloud unit attribute feature vector through a multilayer perceptron. This single point cloud unit attribute feature vector contains a compressed representation of the point cloud unit attribute information. The dimension of the single point cloud unit attribute feature vector is set by the model configuration file and includes multiple feature dimensions, each representing one aspect of the point cloud unit attribute. All single point cloud unit attribute feature vectors are then output to the feature aggregation unit.

[0076] Step S136: The feature aggregation unit performs weighted aggregation processing on the attribute feature vectors of all single point cloud units. The aggregation weight is set based on the anomaly propagation intensity value of each point cloud unit in the potential abnormal point cloud region. The larger the value corresponding to the anomaly propagation intensity value, the larger the value corresponding to the aggregation weight, and the region attribute feature vector is generated.

[0077] Step S1361: Extract the anomaly propagation intensity value corresponding to each point cloud unit in the potential anomaly point cloud region to form an anomaly propagation intensity value list. Each element in the anomaly propagation intensity value list corresponds one-to-one with the attribute feature vector of a single point cloud unit. In this embodiment, firstly, the anomaly propagation intensity value of each point cloud unit is read from the node attribute field of the potential anomaly point cloud region. These values ​​are calculated and stored during the anomaly propagation simulation process. The above anomaly propagation intensity values ​​are arranged according to the order of the point cloud units in the potential anomaly point cloud region to form an anomaly propagation intensity value list. The nth element in the list corresponds to the attribute feature vector of the nth single point cloud unit.

[0078] Step S1362: Calculate the sum of all abnormal propagation intensity values ​​in the abnormal propagation intensity value list. Divide the abnormal propagation intensity value of each point cloud unit by this sum to obtain the normalized weight value corresponding to each point cloud unit. The sum of all normalized weight values ​​is a preset value. In this embodiment, firstly, all values ​​in the abnormal propagation intensity value list are summed to obtain a total. Then, for each point cloud unit, its abnormal propagation intensity value is divided by this sum to obtain a normalized weight value. Since the sum of all abnormal propagation intensity values ​​is used as the denominator, the sum of all normalized weight values ​​will equal 1, which is the preset value.

[0079] Step S1363: Select the first single point cloud unit attribute feature vector, and multiply the value of each feature dimension in the single point cloud unit attribute feature vector by the corresponding normalized weight value to obtain the weighted first attribute feature vector. In this embodiment, firstly, the first single point cloud unit attribute feature vector is selected, which contains values ​​of multiple feature dimensions. Then, the normalized weight value corresponding to the point cloud unit is read, and the value of each feature dimension in the vector is multiplied by the normalized weight value to obtain the weighted first attribute feature vector. The dimension of the first attribute feature vector is consistent with the original vector.

[0080] Step S1364: Select the second single-point cloud unit attribute feature vector, and repeat the above operation of multiplying by the corresponding normalized weight value to obtain the weighted second attribute feature vector. In this embodiment, the second single-point cloud unit attribute feature vector is selected, and the value of each feature dimension in the vector is multiplied by the corresponding normalized weight value in the same way as in step S1363 to obtain the weighted second attribute feature vector.

[0081] Step S1365: Perform the aforementioned weighting process sequentially on the attribute feature vectors of all single point cloud units in the potential anomaly point cloud region to obtain multiple weighted attribute feature vectors. Then, sum the values ​​of all weighted attribute feature vectors along each feature dimension. The sum of values ​​for each feature dimension is the sum of the values ​​of all weighted attribute feature vectors along that feature dimension. In this embodiment, each single point cloud unit attribute feature vector in the potential anomaly point cloud region is processed sequentially to obtain multiple weighted attribute feature vectors. Then, for each feature dimension, the values ​​of all weighted attribute feature vectors along that dimension are summed to obtain the sum of values ​​for that feature dimension.

[0082] Step S1366: The accumulated value of each feature dimension is directly used as the value of the corresponding feature dimension of the regional attribute feature vector. All feature dimension values ​​are arranged in the original feature dimension order to form a regional attribute feature vector that is representative of the regional attributes. In this embodiment, the accumulated value of each feature dimension is used as the value of the corresponding dimension of the regional attribute feature vector, keeping the order of the feature dimensions consistent with the dimensional order of the single point cloud unit attribute feature vector, thereby forming the regional attribute feature vector.

[0083] Step S1367: Perform a feature validity check on the generated regional attribute feature vector to confirm that the average value of each feature dimension is within the preset reasonable range of attribute features. If all dimensions are within the reasonable range, the regional attribute feature vector is retained. If there are dimensions that exceed the reasonable range, the weighted cumulative value corresponding to that feature dimension is recalculated, corrected, and checked again until all dimensions meet the requirements. In this embodiment, a reasonable range of attribute features is first set, which is determined based on the statistical analysis results of historical normal data. Then, the value of each feature dimension of the regional attribute feature vector is calculated, and it is checked whether it is within the corresponding reasonable range. If all dimensions meet the requirements, the vector is retained. If there are dimensions that do not meet the requirements, the weighted cumulative value of that dimension is recalculated, for example, by adjusting the weight of that dimension or reselecting valid samples, corrected, and checked again until all dimensions meet the requirements.

[0084] Step S137: Input the spatial structure feature vector and the regional attribute feature vector into the feature interaction layer of the inspection feature enhancement model. The feature interaction layer includes a bidirectional association mapping unit, which constructs a dimension association table between the spatial structure feature vector and the regional attribute feature vector.

[0085] In this embodiment, the bidirectional association mapping unit of the feature interaction layer is an association mapping module, which includes a dimensional association analysis subunit and an association table construction subunit. When the spatial structure feature vector and the regional attribute feature vector are input, the two vectors are first flattened into one-dimensional vectors, which are then used as input to the dimensional association analysis subunit. The dimensional association analysis subunit calculates the association degree between each feature dimension of the two vectors, using a mutual information algorithm to obtain the association degree matrix. The association table construction subunit constructs a dimensional association table based on the association degree matrix. This dimensional association table contains the correspondence between the feature dimensions of the spatial structure feature vector and the feature dimensions of the regional attribute feature vector, with the correspondence established based on dimensional pairs with an association degree greater than a threshold. The dimensional association table is then output to the subsequent processing unit of the feature interaction layer.

[0086] Step S138: Calculate the correlation coefficient between each dimension of the two feature vectors based on the dimensional correlation table. The correlation coefficient represents the degree of collaborative contribution of different feature dimensions to anomaly identification. The larger the value of the correlation coefficient, the more significant the representation result corresponding to the degree of collaborative contribution.

[0087] In this embodiment, when calculating the correlation coefficient, the feature dimension correspondences in the dimension association table are first read. For each correspondence, the correlation coefficient between the feature dimension of the spatial structure feature vector and the feature dimension of the regional attribute feature vector is calculated. The correlation coefficient is calculated using a weighted summation method, with the weights set based on the correlation degree in the correlation matrix; the higher the correlation degree, the higher the weight. The correlation coefficient ranges from [0, 1], with larger values ​​indicating a more significant degree of collaborative contribution. All correlation coefficients are stored in a temporary data structure indexed by feature dimension pairs.

[0088] Step S139: Based on the correlation coefficient, the spatial structure feature vector and the regional attribute feature vector are weighted and updated. The feature dimension value with the larger the correlation coefficient value is amplified by a higher ratio, and the feature dimension value with the smaller the correlation coefficient value is retained by a higher ratio, thereby generating an interactive feature vector.

[0089] In this embodiment, when updating the weighted feature vector, firstly, for each feature dimension of the spatial structure feature vector, its correlation coefficient with each feature dimension of the regional attribute feature vector is read, and the weighted update coefficient of that feature dimension is calculated. The weighted update coefficient is the weighted sum of all correlation coefficients, and the weights are set based on the importance of the feature dimensions of the regional attribute feature vector. Then, the value of that feature dimension of the spatial structure feature vector is multiplied by the weighted update coefficient to obtain the updated value. Similarly, for each feature dimension of the regional attribute feature vector, its weighted update coefficient is calculated, and the value of that feature dimension is multiplied by the weighted update coefficient to obtain the updated value. The updated spatial structure feature vector and the regional attribute feature vector are concatenated to generate an interactive feature vector. The dimension of this vector is the sum of the dimensions of the two vectors, and each dimension represents the feature after the weighted update.

[0090] Step S1310: Perform feature smoothing on the interactive feature vector, eliminate numerical fluctuations in the interactive feature vector by using the sliding window averaging method, and then perform dimension regularization on the smoothed interactive feature vector so that the feature vector dimension conforms to the preset standard dimension.

[0091] In this embodiment, during feature smoothing, the interactive feature vectors are first arranged in order of feature dimensions. A sliding window averaging method is then used to smooth the values ​​of each feature dimension. The size of the sliding window is set in the model configuration file. For each feature dimension, the average value of all values ​​within the window is calculated as the smoothed value. Then, the smoothed interactive feature vectors undergo dimension normalization. Principal component analysis is used to compress the dimensions of the feature vectors to a preset standard dimension, which is also set in the model configuration file. The dimension-normalized interactive feature vectors contain the main feature information, reducing the feature dimensions and improving the efficiency of subsequent processing.

[0092] Step S1311: Determine the interaction feature vector after dimension normalization as an enhanced point cloud feature set with anomaly feature-specific representation. Each feature dimension in the enhanced point cloud feature set contains the correlation information of spatial structure features and point cloud unit attribute features.

[0093] In this embodiment, when determining the enhanced point cloud feature set, the dimensions of the dimension-normalized interactive feature vectors are first verified to conform to a preset standard dimension. If they do, they are used as the enhanced point cloud feature set. Each feature dimension in the enhanced point cloud feature set contains the association information between spatial structure features and point cloud unit attribute features. This association information is obtained through weighted update processing of the feature interaction layer. The enhanced point cloud feature set is stored in a temporary data structure indexed by the unique identifier of the potential abnormal point cloud region for use in subsequent steps.

[0094] Step S140: Perform feature mapping association processing on the enhanced point cloud feature set and the inspection target abnormal feature sample library to determine the inspection attention area unit with abnormal feature matching degree from the potential abnormal point cloud region.

[0095] Step S141: Collect multiple sets of 3D point cloud anomaly sample data of the inspection target under different anomaly types. Each set of 3D point cloud anomaly sample data is labeled with the corresponding anomaly type label and anomaly area range.

[0096] In this embodiment, when collecting 3D point cloud anomaly sample data, sample data of the inspection target under different anomaly types are first filtered from the historical anomaly data set. Anomaly types include, but are not limited to, deformation, corrosion, and cracks. Each set of sample data includes 3D point cloud data, anomaly type labels, and anomaly region range. The anomaly type labels are represented by enumerated values, and the anomaly region range is represented by a set of unique identifiers for point cloud units. The collected sample data is verified to check the accuracy of the anomaly type labels and anomaly region ranges, ensuring the quality of the sample data. Verified sample data is stored in a temporary data structure for use in subsequent steps.

[0097] Step S142: Perform spatial association network construction processing on each group of 3D point cloud anomaly sample data to obtain the sample spatial association network corresponding to each group of samples.

[0098] In this embodiment, when performing the spatial association network construction process, for each group of 3D point cloud anomaly sample data, the operations of steps S110 to S119 are repeated, including extracting basic attribute information, setting the spatial search range, determining the set of adjacent point cloud units, calculating spatial association parameters, constructing association edges, and integrating the point cloud spatial association network. This yields a sample spatial association network corresponding to each group of samples, which contains the spatial association relationships of point cloud units in the sample data. The sample spatial association network is stored in a temporary data structure indexed by the unique identifier of the sample data.

[0099] Step S143: Perform anomaly propagation simulation processing on the spatial correlation network of each group of samples to locate the potential anomaly point cloud region corresponding to each group of samples.

[0100] In this embodiment, when performing anomaly propagation simulation processing, steps S120 to S1210 are repeated for each group of sample spatial association networks, including setting anomaly propagation rules, selecting initial propagation nodes, calculating propagation weights, propagation anomaly intensity, marking anomalous state nodes, and determining potential anomalous point cloud regions. The potential anomalous point cloud region corresponding to each group of samples is located. This potential anomalous point cloud region contains point cloud units in the sample data that exhibit anomaly propagation characteristics. The potential anomalous point cloud region is stored in a temporary data structure indexed by the unique identifier of the sample data.

[0101] Step S144: Call the inspection feature enhancement model to perform feature interaction enhancement processing on the point cloud data corresponding to the potential abnormal point cloud region of each group of samples, and generate a sample enhanced point cloud feature set corresponding to each group of samples.

[0102] In this embodiment, when the inspection feature enhancement model is invoked, steps S130 to S1311 are repeated for the point cloud data corresponding to the potential abnormal point cloud regions of each group of samples. This includes dividing the spatial structure data and attribute data, extracting spatial structure feature vectors, extracting regional attribute feature vectors, performing feature interaction enhancement processing, and generating an enhanced point cloud feature set. An enhanced point cloud feature set is generated for each group of samples, containing a unique representation of the abnormal features in the sample data. The enhanced point cloud feature set is stored in a temporary data structure indexed by the unique identifier of the sample data.

[0103] Step S145: Associate and store the enhanced point cloud feature set, anomaly type label and anomaly region range corresponding to each group of samples, classify the sample data according to the anomaly type label, and construct a sample library of anomaly features of inspection targets.

[0104] In this embodiment, when associating and storing sample data, firstly, for each group of sample data, its enhanced point cloud feature set, anomaly type label, and anomaly region range are read. Then, the enhanced point cloud feature set is stored in vector form, the anomaly type label in enumerated value form, and the anomaly region range in a set of unique identifiers for point cloud units. These three types of information are associated through the unique identifiers of the sample data and stored in the sample data table of the database. Next, the sample data is classified according to the anomaly type label, and an anomaly type index table is created, recording the list of unique identifiers for each anomaly type. Finally, the sample data table and the anomaly type index table are optimized to improve the query efficiency of subsequent feature mapping association processing, thus constructing a sample library of anomaly features for inspection targets.

[0105] Step S146: Extract the feature distribution information of the enhanced point cloud feature set. The feature distribution information includes the numerical range of each feature dimension, the frequency of numerical occurrence, and the correlation between feature dimensions.

[0106] In this embodiment, when extracting feature distribution information, firstly, the values ​​of all feature dimensions of the enhanced point cloud feature set are read. For each feature dimension, its numerical range is calculated, i.e., the minimum and maximum values ​​of that dimension. The frequency of occurrence of values ​​in that dimension is then statistically analyzed, i.e., the percentage of occurrences of values ​​within different numerical ranges. The correlation between that dimension and all other feature dimensions is calculated using the Pearson correlation coefficient algorithm to obtain a correlation matrix. The numerical range, frequency of occurrence, and correlation with other dimensions of each feature dimension are integrated to form the feature distribution information of the enhanced point cloud feature set. This feature distribution information is stored in a temporary data structure for use in subsequent steps.

[0107] Step S147: Select all enhanced point cloud feature sets corresponding to one type of anomaly from the inspection target anomaly feature sample library, and extract the sample feature distribution information of each enhanced point cloud feature set.

[0108] In this embodiment, when selecting samples corresponding to anomaly types, firstly, an anomaly type is selected from the anomaly type index table of the inspection target anomaly feature sample library, and a list of unique identifiers for sample data corresponding to that anomaly type is obtained. Then, for each unique identifier of sample data, the corresponding sample enhanced point cloud feature set is read from the sample data table. Next, for each sample enhanced point cloud feature set, the operation of step S146 is repeated to extract its sample feature distribution information. This sample feature distribution information includes the numerical range of each feature dimension, the frequency of numerical occurrence, and the correlation between feature dimensions. The sample feature distribution information of all samples is stored in a temporary data structure, which is indexed by the unique identifier of the sample data.

[0109] Step S148: Calculate the distribution similarity between the feature distribution information of the enhanced point cloud feature set and the feature distribution information of each sample. By calculating the overlap of the numerical range of the corresponding feature dimensions, the similarity of the frequency of numerical occurrence, and the correlation deviation of the feature dimensions in the two feature distribution information, a comprehensive distribution similarity is generated.

[0110] Step S1481: Extract the numerical range of the first feature dimension from the feature distribution information of the enhanced point cloud feature set, denoted as the target numerical range, which includes the minimum and maximum values ​​of that feature dimension; extract the sample numerical range corresponding to the first feature dimension from the sample feature distribution information, which includes the minimum and maximum values ​​of that feature dimension. In this embodiment, firstly, the minimum and maximum values ​​of the first feature dimension are read from the feature distribution information of the enhanced point cloud feature set to form the target numerical range. Then, the minimum and maximum values ​​of the samples corresponding to the first feature dimension are read from the sample feature distribution information to form the sample numerical range.

[0111] Step S1482: Calculate the length of the overlapping interval between the target value range and the sample value range. The length of the overlapping interval is the maximum value of the overlapping part minus the minimum value of the overlapping part. If there is no overlap, the length of the overlapping interval is zero. In this embodiment, the overlapping part of the target value range and the sample value range is first determined. The minimum value of the overlapping part is the larger of the two minimum values ​​of the ranges, and the maximum value of the overlapping part is the smaller of the two maximum values ​​of the ranges. If the minimum value of the overlapping part is greater than the maximum value, there is no overlap, and the length of the overlapping interval is zero; otherwise, the length of the overlapping interval is the maximum value of the overlapping part minus the minimum value.

[0112] Step S1483: Divide the length of the overlapping interval by the combined interval length of the target numerical range and the sample numerical range to obtain the numerical range overlap of this feature dimension. In this embodiment, firstly, the combined interval length of the target numerical range and the sample numerical range is calculated. The minimum value of the combined interval is the smaller of the two minimum values ​​of the ranges, and the maximum value of the combined interval is the larger of the two maximum values ​​of the ranges. The combined interval length is the maximum value minus the minimum value. Then, divide the length of the overlapping interval by the combined interval length to obtain the numerical range overlap. The value range of the numerical range overlap is [0, 1], and the larger the value, the higher the degree of overlap.

[0113] Step S1484: Extract the frequency distribution of numerical occurrences in the first feature dimension from the feature distribution information of the enhanced point cloud feature set, denoted as the target frequency distribution. The target frequency distribution includes multiple numerical intervals and the percentage of occurrences for each interval. Extract the frequency distribution of sample numerical occurrences corresponding to the first feature dimension from the sample feature distribution information. The sample numerical occurrence frequency distribution includes multiple sample numerical intervals and the percentage of occurrences for each interval. In this embodiment, firstly, the frequency distribution of numerical occurrences in the first feature dimension is read from the feature distribution information of the enhanced point cloud feature set. This frequency distribution includes multiple numerical intervals and the percentage of occurrences for each interval. Then, the frequency distribution of sample numerical occurrences corresponding to the first feature dimension is read from the sample feature distribution information, including multiple sample numerical intervals and the percentage of occurrences for each interval.

[0114] Step S1485: Align the intervals of the target frequency distribution with the intervals of the sample value occurrence frequency distribution, calculate the absolute difference of the proportion of occurrences within each aligned interval, and take the average of all absolute differences as the numerical occurrence frequency deviation for this feature dimension. In this embodiment, firstly, the intervals of the target frequency distribution and the sample value occurrence frequency distribution are aligned, for example, by adjusting the boundaries of all intervals to the same value. Then, for each aligned interval, the absolute difference of the proportion of occurrences of the target frequency distribution and the sample value occurrence frequency distribution within that interval is calculated. All absolute differences are summed and then divided by the number of aligned intervals to obtain the numerical occurrence frequency deviation.

[0115] Step S1486: Subtract the frequency deviation of the numerical occurrences from the preset baseline value to obtain the similarity of the frequency occurrences of the numerical occurrences of this feature dimension. In this embodiment, the preset baseline value is usually set to 1. Subtract the frequency deviation of the numerical occurrences from 1 to obtain the similarity of the frequency occurrences of the numerical occurrences. The value ranges from [0, 1]. The larger the value, the more similar the frequency distribution.

[0116] Step S1487: Calculate the target correlation coefficients of the first feature dimension with all other feature dimensions in the feature distribution information of the enhanced point cloud feature set, forming a target correlation matrix; calculate the sample correlation coefficients of the first feature dimension with all other feature dimensions in the sample feature distribution information, forming a sample correlation matrix. In this embodiment, firstly, the target correlation coefficients of the first feature dimension with all other feature dimensions are read from the feature distribution information of the enhanced point cloud feature set, and these coefficients form the target correlation matrix. Then, the sample correlation coefficients of the first feature dimension with all other feature dimensions are read from the sample feature distribution information, forming a sample correlation matrix.

[0117] Step S1488: Calculate the absolute difference between corresponding elements in the target correlation matrix and the sample correlation matrix, and take the average of all absolute differences as the feature dimension correlation bias. In this embodiment, the absolute difference is calculated for each corresponding element in the target correlation matrix and the sample correlation matrix. All absolute differences are summed and then divided by the number of matrix elements to obtain the feature dimension correlation bias.

[0118] Step S1489: Perform the above-described operations of calculating the overlap of numerical ranges, the similarity of numerical occurrence frequencies, and the correlation deviation of feature dimensions on all feature dimensions in the feature distribution information of the enhanced point cloud feature set. In this embodiment, following the same method as steps S1481 to S1488, all feature dimensions in the feature distribution information of the enhanced point cloud feature set are processed sequentially, and the overlap of numerical ranges, the similarity of numerical occurrence frequencies, and the correlation deviation of feature dimensions for each feature dimension are calculated.

[0119] Step S14810: Calculate a weighted average of the numerical range overlap, numerical frequency similarity, and feature dimension correlation deviation for all feature dimensions. The weights are set based on the degree of influence of each indicator on abnormal matching. In this embodiment, the weights of each indicator are first set, determined based on the statistical analysis results of historical data. For example, the weight of numerical range overlap is 0.3, the weight of numerical frequency similarity is 0.4, and the weight of feature dimension correlation deviation is 0.3. Then, the weighted sum of the numerical range overlaps for all feature dimensions is calculated and divided by the number of feature dimensions to obtain the average numerical range overlap. Similarly, the average numerical frequency similarity and average feature dimension correlation deviation are calculated.

[0120] Step S14811: Sum the weighted average numerical range overlap, numerical frequency similarity, and (preset benchmark value minus weighted average feature dimension correlation deviation), then divide by the number of indicators to obtain the final distribution similarity. In this embodiment, the preset benchmark value is set to 1. First, 1 is subtracted from the average feature dimension correlation deviation to obtain the corrected feature dimension correlation index. Then, the average numerical range overlap, average numerical frequency similarity, and corrected feature dimension correlation index are summed, then divided by 3 (number of indicators) to obtain the final distribution similarity.

[0121] Step S149: Calculate the average distribution similarity of the enhanced point cloud feature sets of all samples under the same anomaly type to obtain the average distribution similarity corresponding to that anomaly type.

[0122] In this embodiment, when calculating the average distribution similarity, the distribution similarity of all samples corresponding to the same anomaly type is first read. These distribution similarities are then summed and divided by the number of samples to obtain the average distribution similarity for that anomaly type. The average distribution similarity reflects the overall similarity between the enhanced point cloud feature set and all samples of that anomaly type. The average distribution similarity is stored in a temporary data structure indexed by the anomaly type.

[0123] Step S1410: Perform the above operations of extracting sample feature distribution information, calculating distribution similarity and averaging for all anomaly types in the inspection target anomaly feature sample library in sequence to obtain the average distribution similarity corresponding to each anomaly type.

[0124] In this embodiment, when processing all anomaly types sequentially, the anomaly type index table of the inspection target anomaly feature sample library is first traversed to obtain all anomaly types. Then, for each anomaly type, steps S147 to S149 are repeated, including selecting samples, extracting sample feature distribution information, calculating distribution similarity, and averaging to obtain the average distribution similarity corresponding to that anomaly type. The average distribution similarities of all anomaly types are stored in a temporary data structure indexed by the anomaly type.

[0125] Step S1411: Select the anomaly type with the largest value corresponding to the average distribution similarity as the matching anomaly type, and extract the range of all sample anomaly regions corresponding to the matching anomaly type.

[0126] In this embodiment, when selecting a matching anomaly type, the average distribution similarity of all anomaly types is first read, and the values ​​are compared. The anomaly type with the largest value is selected as the matching anomaly type. Then, a list of unique identifiers for sample data corresponding to the matching anomaly type is obtained from the anomaly type index table in the inspection target anomaly feature sample library. For each unique identifier, the corresponding sample anomaly region is read from the sample data table. This sample anomaly region is stored in the form of a set of unique identifiers for point cloud units. All sample anomaly region ranges are stored in a temporary data structure indexed by the unique identifiers of the sample data.

[0127] Step S1412: Calculate the regional overlap between the potential abnormal point cloud region and the range of each sample abnormal region. The regional overlap is the number of point cloud units in both regions divided by the total number of point cloud units in the potential abnormal point cloud region.

[0128] In this embodiment, when calculating the region overlap, the first step is to read the set of unique identifiers of point cloud units in the potential abnormal point cloud region, and the set of unique identifiers of point cloud units in the abnormal region range of each sample. Then, the number of point cloud units shared by both sets is calculated, i.e., the number of elements in the intersection of the two sets. Next, the total number of point cloud units in the potential abnormal point cloud region is calculated, i.e., the number of elements in that set. The region overlap is obtained by dividing the number of shared point cloud units by the total number. The region overlap range is [0, 1], with a larger value indicating a higher degree of overlap. The region overlap of each sample is stored in a temporary data structure indexed by the unique identifier of the sample data.

[0129] Step S1413: Select the sample abnormal region range with the largest corresponding regional overlap as the reference abnormal region range, and determine the range of point cloud units in the potential abnormal point cloud that are common to the reference abnormal region range as the inspection focus area unit with abnormal feature matching degree.

[0130] In this embodiment, when selecting the reference anomaly region range, the region overlap of all samples is first read, and the values ​​are compared. The anomaly region range of the sample with the largest value is selected as the reference anomaly region range. Then, the set of unique identifiers of point cloud units common to the potential anomaly point cloud region and the reference anomaly region range is calculated. This set of unique identifiers of point cloud units is the inspection focus area unit. The inspection focus area units are stored in a temporary data structure, which is indexed by the unique identifiers of the potential anomaly point cloud region for use in subsequent steps.

[0131] Step S150: Based on the potential abnormal point cloud region information and enhanced point cloud feature set corresponding to the inspection focus area unit, generate an automatic inspection perception result containing regional spatial association description and abnormal feature association expression.

[0132] In this embodiment, when generating automatic inspection and perception results, the potential abnormal point cloud region information corresponding to the inspection focus area unit is first extracted. This potential abnormal point cloud region information includes the number of point cloud units within the region, the spatial association parameters of the associated edges between point cloud units, the abnormal propagation intensity value of each point cloud unit, and the spatial coordinate range of the region. Then, key feature dimensions are extracted from the enhanced point cloud feature set. These feature dimensions are the association dimensions corresponding to the sample feature distribution information with the highest distribution similarity in feature mapping association processing. Next, the regional spatial association information is converted into a natural language description, including a description of the number of point clouds, a description of the association parameter distribution, a description of the propagation intensity, and a description of the spatial range, which are combined to form a regional spatial association description text. Simultaneously, the feature values ​​of the key feature dimensions are associated with the matching abnormality type information to form an abnormal feature association description text. Finally, the regional spatial association description text and the abnormal feature association description text are integrated, transitional statements are added, the description is optimized, and the automatic inspection and perception results are generated.

[0133] Step S151: Extract the potential abnormal point cloud region information corresponding to the inspection focus area unit. The potential abnormal point cloud region information includes the number of point cloud units in the region, the spatial association parameters of the associated edges between point cloud units, the abnormal propagation intensity value of each point cloud unit, and the spatial coordinate range of the region.

[0134] In this embodiment, when extracting potential abnormal point cloud region information, the unique identifier of the potential abnormal point cloud region corresponding to the inspected area of ​​interest unit is first read, and the node set and associated edge set of the region are obtained from the point cloud spatial association network. Then, the number of point cloud units in the region is counted, i.e., the number of elements in the node set; the spatial association parameters of the associated edges between point cloud units are read, i.e., the spatial direction parameter and normal vector association parameter in the associated edge attribute field; the abnormal propagation intensity value of each point cloud unit is read, i.e., the abnormal propagation intensity value in the node attribute field; and the spatial coordinate range of the region is calculated, i.e., the minimum and maximum values ​​of the spatial coordinates of all point cloud units. The above information is integrated to form potential abnormal point cloud region information. The potential abnormal point cloud region information is stored in a temporary data structure for use in subsequent steps.

[0135] Step S152: Convert the number of point cloud units in the region into a quantity description in natural language to form a point cloud quantity description item.

[0136] In this embodiment, when converting the quantity description, the number of point cloud units in the region is first read, and different natural language expressions are used according to the quantity. For example, if the quantity is less than a preset threshold, it is described as "a small number of point cloud units"; if the quantity is between the preset threshold, it is described as "a medium number of point cloud units"; and if the quantity is greater than the preset threshold, it is described as "a large number of point cloud units". The expression result is used as a point cloud quantity description item and stored in a temporary data structure.

[0137] Step S153: Classify the spatial association parameters of the associated edges between point cloud units into spatial direction parameters and normal vector association parameters, statistically analyze the distribution of the two types of parameters, convert them into a natural language description of the association parameter distribution, and form an association parameter description item.

[0138] In this embodiment, when statistically analyzing the distribution of association parameters, the spatial direction parameters and normal vector association parameters of all associated edges are first read, and the average, variance, and maximum value of these two types of parameters are calculated respectively. Then, based on the statistical results, the distribution of association parameters is described using natural language. For example, if the average value of the spatial direction parameters is large, it is described as "good consistency in spatial direction"; if the variance of the normal vector association parameters is small, it is described as "high similarity in normal vectors". The description results are stored as association parameter descriptors in a temporary data structure.

[0139] Step S154: Convert the average and maximum values ​​of the anomaly propagation intensity of each point cloud unit into an intensity description expressed in natural language to form a propagation intensity description item.

[0140] In this embodiment, when converting the intensity description, the average and maximum values ​​of the anomaly propagation intensity for each point cloud unit are first calculated. Then, based on the magnitude of the average and maximum values, the anomaly propagation intensity is described using natural language. For example, if both the average and maximum values ​​are large, it is described as "high anomaly propagation intensity"; if both the average and maximum values ​​are small, it is described as "low anomaly propagation intensity". The description results are stored as propagation intensity description items in a temporary data structure.

[0141] Step S155: Convert the spatial coordinate range of the region into a coordinate range description expressed in natural language to form a spatial range description item.

[0142] Step S1551: Extract the spatial coordinate information of all point cloud units in the potential anomaly point cloud region information, and separate the coordinate components of each point cloud unit on the X, Y, and Z axes of the three-dimensional coordinate system. In this embodiment, firstly, the spatial coordinate information of all point cloud units is read from the potential anomaly point cloud region information. Each spatial coordinate information contains coordinate components of the X, Y, and Z axes. The coordinate components of each point cloud unit are separated, and the coordinate component lists of the X, Y, and Z axes are stored respectively.

[0143] Step S1552: Calculate the minimum and maximum values ​​of the X-axis coordinate components of all point cloud units, recording the minimum value as the minimum X-axis coordinate and the maximum value as the maximum X-axis coordinate; calculate the minimum and maximum values ​​of the Y-axis coordinate components of all point cloud units, recording the minimum value as the minimum Y-axis coordinate and the maximum value as the maximum Y-axis coordinate; calculate the minimum and maximum values ​​of the Z-axis coordinate components of all point cloud units, recording the minimum value as the minimum Z-axis coordinate and the maximum value as the maximum Z-axis coordinate. In this embodiment, for the X-axis coordinate component list, traverse all elements to find the minimum and maximum values, recording them as the minimum and maximum X-axis coordinates, respectively. Calculate the minimum and maximum values ​​of the Y-axis and Z-axis coordinate components using the same method.

[0144] Step S1553: Construct a coordinate range description template, which includes a description framework for the X-axis range, Y-axis range, and Z-axis range. In this embodiment, a coordinate range description template is pre-constructed, for example, "The spatial coordinate range of this region is: X-axis from [Xmin] to [Xmax], Y-axis from [Ymin] to [Ymax], Z-axis from [Zmin] to [Zmax]", where [Xmin], [Xmax], etc., are placeholders.

[0145] Step S1554: Fill the minimum X-axis coordinate, maximum X-axis coordinate, minimum Y-axis coordinate, maximum Y-axis coordinate, minimum Z-axis coordinate, and maximum Z-axis coordinate into the corresponding placeholder positions in the coordinate range description template. In this embodiment, the statistically obtained minimum X-axis coordinate, maximum X-axis coordinate, and other values ​​are filled into the corresponding placeholder positions in the coordinate range description template, replacing the placeholders.

[0146] Step S1555: Adjust the fluency of the coordinate range description template after the coordinates are entered, so that the coordinate values ​​and the description sentences blend naturally. In this embodiment, the description sentences after the coordinates are entered are checked, and the sentence structure is adjusted, for example, by adding appropriate conjunctions, so that the coordinate values ​​and the description sentences blend naturally and improve readability.

[0147] Step S1556: Check whether the adjusted description fully covers the spatial coordinate range of the area, and confirm that no coordinate components are missing or the values ​​are incorrect. In this embodiment, check whether the adjusted description includes the range of the X-axis, Y-axis, and Z-axis, confirm that no coordinate components are missing, and verify the accuracy of the coordinate values.

[0148] Step S1557: The confirmed description is determined as a spatial extent description item, which is used for the subsequent combination of regional spatial association description text. In this embodiment, the checked and adjusted description is determined as a spatial extent description item and stored in a temporary data structure for use in subsequent steps to combine regional spatial association description text.

[0149] Step S156: Combine the point cloud quantity description item, the associated parameter description item, the propagation intensity description item, and the spatial range description item in logical order, add connection statements, and form a regional spatial association description text.

[0150] In this embodiment, when combining descriptive items, the four descriptive items are first arranged in the logical order of point cloud quantity, associated parameter distribution, propagation intensity, and spatial range. Then, connecting statements are added between the descriptive items, such as "This region contains XX point cloud units, the spatial orientation consistency between point cloud units is good, the normal vector similarity is high, the anomaly propagation intensity is high, and the spatial range is XX". The combined text is stored in a temporary data structure as the region spatial association description text.

[0151] Step S157: Extract the key feature dimension from the enhanced point cloud feature set. The key feature dimension is the association dimension corresponding to the sample feature distribution information with the largest distribution similarity in the feature mapping association processing.

[0152] In this embodiment, when extracting key feature dimensions, the feature distribution information of the enhanced point cloud feature set and the feature distribution information of the sample with the highest distribution similarity in the feature mapping association processing are first read. Then, the association dimensions of the two feature distribution information are compared, and the feature dimension with an association degree greater than a threshold is selected as the key feature dimension. The key feature dimension is stored in a temporary data structure, which is indexed by the feature dimension identifier.

[0153] Step S158: Obtain the feature values ​​and corresponding matching anomaly type information for each key feature dimension, associate the feature values ​​with the matching anomaly type information to form an anomaly feature description item corresponding to each key feature dimension.

[0154] In this embodiment, when forming an anomaly feature description, the feature values ​​of each key feature dimension and the corresponding matching anomaly type information are first read. Then, the feature values ​​and the matching anomaly type information are associated and described, for example, "The value of key feature dimension XX is XX, and the corresponding anomaly type is XX". The description result is stored in a temporary data structure as the anomaly feature description item corresponding to each key feature dimension.

[0155] Step S159: Arrange the abnormal feature descriptions corresponding to all key feature dimensions in order of feature importance. The feature importance is set based on the weight of the feature dimension in the distribution similarity calculation. The larger the weight value of the feature importance, the earlier it is arranged, forming an abnormal feature association description text.

[0156] In this embodiment, when arranging the anomaly feature descriptions, the weight of each key feature dimension in the distribution similarity calculation is first read; a higher weight indicates higher feature importance. Then, all anomaly feature descriptions are arranged in descending order of feature importance. Connecting statements are added between the descriptions, such as "The value of key feature dimension XX (highest importance) is XX, corresponding to anomaly type XX; the value of key feature dimension XX (second highest importance) is XX, corresponding to anomaly type XX." The arranged text is stored as the anomaly feature association description text in a temporary data structure.

[0157] Step S1510: Integrate the regional spatial association description text and the abnormal feature association description text, and add transitional statements between the two to make the integrated text logically coherent.

[0158] In this embodiment, when integrating the text, the spatial association description text of the region and the anomalous feature association description text are read first. Then, a transitional statement is added between the two, such as "The spatial association feature of this region is XX, and its anomalous features are mainly manifested as XX". The integrated text is then stored in a temporary data structure.

[0159] Step S1511: Optimize the expression of the integrated text, correct the repetitive or awkward parts, and make the text conform to the expression specifications of the inspection perception results.

[0160] In this embodiment, when optimizing the description, the integrated text is first read and checked for duplicate or grammatically incorrect statements. For example, if duplicate content is found, the duplicates are deleted; if grammatically incorrect statements are found, the sentence structure is adjusted to make them fluent. Then, the text is checked to ensure it conforms to the description specifications of the inspection and perception results, including terminology usage and logical structure. The optimized text is then stored in a temporary data structure.

[0161] Step S1512: Determine the optimized text as an automatic inspection and perception result containing regional spatial association descriptions and abnormal feature association statements.

[0162] In this embodiment, when determining the automatic inspection and sensing result, the optimized text is first verified to see if it contains regional spatial association descriptions and abnormal feature association statements. If it does, it is used as the automatic inspection and sensing result. The automatic inspection and sensing result is stored in a local file or database for user viewing or subsequent processing.

[0163] Furthermore, Figure 2 A schematic diagram of the hardware structure of an automatic inspection and perception system 100 based on three-dimensional point cloud data for implementing the method provided in the embodiments of this application is shown. Figure 2 As shown, the automatic inspection and perception system 100 based on 3D point cloud data may include at least one processor 102 (the processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, a transmission device 106 for communication functions, and a controller 108. Those skilled in the art will understand that... Figure 2 The structure shown is for illustrative purposes only and does not limit the structure of the automatic inspection and sensing system 100 based on 3D point cloud data. For example, the automatic inspection and sensing system 100 based on 3D point cloud data may also include components that are more complex than those shown in the diagram. Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown.

[0164] The memory 104 can be used to store software programs and modules for application software, such as the program instructions corresponding to the method embodiments described above in this application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-described automatic inspection and perception method based on three-dimensional point cloud data. The transmission device 106 is used to acquire or send data via a network.

[0165] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

Claims

1. An automatic inspection and perception method based on three-dimensional point cloud data, characterized in that, The method includes: A spatial association network is constructed on the three-dimensional point cloud data set corresponding to the inspection target to form a point cloud spatial association network containing the spatial association relationship of point cloud units. Each node in the point cloud spatial association network corresponds to a point cloud unit in the three-dimensional point cloud data set, and the association edge between nodes corresponds to the spatial association parameter between point cloud units. The spatial association parameter includes the spatial direction parameter between point cloud units and the normal vector association parameter of adjacent point cloud units. Anomaly propagation simulation processing is performed on the point cloud spatial association network. Based on the preset anomaly propagation rules, potential abnormal point cloud regions with anomaly propagation characteristics are located from the point cloud spatial association network. The potential abnormal point cloud regions are composed of nodes and associated edges corresponding to multiple consecutive point cloud units. The pre-trained inspection feature enhancement model is invoked to perform feature interaction enhancement processing on the point cloud data corresponding to the potential abnormal point cloud region, generating an enhanced point cloud feature set with anomaly feature-specific representation. The enhanced point cloud feature set includes the correlation representation of the spatial structure features and point cloud unit attribute features of the potential abnormal point cloud region. The enhanced point cloud feature set is associated with the inspection target anomaly feature sample library through feature mapping, and inspection interest area units with anomaly feature matching degree are determined from the potential anomaly point cloud region. Based on the potential abnormal point cloud region information and enhanced point cloud feature set corresponding to the inspection focus area unit, an automatic inspection perception result containing regional spatial association description and abnormal feature association expression is generated. The pre-trained inspection feature enhancement model is invoked to perform feature interaction enhancement processing on the point cloud data corresponding to the potential abnormal point cloud region, generating an enhanced point cloud feature set with specific representations of abnormal features, including: The point cloud data corresponding to the potential abnormal point cloud region is divided into spatial structure data and attribute data. The spatial structure data includes the spatial coordinate relationship of the point cloud units in the region, and the attribute data includes the normal vector information and basic attribute information of the point cloud units in the region. The spatial structure data is input into the spatial structure feature extraction layer of the inspection feature enhancement model. The spatial structure feature extraction layer contains three-level feature transformation units. The first-level transformation unit performs coordinate association encoding on the spatial structure data and converts the spatial coordinate association relationship into an initial structural feature vector. The second-level transformation unit performs dimensional expansion processing on the initial structural feature vector. By adding feature dimensions derived from coordinate relationships to the initial structural feature vector, an expanded structural feature vector is generated. The derived feature dimensions include the statistical features of coordinate differences between adjacent point cloud units and the spatial distribution density features. The third-level transformation unit performs feature recombination processing on the extended structure feature vector, and groups and fuses the similar feature dimensions in the extended structure feature vector to generate a spatial structure feature vector that is representative of the spatial structure. The attribute data is input into an attribute feature extraction layer of the inspection feature enhancement model, the attribute feature extraction layer includes an attribute mapping unit and a feature aggregation unit, the attribute mapping unit converts normal vector information and basic attribute information of each point cloud unit into a single point cloud unit attribute feature vector; The feature aggregation unit performs weighted aggregation processing on all single point cloud unit attribute feature vectors, the aggregation weight is set based on an abnormal propagation intensity value of each point cloud unit in the potential abnormal point cloud region, the greater the value corresponding to the abnormal propagation intensity value, the greater the value corresponding to the aggregation weight, and a region attribute feature vector is generated; The spatial structure feature vector and the region attribute feature vector are input into a feature interaction layer of the inspection feature enhancement model, the feature interaction layer includes a bidirectional association mapping unit, and the bidirectional association mapping unit constructs a dimension association table of the spatial structure feature vector and the region attribute feature vector; Based on the dimension association table, a correlation coefficient between dimensions of the two feature vectors is calculated, the correlation coefficient represents a collaborative contribution degree of different feature dimensions to abnormal identification, and the greater the value corresponding to the correlation coefficient, the more significant the representation result of the collaborative contribution degree; According to the correlation coefficient, the spatial structure feature vector and the region attribute feature vector are updated, the feature dimension value corresponding to the greater value of the correlation coefficient is amplified at a higher ratio, and the feature dimension value corresponding to the smaller value of the correlation coefficient is retained at a larger ratio, and an interaction feature vector is generated; The interaction feature vector is subjected to feature smoothing processing, and numerical fluctuations in the interaction feature vector are eliminated by a sliding window average method, and then the smoothed interaction feature vector is subjected to dimension regularization, so that the feature vector dimension meets a preset standard dimension; The dimension-regularized interaction feature vector is determined as a strengthened point cloud feature set with an abnormal feature exclusive representation, each feature dimension in the strengthened point cloud feature set includes associated information of a spatial structure feature and a point cloud unit attribute feature. 2.The automatic inspection and perception method based on three-dimensional point cloud data according to claim 1, characterized in that, The three-dimensional point cloud data set corresponding to the inspection target is subjected to spatial association network construction processing, and a point cloud spatial association network including point cloud unit spatial association relationships is formed, including: Basic attribute information of each point cloud unit in the three-dimensional point cloud data set is extracted, the basic attribute information includes spatial coordinate information and normal vector information of the point cloud unit, the spatial coordinate information represents the position of the point cloud unit in a three-dimensional coordinate system, and the normal vector information represents the spatial orientation of the surface of the point cloud unit; A first point cloud unit in the three-dimensional point cloud data set is selected as an initial target point cloud unit, and a spatial search range is set based on the spatial coordinate information of the initial target point cloud unit, the size of the spatial search range is determined based on structure feature parameters of the inspection target, and is used to screen adjacent point cloud units having a spatial association with the initial target point cloud unit; Other point cloud units in the three-dimensional point cloud data set are traversed in the spatial search range, and all point cloud units in the spatial search range are determined as a set of adjacent point cloud units of the initial target point cloud unit. The spatial direction parameter between the initial target point cloud unit and each adjacent point cloud unit in the adjacent point cloud unit set is calculated, the spatial coordinate difference vector is obtained by subtracting the spatial coordinate information of the adjacent point cloud unit from the spatial coordinate information of the initial target point cloud unit, and the spatial coordinate difference vector is normalized to generate the spatial direction parameter representing the relative direction of the two; The normal vector correlation parameter between the normal vector information of the initial target point cloud unit and the normal vector information of the adjacent point cloud unit is calculated, and the included angle cosine value between the two normal vectors is calculated as the normal vector correlation parameter representing the similarity of the two normal vectors; The initial target point cloud unit is taken as a node of the point cloud space correlation network, and each adjacent point cloud unit in the adjacent point cloud unit set is taken as a correlation node corresponding to the node, and a correlation edge is constructed between the node and the correlation node corresponding to the initial target point cloud unit; The calculated spatial direction parameter and normal vector correlation parameter are taken as the spatial correlation parameter corresponding to the correlation edge and stored in the attribute field of the correlation edge; The next unprocessed point cloud unit in the three-dimensional point cloud data set is selected as a new target point cloud unit, and the above operations of setting a spatial search range, determining an adjacent point cloud unit set, calculating a spatial correlation parameter, constructing a correlation edge, and storing an attribute field are repeatedly performed; After all point cloud units in the three-dimensional point cloud data set complete node creation and correlation edge construction, all nodes, correlation edges, and correlation edge attribute fields are integrated to form a point cloud space correlation network containing point cloud unit spatial correlation relationships. 3.The automatic inspection and perception method based on three-dimensional point cloud data according to claim 1, characterized in that, The point cloud space correlation network is subjected to abnormal propagation simulation processing, and based on a preset abnormal propagation rule, a potential abnormal point cloud region with an abnormal propagation feature is located from the point cloud space correlation network, including: Obtaining the point cloud unit spatial correlation change rule of the inspection target in the abnormal state, setting the abnormal propagation rule based on the point cloud unit spatial correlation change rule, the abnormal propagation rule including a correlation edge propagation weight calculation method, a node abnormal state trigger condition, and a propagation path termination condition; A node is randomly selected from the point cloud space correlation network as an initial propagation node, and an initial abnormal propagation intensity value is assigned to the initial propagation node, the initial abnormal propagation intensity value being set based on the feature change amplitude of the point cloud unit in the abnormal state; According to the correlation edge propagation weight calculation method in the abnormal propagation rule, the propagation weights of all correlation edges of the initial propagation node are calculated, the propagation weight being related to the spatial correlation parameter corresponding to the correlation edge, the greater the value corresponding to the direction consistency evaluation result of the spatial direction parameter, the greater the value corresponding to the propagation weight, and the greater the value corresponding to the normal vector correlation parameter, the greater the value corresponding to the propagation weight; Based on the propagation weight and the initial abnormal propagation intensity value, the abnormal propagation intensity value of each correlation node of the initial propagation node is calculated, and the abnormal propagation intensity value of the correlation node is the initial abnormal propagation intensity value multiplied by the propagation weight of the corresponding correlation edge. The abnormal propagation strength value of each associated node is compared with the node abnormal state triggering condition in the abnormal propagation rule. If the abnormal propagation strength value of the associated node meets the node abnormal state triggering condition, the associated node is marked as an abnormal state node, and the abnormal propagation path of the node is recorded; The associated node marked as an abnormal state is taken as a new propagation node, and the above operations of calculating the associated edge propagation weight, calculating the abnormal propagation strength value of the new associated node, comparing the triggering condition, and marking the abnormal state node are repeatedly performed; The current state of the propagation process is compared with the propagation path termination condition in the abnormal propagation rule. If the propagation process meets the propagation path termination condition, the propagation process of the propagation path is stopped. The propagation path termination condition includes that the abnormal propagation strength value is lower than a preset propagation threshold and the number of propagation nodes reaches a preset upper limit; The next node in the point cloud space association network that is not selected as an initial propagation node is selected as a new initial propagation node, and the above operations of assigning an initial abnormal propagation strength value, calculating a propagation weight, propagating an abnormal strength, and terminating propagation are repeatedly performed; All sets of nodes marked as abnormal state nodes are collected, and each continuously connected abnormal state node cluster and the corresponding associated edge are determined as a potential abnormal point cloud region with an abnormal propagation feature; All determined potential abnormal point cloud regions are processed to remove repeated or containing regions, and a final potential abnormal point cloud region set is formed. 4.The automatic inspection and perception method based on three-dimensional point cloud data according to claim 1, characterized in that, The feature aggregation unit performs weighted aggregation processing on all single point cloud unit attribute feature vectors. The aggregation weight is set based on the abnormal propagation strength value of each point cloud unit in the potential abnormal point cloud region. The greater the value corresponding to the abnormal propagation strength value, the greater the value corresponding to the aggregation weight. A region attribute feature vector is generated, including: The abnormal propagation strength value corresponding to each point cloud unit in the potential abnormal point cloud region is extracted to form an abnormal propagation strength value list. Each element in the abnormal propagation strength value list corresponds to a single point cloud unit attribute feature vector. The sum of all abnormal propagation strength values in the abnormal propagation strength value list is calculated. The abnormal propagation strength value of each point cloud unit is divided by the sum to obtain a normalized weight value corresponding to each point cloud unit. The sum of all normalized weight values is a preset value. The first single point cloud unit attribute feature vector is selected. Each feature dimension value in the single point cloud unit attribute feature vector is multiplied by the corresponding normalized weight value to obtain a weighted first attribute feature vector. The second single point cloud unit attribute feature vector is selected. The above operation of multiplying the corresponding normalized weight value is repeated to obtain a weighted second attribute feature vector. The above weighting processing is sequentially performed on all single point cloud unit attribute feature vectors in the potential abnormal point cloud region to obtain multiple weighted attribute feature vectors. The values of all weighted attribute feature vectors are accumulated according to the feature dimensions. The accumulation value of each feature dimension is the sum of the values of all weighted attribute feature vectors in the feature dimension. The accumulated value of each feature dimension is directly taken as the value of the corresponding feature dimension of the regional attribute feature vector, and the values of all feature dimensions are arranged in the original feature dimension order to form a regional attribute feature vector with regional attribute representation; The generated regional attribute feature vector is subjected to feature validity check to confirm that the average value of each feature dimension is within the preset reasonable range of attribute features, and if all dimensions are within the reasonable range, the regional attribute feature vector is retained; if there is a dimension exceeding the reasonable range, the weighted accumulated value corresponding to the feature dimension is recalculated, and after correction, it is checked again until all dimensions meet the requirements. 5.The automatic inspection and perception method based on three-dimensional point cloud data according to claim 1, wherein, The feature mapping association processing of the enhanced point cloud feature set and the inspection target abnormal feature sample library includes: Collecting multiple sets of three-dimensional point cloud abnormal sample data of the inspection target under different abnormal types, each set of three-dimensional point cloud abnormal sample data being labeled with a corresponding abnormal type label and an abnormal region range; Performing spatial correlation network construction processing on each set of three-dimensional point cloud abnormal sample data to obtain a sample spatial correlation network corresponding to each set of sample; Performing abnormal propagation simulation processing on each sample spatial correlation network to locate a sample potential abnormal point cloud region corresponding to each set of sample; Calling an inspection feature enhancement model to perform feature interaction enhancement processing on the point cloud data corresponding to each sample potential abnormal point cloud region to generate a sample enhanced point cloud feature set corresponding to each set of sample; Storing the sample enhanced point cloud feature set, the abnormal type label and the abnormal region range corresponding to each set of sample in association, classifying the sample data according to the abnormal type label, and constructing an inspection target abnormal feature sample library; Extracting feature distribution information of the enhanced point cloud feature set, the feature distribution information including a value range, a value occurrence frequency and a correlation between feature dimensions of each feature dimension; Selecting all sample enhanced point cloud feature sets corresponding to one abnormal type from the inspection target abnormal feature sample library, and extracting sample feature distribution information of each sample enhanced point cloud feature set; Calculating the distribution similarity between the feature distribution information of the enhanced point cloud feature set and each sample feature distribution information by calculating the value range overlap, the value occurrence frequency similarity and the feature dimension correlation deviation of the corresponding feature dimensions in the two kinds of feature distribution information, and comprehensively generating the distribution similarity; Performing average calculation on the distribution similarity of all sample enhanced point cloud feature sets under the same abnormal type to obtain an average distribution similarity corresponding to the abnormal type; Performing the above operations of extracting sample feature distribution information, calculating distribution similarity and average calculation on all abnormal types in the inspection target abnormal feature sample library in turn to obtain an average distribution similarity corresponding to each abnormal type; Selecting the abnormal type with the maximum value corresponding to the average distribution similarity as a matching abnormal type, and extracting all sample abnormal region ranges corresponding to the matching abnormal type; calculating a region overlap degree of the potential abnormal point cloud region and each sample abnormal region range, the region overlap degree being a number of point cloud cells common to both divided by a total number of point cloud cells of the potential abnormal point cloud region; selecting a sample abnormal region range corresponding to a maximum value of the region overlap degrees as a reference abnormal region range, and determining a range of point cloud cells common to the reference abnormal region range in the potential abnormal point cloud as a patrol attention region cell having the abnormal feature matching degree.

6. The method of claim 5, wherein, the distribution similarity between the feature distribution information of the reinforced point cloud feature set and each sample feature distribution information is calculated, the distribution similarity being generated by calculating a value range overlap degree, a value frequency similarity and a feature dimension correlation deviation of corresponding feature dimensions in the two kinds of feature distribution information, and comprising: extracting a value range of a first feature dimension in the feature distribution information of the reinforced point cloud feature set, denoted as a target value range, the target value range containing a minimum value and a maximum value of the feature dimension; extracting a sample value range of the corresponding first feature dimension in the sample feature distribution information, the sample value range containing a sample minimum value and a sample maximum value of the feature dimension; calculating an overlap interval length of the target value range and the sample value range, the overlap interval length being a maximum value of the overlap part minus a minimum value of the overlap part, and being zero if there is no overlap; dividing the overlap interval length by a combined interval length of the target value range and the sample value range to obtain the value range overlap degree of the feature dimension; extracting a value frequency distribution of the first feature dimension in the feature distribution information of the reinforced point cloud feature set, denoted as a target frequency distribution, the target frequency distribution containing a plurality of value intervals and a proportion of occurrence times corresponding to each interval; extracting a sample value frequency distribution of the corresponding first feature dimension in the sample feature distribution information, the sample value frequency distribution containing a plurality of sample value intervals and a proportion of sample occurrence times corresponding to each interval; aligning the intervals of the target frequency distribution and the sample value frequency distribution, calculating an absolute difference value of the proportion of occurrence times in each aligned interval, and taking an average value of all absolute difference values as the value frequency deviation of the feature dimension; subtracting the value frequency deviation from a preset reference value to obtain the value frequency similarity of the feature dimension; calculating a target correlation coefficient of the first feature dimension and all other feature dimensions in the feature distribution information of the reinforced point cloud feature set to form a target correlation matrix; calculating a sample correlation coefficient of the corresponding first feature dimension and all other feature dimensions in the sample feature distribution information to form a sample correlation matrix; calculating an absolute difference value of corresponding elements in the target correlation matrix and the sample correlation matrix, and taking an average value of all absolute difference values as the feature dimension correlation deviation of the feature dimension; the above operations of calculating the value range overlap degree, the value frequency similarity and the feature dimension correlation deviation are sequentially performed on all feature dimensions in the feature distribution information of the reinforced point cloud feature set. The weighted average of the numerical range overlap degree, the numerical frequency similarity, and the feature dimension correlation deviation is obtained by weighting and averaging the numerical range overlap degree, the numerical frequency similarity, and the feature dimension correlation deviation of all feature dimensions, and the weight is based on the influence degree of each index on the abnormal matching; The weighted average of the numerical range overlap degree, the numerical frequency similarity, and the feature dimension correlation deviation is obtained by weighting and averaging the numerical range overlap degree, the numerical frequency similarity, and the feature dimension correlation deviation of all feature dimensions, and the weight is based on the influence degree of each index on the abnormal matching; 7. The method of claim 1, wherein, The automatic inspection perception result containing the region space correlation description and the abnormal feature correlation expression is generated based on the potential abnormal point cloud region information and the reinforced point cloud feature set corresponding to the inspection attention region unit, and includes: The potential abnormal point cloud region information corresponding to the inspection attention region unit is extracted, and the potential abnormal point cloud region information includes the number of point cloud units in the region, the spatial correlation parameters of the associated edges between the point cloud units, the abnormal propagation intensity value of each point cloud unit, and the spatial coordinate range of the region; The number of point cloud units in the region is converted into a quantity description in natural language to form a point cloud quantity description item; The spatial correlation parameters of the associated edges between the point cloud units are classified according to the spatial direction parameters and the normal vector correlation parameters, and the distribution of the two types of parameters is respectively counted and converted into an associated parameter distribution description in natural language to form an associated parameter description item; The average value and the maximum value of the abnormal propagation intensity value of each point cloud unit are converted into an intensity description in natural language to form a propagation intensity description item; The spatial coordinate range of the region is converted into a coordinate range description in natural language to form a spatial range description item; The point cloud quantity description item, the associated parameter description item, the propagation intensity description item, and the spatial range description item are combined in a logical order, a connection sentence is added, and a region space correlation description text is formed; The key feature dimensions in the reinforced point cloud feature set are extracted, and the key feature dimensions are the associated dimensions corresponding to the sample feature distribution information with the maximum value of the distribution similarity in the feature mapping association processing; The feature values and the corresponding matching abnormal type information of each key feature dimension are obtained, the feature values are associated with the matching abnormal type information, and an abnormal feature description item corresponding to each key feature dimension is formed; The abnormal feature description items corresponding to all key feature dimensions are arranged in order of feature importance, the feature importance is set based on the weight of the feature dimension in the distribution similarity calculation, the greater the weight value corresponding to the feature importance, the earlier the arrangement position, and an abnormal feature correlation expression text is formed; The region space correlation description text and the abnormal feature correlation expression text are integrated, a transition sentence is added between the two, and the integrated text is logically coherent; The integrated text is optimized in expression, and the parts with repeated expressions or incoherent sentences are corrected to make the text comply with the expression specification of the inspection perception result; The optimized text is determined as the automatic inspection perception result containing the region space correlation description and the abnormal feature correlation expression. 8.The automatic inspection and perception method based on three-dimensional point cloud data according to claim 7, characterized in that, The spatial coordinate range of the region is converted into a coordinate range description in natural language to form a spatial range description item, which includes: Extracting the spatial coordinate information of all point cloud units in the potential abnormal point cloud region information, separating the coordinate component of each point cloud unit on the X-axis, Y-axis and Z-axis of the three-dimensional coordinate system; Statistically determining the minimum and maximum values of the X-axis coordinate components of all point cloud units, taking the minimum value as the minimum X-axis coordinate and the maximum value as the maximum X-axis coordinate; Statistically determining the minimum and maximum values of the Y-axis coordinate components of all point cloud units, taking the minimum value as the minimum Y-axis coordinate and the maximum value as the maximum Y-axis coordinate; Statistically determining the minimum and maximum values of the Z-axis coordinate components of all point cloud units, taking the minimum value as the minimum Z-axis coordinate and the maximum value as the maximum Z-axis coordinate; Constructing a coordinate range expression template, which contains an expression framework of the X-axis range, Y-axis range and Z-axis range; Filling the minimum X-axis coordinate, maximum X-axis coordinate, minimum Y-axis coordinate, maximum Y-axis coordinate, minimum Z-axis coordinate and maximum Z-axis coordinate into the corresponding placeholder positions in the coordinate range expression template; Adjusting the sentence fluency of the coordinate range expression template after filling the coordinates, so that the coordinate values and the expression sentences are naturally integrated; Checking whether the adjusted expression completely covers the spatial coordinate range of the region, confirming that there is no coordinate component omission or value error; Determining the confirmed expression as a spatial range description item, which is used for the combination of subsequent regional spatial correlation description texts.

9. An automatic inspection and perception system based on three-dimensional point cloud data, characterized in that, The application relates to a processor and a readable storage medium, wherein the readable storage medium stores a program which is executed by the processor to realize the automatic inspection perception method based on three-dimensional point cloud data according to any one of claims 1-8.

Images