Point cloud clustering and image texture association fusion-based complex scene three-dimensional target identification and positioning method

Abstract

The invention discloses a point cloud clustering and image texture association fusion complex scene three-dimensional target identification and positioning method. The method comprises the following steps: acquiring an image texture data set and a three-dimensional point cloud data set through a three-dimensional visual sensor; preprocessing the collected image texture data, and adopting a deep learning training-reasoning method to carry out labeling training and reasoning on an image texture data set to obtain image position data of the region of interest; searching and segmenting 3D point cloud data of the component by taking the position of the component in the image as an index; filtering and denoising processing is carried out on the obtained 3D point cloud data; and judging whether a relatively large point cloud cluster exists around according to the current component point cloud, and outputting a spatial position where the point cloud cluster exists. The method mainly aims at a complex scene, adopts the technical means of point cloud space information clustering, image texture detail extraction and correlation fusion of the two, and realizes three-dimensional target identification and positioning with relatively high precision and rapid calculation.

Description

Technical Field

[0001] The present invention relates to the technical field, and in particular to a complex scene three-dimensional target recognition and positioning method by integrating point cloud clustering with image texture association. Background Art

[0002] In recent years, with the continuous rapid development and in-depth exploration of the field of computer vision, a series of innovative and efficient target detection methods have emerged, which greatly enriched the research content of this field and promoted its extensive expansion in practical applications. These methods have shown significant diversity and flexibility in terms of input data modality, mainly covering three categories: the first is target detection based on single-modal images, which makes full use of the rich information in two-dimensional images and realizes accurate recognition and positioning of target objects through complex algorithm models; the second is single-modal point cloud target detection, which focuses on three-dimensional point cloud data and uses the precise coordinate information of point clouds in three-dimensional space to realize three-dimensional reconstruction and accurate detection of target objects; the third is multi-modal target detection that integrates image and point cloud information. This method cleverly integrates two-dimensional images with three-dimensional point cloud data, making full use of the complementarity of the two in feature representation, thereby further improving the accuracy and robustness of target detection.

[0003] Single-modal image object detection methods are mainly based on 2D or 3D images for object detection and positioning. The visual features of the image are used to detect the target object. Single-modal image object detection can be further divided into two categories. One is represented by the Fast R-CNN, which uses a candidate region extractor to generate potential target regions and then classifies the objects in the candidate box region; the other is represented by YOLO (you only look once), which completes the positioning and classification tasks in one step.

[0004] Single-modal point cloud object detection can be divided into three categories: (1) directly inputting the raw point cloud data into the network for end-to-end recognition, such as PointNet directly performing feature extraction and aggregation on point cloud data; (2) converting the point cloud into an image and inputting it into the network, such as PointPillars applying traditional image processing technology and convolutional neural networks on a bird's-eye view; (3) converting the point cloud into voxel format and inputting it into the network, such as Voxelnet dividing the point cloud data into a regular three-dimensional voxel grid and applying a convolutional neural network on the voxel grid.

[0005] However, when faced with complex environmental perception tasks, a single sensor often cannot meet all requirements. Although binocular cameras can provide rich color and texture information, the depth information is not accurate enough; LiDAR can provide accurate distance measurement and motion detection, has good anti-interference and the ability to adapt to complex environments, but is relatively weak in obtaining details such as color and texture. The multimodal fusion method fuses the data of different sensors such as cameras and LiDARs, fully utilizing the advantages of each sensor and making up for each other's shortcomings, thereby providing more comprehensive and accurate environmental perception information.

[0006] Multimodal fusion methods can be divided into three categories: (1) data-level fusion, which directly fuses the raw data of multiple sensors, such as using vector fusion laser scanner and two short-range radar data; (2) feature-level fusion, which fuses the feature representations of image and point cloud. Feature alignment and fusion can be achieved by sharing weights, such as MV3D and AVOD; (3) decision-level fusion, which fuses the prediction results of image and point cloud, such as using the Hungarian algorithm to fuse the detection results of YOLOv3 and PointRCNN.

[0007] Multimodal fusion methods fuse data from different sensors such as cameras and lidars, making full use of the advantages of each sensor and making up for each other's shortcomings, thereby providing more comprehensive and accurate environmental perception information. However, images and point clouds have different data structures and feature representations, and how to effectively fuse them is a key issue in modal fusion methods. Summary of the invention

[0008] Purpose of the invention: To solve the above problems, the present invention proposes a method for three-dimensional target recognition and positioning in complex scenes by fusing point cloud clustering with image texture association.

[0009] Technical solution: The present invention provides a complex scene three-dimensional target recognition and positioning method by point cloud clustering and image texture association fusion, comprising the following steps: step S1, collecting image data and point cloud data through a sensor to form an image data set and a three-dimensional point cloud data set; Step S2, processing the collected image data, using a deep learning training-inference method to perform annotation training and inference on the 2D image data set to obtain image position data of the area of ​​interest; Step S3, using the position of the component in the image as an index, searching for and segmenting the 3D point cloud data of the component; Step S4, filtering and denoising the obtained 3D point cloud data to eliminate noise and clutter therein; Step S5, starting from an arbitrarily selected random point, and performing corresponding processing according to different component features; Step S6, using the current component point cloud to determine whether there are any point cloud cluster components around, and outputting the spatial position of the point cloud cluster.

[0010] Preferably, step S2 specifically includes first annotating the region of interest by annotating software, converting the annotated image file into a file in a data set format required for model training, and then using the file as input data to train it by a deep learning training-inference method to obtain a model that matches the data; inferring the region of interest of the two-dimensional image data based on the model, and binarizing the region of interest to obtain a binary image; The binary image is expanded and eroded. The specific representation of erosion is as follows: ; ; Where C represents the erosion operation, (x, y) is the pixel position in image I, and (i, j) is the relative position in the structural element S; The specific expression of expansion is as follows: ; ; Where E represents the dilation operation, (x, y) is the pixel position in image I, and (i, j) is the relative position in the structural element S; Traverse the processed binary image from top to bottom to find the pixel points with a value of 255, and arrange the component areas composed of the points that meet the requirements one by one according to the eight-neighborhood algorithm to obtain the specific number of components in the image and the corresponding component numbers, and obtain the position point coordinate set of each component and the corresponding position point index; remove the position point coordinate set that is less than a certain threshold, so as to obtain the total number of components in the image and the index set of the two-dimensional coordinates of the pixel points corresponding to each component.

[0011] Preferably, in step S3, the index set of the two-dimensional coordinates of the component in the image obtained in step S2 is used as an index to obtain a 3D point cloud data set that is searched and segmented; there are two optional methods, one is to realize the conversion between the image and the 3D point cloud by converting the points in the pixel coordinate system to the world coordinate system; the other is to filter the corresponding point cloud space through the position point index pointID to obtain the point cloud data of the component.

[0012] Preferably, the method of converting the points in the pixel coordinate system to the world coordinate system to realize the conversion between the image and the 3D point cloud specifically includes: Step S3.1: converting the acquired two-dimensional coordinate set Convert from pixel coordinate system to image coordinate system, where (ui,vi) represents the pixel at row v and column u in the image corresponding to the i-th point in the position point coordinate set pointSet; Step S3.2: Convert from image coordinate system to camera coordinate system; Step S3.3: Convert from camera coordinate system to world coordinate system; Through steps S3.1 to S3.3, the conversion between the pixel coordinate system and the world coordinate system is completed by the following formula: ; in, is the point coordinate in the pixel coordinate system, Indicates that the camera optical center is perpendicular to the point in the image coordinate system, and is the focal length of the camera, representing the horizontal and vertical focal lengths, and They are the rotation matrix and translation matrix from the camera coordinate system to the world coordinate system, It is the coordinate of 3D point cloud data in the world coordinate system.

[0013] Preferably, the filtering operation on the corresponding point cloud space through the position point index pointID to obtain the point cloud data of the component specifically includes: step S3.1: placing the three-dimensional point cloud acquisition device and the two-dimensional image acquisition device in the same space and at the same time to perform data acquisition, and collecting image data and point cloud data of the area containing the component in the same space and at the same time; step S3.2: performing step S2 processing on the image data to obtain the position point index pointID of the component; step S3.3: using the ExtractIndices filter to extract the point cloud subset pointcloudSet according to the original point cloud collected in step S3.1 and the point index pointID obtained in step S3.2, the specific steps include: 1) using the voxel grid downsampling method for downsampling; 2) using the SAC plane parameter model to extract the point cloud data subset that conforms to the geometric model; 3) using the negative variable to extract the remaining point cloud of the opposite point cloud set; 4) using the remaining point cloud as the point cloud to be processed, returning to step 2, and entering the next cycle.

[0014] Preferably, in step S4, the 3D point cloud data set of the component is obtained through step S3, and the obtained 3D point cloud data is subjected to statistical outlier filtering, a statistical analysis is performed on the field of each point obtained, and points that do not meet certain standards are trimmed, thereby achieving the effect of removing outliers in the point cloud data and obtaining a processed 3D point cloud data set; the statistical outlier filtering of the obtained 3D point cloud data specifically includes: 1) traversing the point cloud and finding its k nearest neighbors for each point in the point cloud; 2) calculating the average distance from the point to the k nearest neighbors ; 3) Calculate the average distance The average and standard deviation ;4) Set distance threshold , whose expression is , where is the scaling factor for the distance threshold, is the standard deviation multiplier, which is a non-zero natural number; 5) Compare the set distance threshold and , points exceeding the threshold are marked as outliers and removed, while points below the threshold are marked as inliers.

[0015] Preferably, in step S5, based on the 3D point cloud data set filtered in step S4, an arbitrarily selected random point is used as the starting point, and corresponding processing or no processing is performed according to the characteristics of different components; for those components showing obvious spherical features, starting from the starting point, a sphere fitting algorithm is used to accurately depict the outline of the sphere, and its center and radius are determined accordingly; through this information, the sphere is segmented, cropped or otherwise processed; if the point cloud data contains components with cylindrical features, starting from the starting point, a cylinder fitting algorithm is used to accurately capture the axis and radius of the cylinder, and then determine the positions of its two ends; through this information, the cylinder is cut, adjusted or otherwise processed; for components with more complex shapes and more diverse features, machine learning algorithms are used for feature classification and recognition, as well as refined processing combined with shape analysis and geometric transformation methods.

[0016] Preferably, in step S6, the two points with the longest distance on the horizontal section of the current component point cloud are calculated, and the distance is recorded as ,by As the diameter, a dynamic sphere is synthesized, where δ is the range to be searched near the current part point cloud. If it is greater than 0, the center point cloud coordinates of the same plane of the component are used as the center of the sphere, and the center position of the dynamic sphere is constantly changed. External There are point cloud clusters in the memory, and the spatial locations of the point cloud clusters are output.

[0017] Preferably, the specific steps of step S6 are as follows: 1) Calculate the two points with the farthest distance on the horizontal section of the current component point cloud; the component point cloud is P, which contains n points, that is, ,in ; Find the two points farthest apart on the transverse section and , so that ; 2) Determine the diameter and search range of the dynamic sphere; ; ; by As the diameter, a dynamic sphere is synthesized with radius R, where It is the range to be searched near the current component point cloud; 3) Determine the center of the dynamic sphere; use the center point cloud coordinates of the same plane as the component as the center of the sphere : ; in is the mean or a fixed value of the z coordinate; 4) Search for the point cloud cluster around the element; if found External There are point cloud clusters in the memory, and the spatial positions of the point cloud clusters are output: for a certain sphere center position and radius , if there exists a point cloud p that satisfies: ; Then output the spatial location of the point cloud cluster.

[0018] Compared with the prior art, the present invention has the following beneficial effects: the present invention mainly aims at complex scenes, and adopts the technical means of point cloud spatial information clustering, image texture detail extraction and the correlation and fusion of the two to achieve high-precision and fast calculation of three-dimensional target recognition and positioning; the present invention filters the 3D point cloud data by a statistical outlier filtering method, and trims points that do not meet certain standards, thereby achieving the effect of removing outliers in the point cloud data; the present invention performs different processing according to the characteristics of different components, and can more accurately capture the complex shape and internal structure of the components, providing a more refined and reliable data basis for subsequent applications; the present invention is based on technologies in the fields of computer vision, pattern recognition, machine learning, etc., combined with sensors to take the original data and image data of the point cloud, use image texture details to achieve rapid extraction of targets of interest, filter out unnecessary data information, reduce the amount of point cloud data, and improve calculation speed. This method can achieve accurate positioning, morphological analysis and fault diagnosis of components in complex scenes, thereby improving the reliability and safety of the components. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a complex scene three-dimensional target recognition and positioning method by point cloud clustering and image texture association fusion of the present invention; Figure 2 It is a flow chart of step S2 of a method for identifying and locating three-dimensional objects in complex scenes by integrating point cloud clustering with image texture association of the present invention; Figure 3This is an optional solution flow chart of step S3 of a complex scene three-dimensional target recognition and positioning method by point cloud clustering and image texture association fusion of the present invention; Figure 4 It is another optional solution flow chart of step S3 of a complex scene three-dimensional target recognition and positioning method by point cloud clustering and image texture association fusion of the present invention; Figure 5 It is a flow chart of filtering 3D point cloud data using a statistical outlier filtering method of the present invention; Figure 6 It is a flowchart of step S5 of a complex scene three-dimensional target recognition and positioning method by point cloud clustering and image texture association fusion of the present invention; Figure 7 : is a flowchart of the implementation of the practical application of the present invention, wherein (a) is the collected image data; (b) is the binary image after binarization processing; (c) is the binary image after closing operation; (d) is the contour extraction map; (e) is the original 3D point cloud data; (f) is the fusion result map; (g) is the result map after 3D point cloud filtering; (h) is the point cloud group search situation map; Figure 8 It is a data result diagram of the practical application of the present invention; Fig. 9 It is a principle block diagram of a complex scene three-dimensional target recognition and positioning method of the present invention that combines point cloud clustering with image texture association. DETAILED DESCRIPTION

[0020] Many modifications and changes made by those skilled in the art based on the spirit of the present invention belong to the protection scope of the present invention.

[0021] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when an element or component is said to be "connected" to another element or component, it may be directly connected to the other element or component, or there may be an intermediate element or component. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, an embodiment of the present invention provides a complex scene three-dimensional target recognition and positioning method by associating and fusing point cloud clustering with image texture, including: Step S1, collecting image data and point cloud data through sensors to form an image data set and a three-dimensional point cloud data set; Step S2, processing the collected image data, using a deep learning training-inference method to perform annotation training and inference on the 2D image data set to obtain image position data of the area of ​​interest; Step S3, using the position of the component in the image as an index, searching for and segmenting the 3D point cloud data of the component; Step S4, filtering and denoising the obtained 3D point cloud data to eliminate noise and clutter therein; Step S5, starting from an arbitrarily selected random point, and performing corresponding processing according to different component features; Step S6, using the current component point cloud to determine whether there is a larger point cloud cluster (component) around it, and outputting the spatial location of the point cloud cluster.

[0024] like Fig. 9 The figure shows a principle block diagram of a complex scene three-dimensional target recognition and positioning method of the present invention by integrating point cloud clustering with image texture association.

[0025] Further, in step S1, its implementation environment includes a three-dimensional point cloud acquisition device, a two-dimensional image acquisition device, and a computer device, wherein the three-dimensional point cloud acquisition device can be a laser radar scanning device, or a binocular camera and other devices, and the two-dimensional image acquisition device can be an optical camera device. It is necessary to ensure that the three-dimensional point cloud acquisition device and the two-dimensional image acquisition device are horizontally placed on the same bracket, and the three-dimensional point cloud acquisition device is installed directly above the two-dimensional image acquisition device. The three-dimensional point cloud acquisition device and the two-dimensional image acquisition device must be kept horizontal. The three-dimensional point cloud acquisition device and the two-dimensional image acquisition device can also be any device that can realize three-dimensional information scanning or two-dimensional information acquisition integrated in the same device, and the present invention does not impose any restrictions on this. Relevant personnel can obtain three-dimensional point cloud data and two-dimensional image information of the target scene including the target object through the three-dimensional point cloud acquisition device and the two-dimensional image acquisition device. According to actual needs, the collected two-dimensional image information is cleaned to remove blurred, repeated or non-compliant images, and the image is pre-processed, such as cropping, scaling, rotation, denoising, etc., to improve image quality. The collected 3D point cloud data is also preprocessed, such as denoising, registration, filtering, segmentation and other operations, to remove redundant information and improve the quality of the data. The preprocessed 3D point cloud data and 2D image information are stored in the specified format and order to form an image data set and a 3D point cloud data set.

[0026] Further, in step S2, if Figure 2 As shown, the steps include: Step S2.1: The region of interest is first annotated using annotation software, and the annotated image file is converted into a file in a specific data set format required for model training, and then the file is used as input data to train it through a deep learning training-inference method to obtain a model that matches the data. The region of interest of the two-dimensional image data is inferred based on the model, and the region of interest is binarized, the grayscale value of the background area is 0, displayed in black, and the grayscale value of the region of interest is 255, displayed in white, and segmented from the two-dimensional image data to obtain a binary image.

[0027] Specifically, in step S2.1, the deep learning training-inference method is based on pytorch to implement the Deeplabv3+ inference algorithm. The main body of its Encoder is a DCNN with dilated convolution. A commonly used classification network can be used, followed by a spatial pyramid pooling module with dilated convolution, mainly to introduce multi-scale information; compared with DeepLabv3, Deeplabv3+ introduces a Decoder module, which further integrates the underlying features with the high-level features to improve the accuracy of the segmentation boundary. DeepLabv3+ introduces the idea of ​​EncoderDecoder based on DilatedFCN. For DilatedFCN, the main thing is to modify the back block of the classification network and replace the downsampling layer with stride=2 with dilated convolution. The image dataset of the component uses a specific training format to segment the dataset into a training set and a test set, which is loaded with the trained model, and the vscode programming environment and python language programming reasoning are used to obtain the binary image after binarization and the pixel position of the component through reasoning.

[0028] Step S2.2: Dilate and erode the binary image. Dilate to fill defects in the region of interest in the binary image, and erode to remove burrs around the region of interest. Combine the two, erode first and then dilate, remove burrs in the region of interest, and retain the original image size; dilate first and then erode, bridge the narrow discontinuities and elongated gullies in the region of interest, eliminate small holes, fill breaks, and retain the original image size. The specific representation of erosion is as follows: ; ; Where C represents the erosion operation, (x, y) is the pixel position in the image, and (i, j) is the relative position in the structural element S. For each pixel position (x, y) in image I, the erosion operation finds the minimum value of all pixels covered by the structural element S and assigns this minimum value to the pixel at the corresponding position (x, y) in the result image.

[0029] The specific expression of expansion is as follows: ; ; Similarly, E represents the dilation operation, (x, y) is the pixel position in the image, and (i, j) is the relative position in the structure element S. For each pixel position (x, y) in image I, the dilation operation finds the maximum value among all pixels covered by the structure element S and assigns this maximum value to the pixel at the corresponding position (x, y) in the result image.

[0030] Step S2.3: Traverse the processed binary image from top to bottom, find the pixel points with a value of 255, and arrange the component areas formed by the points that meet the requirements one by one according to the eight-neighborhood algorithm to obtain the specific number of components in the image and the corresponding component numbers. According to the following formula ; ; Get the position point index pointID of each component, and get the position point coordinate set pointSet of each component. pointSet contains the X coordinate pointX and Y coordinate pointY of each point, where v represents each row of the image, u represents each column of the image, and width represents the width of the image.

[0031] Specifically, in step S2.3, the eight-neighborhood algorithm takes the first pixel with a grayscale value of 255 as the center point, traverses the eight neighboring pixels clockwise, and checks the grayscale value difference between the current pixel and the pixels in its eight neighborhoods. If there is a difference in grayscale value, it is considered that the pixel may be on the edge. Repeat the above process, and connect the detected edge points using the edge tracking algorithm to form a continuous edge contour. The edge tracking algorithm extends the edge contour by checking whether there are other edge points in the eight neighborhoods of the current edge point. Fill the interior of the contour area according to the obtained contour. The specific method is to uniformly fill the internal area surrounded by the contour with a grayscale value of 255 (i.e., white), so as to clearly distinguish and identify each component area in the image.

[0032] Step S2.4: Process the pointSet formed each time, that is, process a component area in the image each time. When the number of elements in the pointSet (that is, the number of pixels pointNum) is less than a certain number of pixels T, the pointSet is regarded as non-component data and removed. The specific expression is as follows: ; Step S2.5: By setting boundary conditions, the area formed by pointSet is used as the first component area on the image, the component number lineID = 1 is recorded, and pointSet, pointID and lineID are combined to form the first element of the component two-dimensional coordinate set inspectUnit; Specifically, in step S2.5, the boundary conditions are expressed as follows: ; in, Represents the eight pixels adjacent to the central pixel. Represents the gray value of the i-th pixel. The above formula indicates that when the sum of the gray values ​​of the eight pixels in the eight neighborhoods is 0, the search for the first component area is completed.

[0033] Repeat steps S2.3 to S2.5, arrange the component areas constituted by pointSets in different areas of the image in sequence, attach a corresponding lineID to each component, form an index set inspectUnit of the corresponding two-dimensional coordinates of each component in the 2D image, and obtain the image position data of each component.

[0034] Furthermore, in step S3, the two-dimensional coordinate set and point index of the component in the image obtained in step S2 are used to search for and segment the point cloud data set of the region of interest. There are two optional methods, as described below.

[0035] An alternative method is Figure 3 As shown, the specific steps are as follows: Step S3.1: Since the obtained pointSet is a two-dimensional coordinate set in the pixel coordinate system, first convert the obtained two-dimensional coordinate set Convert from pixel coordinate system to image coordinate system, where (u i ,v i ) indicates that the i-th point in the two-dimensional coordinate set pointSet corresponds to the pixel in the v-th row and u-th column in the image, and pointNum is the number of points; Specifically, the conversion between the two coordinate systems can be completed by the following formula: ; Matrix representation using homogeneous coordinates: ; in and Indicates the physical size of each pixel in the u-axis and v-axis directions. Represents the coordinates in the two-dimensional coordinate set pointSet, represents the point where the optical center is perpendicular to the image coordinate system, Indicates the coordinates of the pixel points in the two-dimensional coordinate set in the image coordinate system; Step S3.2: Convert from the image coordinate system to the camera coordinate system; Specifically, the coordinates of the pixel point in the image coordinate system can be obtained through step S3.1 , the conversion from the image coordinate system to the camera coordinate system can be completed using the following formula: ; in, Indicates the position of the pixel in the camera coordinate system. Indicates the focal length of the camera; Step S3.3: Convert from the camera coordinate system to the world coordinate system; Specifically, when transforming from the camera coordinate system to the world coordinate system, the object will not be deformed, only rotation and translation are required. The following formula can be used to complete the transformation from the camera coordinate system to the world coordinate system: ; in, and are the coordinates of the point in the camera coordinate system and the 3D point cloud data in the world coordinate system, respectively. R and T are the rotation matrix and translation matrix from the camera coordinate system to the world coordinate system, respectively. From steps S3.1 to S3.3, it can be seen that the conversion between the pixel coordinate system and the world coordinate system can be completed by the following formula: ; in, is the point coordinate in the pixel coordinate system, Indicates that the camera optical center is perpendicular to the point in the image coordinate system, and is the focal length of the camera, representing the horizontal and vertical focal lengths respectively. R and T are the rotation and translation matrices from the camera coordinate system to the world coordinate system respectively. is the 3D point cloud data coordinate in the world coordinate system. According to this formula, the two-dimensional coordinate set obtained in step S2 can be converted into a 3D point cloud data set. ; Where i represents the point cloud coordinates of the i-th position, and pointcloudNum represents the number of point clouds.

[0036] Another alternative method is Figure 4 As shown, the specific steps are as follows: In step S3, the point cloud data of the corresponding image is obtained through step S1. Since step S2 obtains the point index pointID of the component on the binary image, the corresponding point cloud space can be filtered through the point index pointID to obtain the point cloud data of the component.

[0037] Step S3.1: placing the 3D point cloud acquisition device and the 2D image acquisition device in the same space and at the same time to perform data acquisition, and acquiring image data and point cloud data of the area containing the component in the same space and at the same time; Specifically, in step S3.1, a 3D point cloud acquisition device, such as a laser radar (LiDAR) or a structured light scanner, emits a laser beam or a structured light pattern, which intersects with the surface of the target object and reflects back. By calculating the time difference or phase difference of the reflected light and other parameters, the device can accurately measure the 3D coordinates of each reflection point, thereby constructing a 3D point cloud map of the target area. These point cloud data describe the shape, size and spatial position relationship of the object in detail. At the same time, a 2D image acquisition device, such as a high-definition camera, also shoots the same area at the same time to capture the 2D image information of the area. These image data not only contain visual features such as the color and texture of the object, but also reflect the layout and relative position of the object on the 2D plane. By synchronously acquiring these two types of data, we can obtain a comprehensive data set that contains both depth information (through point cloud data) and rich texture features (through 2D image data).

[0038] Step S3.2: Process the image data in step S2 to obtain the point index pointID of the component; Step S3.3: filtering the acquired point cloud data with pointID to obtain the point cloud data of the component; Specifically, in step S3.3, the point cloud subset is extracted by using the ExtractIndices filter according to the original point cloud collected in step S3.1 and the point index pointID obtained in step S3.2 , the specific steps of the ExtractIndices filter to extract a point cloud subset are as follows: 1) Downsampling using the voxel grid downsampling method: First, the original point cloud data is processed by voxel grid downsampling. The purpose of this step is to reduce the amount of point cloud data and improve the speed and efficiency of subsequent processing. The voxel grid downsampling method divides the point cloud data into multiple small three-dimensional cubes (voxels) and selects representative points in each voxel to generate downsampled point cloud data.

[0039] Principle of voxel grid downsampling method: A. Determine the minimum bounding box side length : ; in, as well as are the maximum and minimum values ​​of the minimum bounding box in the x, y, and z directions respectively. is the parameter for correcting the side length.

[0040] B. The calculation formula for the side length d of the voxel grid is: ; ; Where V is the volume of the voxel, It is the scaling factor used to adjust the voxel side length, s is the scaling factor, and N represents the total number of points in the point cloud data.

[0041] C. Calculate the center of gravity : ; in, is the coordinate of the ith point in the voxel grid, and n is the total number of points in the grid. The point closest to the centroid is selected and retained, and the other points in the grid are deleted to achieve point cloud downsampling.

[0042] 2) The SAC plane parameter model extracts a subset of point cloud data that conforms to the geometric model: The SAC (random sampling consistency) plane parameter model is used to extract a subset of point cloud data that conforms to the geometric model. The SAC algorithm fits the plane model by randomly selecting a set of points in the point cloud and calculates the parameters of the fitted plane. Then, according to the set threshold, the point cloud data that conforms to the plane model is extracted to form a subset of point cloud data that conforms to the geometric plane. In this process, segmentation algorithms (such as region growing, Euclidean clustering, etc.) can be used to further extract a subset of point cloud data that conforms to the geometric plane to improve the accuracy and completeness of the extraction.

[0043] 3) Use the negative variable to extract the remaining point cloud of the opposite point cloud set: By setting the value of the negative variable to True or False, you can choose to extract point cloud data that conforms to the model (negative=False) or does not conform to the model (negative=True).

[0044] 4) Use the remaining point cloud as the point cloud to be processed, return to step 2, and enter the next cycle: The remaining point clouds are taken as new point clouds to be processed, and the process returns to step 2 for the next round of loop processing. In this round of loop processing, you can choose to use different geometric models (such as spheres, cylinders, etc.) or adjust the parameters of the SAC algorithm to extract point cloud data subsets that conform to the new geometric model. Through multiple loop processing, different geometric features in the original point cloud data can be gradually extracted to form multiple point cloud subsets.

[0045] Further, in step S4, the 3D point cloud data set of the component is obtained through step S3, and the obtained 3D point cloud data is subjected to statistical outlier filtering, a statistical analysis is performed on the field of each point obtained, and points that do not meet certain standards are trimmed, thereby achieving the effect of removing outliers in the point cloud data and obtaining a processed 3D point cloud data set, which is specifically implemented as follows: Step S4.1: Perform statistical outlier filtering on the 3D point cloud data obtained in step S3, perform a statistical analysis on the area of ​​each point obtained, and trim off those points that do not meet certain standards.

[0046] Specifically, in step S4.1, the obtained 3D point cloud data is subjected to statistical outlier filtering, and a statistical analysis is performed on the neighborhood of each point. Based on the distance distribution characteristics from the point to all neighboring points, some outlier points that do not meet the requirements are filtered out. Figure 5 As shown in the figure, the principle of filtering 3D point cloud data by statistical outlier filtering method is: 1) Traverse the point cloud and find the k nearest neighbors for each point in the point cloud; 2) Calculate the average distance from the point to the k nearest neighbor points ; 3) Calculate the average distance The average and standard deviation ; The expression for the mean is

[0047] The expression for standard deviation is

[0048] 4) Set the distance threshold , whose expression is ; In the formula, is the scaling factor for the distance threshold, is the standard deviation multiplier, which is a non-zero natural number. In the point cloud denoising process, the scale factor is introduced , which aims to achieve a balance between denoising effect and preservation of local features of point cloud models. When it increases, the distance threshold will be enlarged, which will increase the amount of point cloud model data after denoising. On the contrary, when When the distance threshold is reduced, it will be reduced accordingly, resulting in a reduction in the amount of point cloud model data after denoising. The value of can be used to control the degree of simplification of point cloud data while removing noise according to actual needs to meet the needs of different application scenarios; 5) Compare the set distance threshold and , points exceeding the threshold are marked as outliers and removed, while points below the threshold are marked as inliers.

[0049] Further, in step S5, based on the 3D point cloud data set filtered in step S4, a random point selected at random is used as the starting point, and corresponding processing or no processing is performed according to different component features. For those components showing obvious spherical features, the outline of the sphere will be accurately depicted from the starting point using the sphere fitting algorithm, and its center and radius will be determined accordingly. With this information, the sphere can be segmented, cropped or otherwise processed according to actual needs. If the point cloud data contains a component with cylindrical features, the axis and radius of the cylinder can be accurately captured from the starting point using the cylinder fitting algorithm, and the positions of its two ends can be determined. With this information, the cylinder can be cut, adjusted or otherwise processed according to actual needs. For components with more complex shapes and more diverse features, a more flexible processing method is adopted. This includes using machine learning algorithms for feature classification and recognition, as well as combining shape analysis and geometric transformation methods for refined processing. These methods can more accurately capture the complex morphology and internal structure of components, providing a more refined and reliable data basis for subsequent applications. There is an example, which is based on the 3D point cloud data set filtered in step S4, takes an arbitrarily selected random point as a starting point, and performs corresponding processing according to different component features, as described below.

[0050] One example is 3D point cloud data based on cylindrical features. Taking the conductors in the power transformer valve as an example, if the conductor layout design inside the power transformer valve is improper, especially when the distance between the conductors is too close, it is very easy to cause discharge, which will not only damage the insulation of the equipment and reduce system efficiency, but may also trigger more serious safety accidents, such as short circuits, fires, and even equipment explosions, posing a major threat to personnel safety and the environment. In order to protect the safety of the staff and ensure the safe operation of the power transformer valve, safe distance detection is required. Removing the point cloud data at the head and tail of the conductor (the insulating glue part or the non-conductor part) can more accurately describe the shape and position of the conductor, such as Figure 6 As shown, the specific implementation method is as follows: Step S5.1: Based on the 3D point cloud data set of the wire filtered in step S4, the 3D point cloud data set of the wire filtered is , where each point Represents a three-dimensional coordinate . Pick any random point is the starting search point, and all points within the neighborhood radius r are searched through the starting search point, which is specifically expressed as follows: 1) Randomly select the starting search point: ; 2) Given a field radius: ; 3) Search for points in the neighborhood: Define a collection Indicates All points in a spherical area with a center and a radius of r, that is: ; in, represents the Euclidean distance, that is: ; Step S5.2: According to the radial characteristics of the wire, find the farthest point p1 from the current initial search position in all points within the neighborhood radius r, record the position coordinates of point p1, connect the current point and the farthest point p1 to calculate the direction d1, if the direction d1 is upward along the radial characteristics of the wire, then determine the direction d1 as the target direction, otherwise continue to search from far to near within the field radius r for points whose direction meets the upward radial characteristics of the wire, specifically expressed as follows: 1) Find the farthest point p1: exist In the equation, the farthest p1 can be expressed as:

[0051] 2) Record the position coordinates of point p1; 3) Calculate direction d1: The direction d1 is given by The vector pointing to P1 can be expressed as:

[0052] 4) Determine whether the direction d1 is upward along the radial feature of the wire. , which accepts a direction vector Takes as input and returns a Boolean value indicating whether the direction is up along the wire radial feature. is true, then d1 is the target direction, otherwise continue in Search for a point that meets the radial characteristics of the wire upward.

[0053] Step S5.3: Find another point p2 from far to near in the opposite direction to d1, record the position coordinates of point p2, and calculate the direction d2 based on the line connecting the current point and point p2. If the direction d2 is downward along the radial feature of the wire, then determine the direction d2 as the target direction. Otherwise, continue to find another point from far to near in the field radius r in the opposite direction to d1 and the point direction meets the radial feature of the wire downward. The specific expression is as follows: 1) Find the point p2 that is in the opposite direction to d1. By calculating the set Each point and The angle between the vector formed and -d1 is calculated, and the point with the smallest angle is selected to achieve: A. Calculation Set All points in Vector set formed : ; ; in, Representing a collection The i-th point and The vector formed.

[0054] B. Calculate the set of angles between the vector and -d1 : ; C. Select the one with the smallest angle (or The point closest to 180 degrees is taken as point p2: ; in, is a collection of point clouds that accepts direction vector d1 and returns a function for point p2 which is roughly opposite to d1.

[0055] 2) Calculate direction d2: ; 3) Determine whether direction d2 is downward along the radial feature of the wire. Similar to step S5.2.

[0056] Step S5.4: Use p1 as the initial search point to search again, search all points within the neighborhood radius r of p1 as the origin, find point p3 with the same direction as the target and the farthest position from far to near, and update the position coordinate information of point p1 to p3, calculate direction d3, if direction d3 is upward along the radial feature of the wire, then determine direction d3 as the updated target direction, otherwise continue to search for points whose direction meets the radial feature of the wire upward from far to near within the field radius r, specifically expressed as follows: 1) Search for points within the area: ; 2) Find point p3 with the same direction as the target: Similar to step S5.3, the point with the smallest angle (or closest to 0 degrees, indicating the same direction) and the farthest position is selected as p3.

[0057] ; in, is a collection of point clouds that accepts direction vector d1 and returns a function for point p3 that is approximately the same as d1; 3) Calculate direction d3: ; 4) Update the position coordinate information of point p1: ; 5) Determine whether direction d3 is upward along the radial feature of the wire. Similar to step S5.2.

[0058] Step S5.5: Similarly, search for all points within the neighborhood radius r with p2 as the origin, and determine the farthest point p4 in the same direction as d2, and update the position coordinate information of point p2 to p4, and calculate the new direction d4 by connecting the current point and the farthest point p4. If the direction d4 is downward along the radial feature of the wire, then determine the direction d4 as the target direction, otherwise continue to find points from far to near within the field radius r whose point direction meets the radial feature of the wire downward, which is specifically expressed as follows: 1) Search for points within the area: ; 2) Find point p4 with the same direction as the target: Similar to step S5.4, the point with the smallest angle (or closest to 0 degrees, indicating the same direction) and the farthest position is selected as p4.

[0059] ; in, is the direction d2 and the vector The angle between It is a preset angle threshold used to determine the standard of the same direction.

[0060] 3) Calculate direction d4: ; 4) Update the position coordinate information of point p1: ; 5) Determine whether direction d4 is downward along the radial feature of the wire. Similar to step S5.2.

[0061] Step S5.6: Repeat this process until no more points are added, and output the position coordinate information stored in p1 and p2 to obtain the point cloud positions of the wire head and tail.

[0062] Step S5.7: Obtain the point cloud position of the wire head and tail through step S5.6, remove the data points of the wire head and tail within a certain safety threshold range L, and obtain the point cloud data without the wire head and tail.

[0063] Specifically, according to step S5.6, the point cloud position of the wire head and tail is obtained, and the point cloud position of the wire head is taken as the origin, and a certain safety threshold range L is taken as the radius, and the data within the radius L is filtered out; similarly, the point cloud position of the wire tail is taken as the origin, and a certain safety threshold range L is taken as the radius, and the data within the radius L is filtered out, which is specifically expressed as follows: set up , the safety threshold range is L, for any point in the point cloud dataset P but: ; like , then point It is within the safety threshold and should be filtered out.

[0064] Further, in step S6, the point cloud of the current component is used to determine whether there is a larger point cloud cluster around it, and the detection result is output; Specifically, calculate the two points with the longest distance on the horizontal section of the current component point cloud, and record the distance as ,by As the diameter, a dynamic sphere is synthesized, where It is the range to be searched near the current component point cloud. Must be greater than 0, take the center point cloud coordinates of the same plane of the component as the center of the sphere, and keep changing the center position of the dynamic sphere. External There are point cloud clusters in the memory, and the spatial locations of the point cloud clusters are output. The specific expressions are as follows: 1) Calculate the two points with the longest distance on the horizontal section of the current component point cloud: The component point cloud is P, which contains n points, namely ,in . Find the two points farthest apart on the transverse plane and , so that ; 2) Synthesize dynamic sphere: ; ; by As the diameter, a dynamic sphere is synthesized with radius R, where It is the range that needs to be searched near the current component point cloud.

[0065] 3) Determine the initial center of the sphere: The center point cloud coordinates of the same plane of the component are used as the center of the sphere : ; in is the mean or a fixed value of the z coordinate.

[0066] 4) Search for the surrounding point cloud cluster; if found External There are point cloud clusters in the memory, and the spatial locations of the point cloud clusters are output: For a sphere center position and radius , if there exists a point cloud p that satisfies: ; Then output the spatial location of the point cloud cluster.

[0067] It should be noted that the technical solution of the present invention can be further illustrated by the following experiments.

[0068] 1. Experimental background: As a key component of the smart grid, substations undertake the core functions of power conversion and power transmission. If the layout design of the wires inside the power switching valve is improper, especially when the distance between the wires is too close, it is very easy to cause discharge, which will not only damage the insulation of the equipment and reduce the efficiency of the system, but also may trigger more serious safety accidents, such as short circuits, fires, and even equipment explosions, posing a major threat to personnel safety and the environment. In order to ensure the safety of the staff and ensure the safe operation of the power switching valve, safe distance detection is required. Therefore, the method in this paper can be used to carry out target detection research on substation wires.

[0069] 2. Experimental conditions: This experiment uses Python 3.8.18 and C++ programming languages ​​on a computer with Windows 10. The operating environment is implemented using Anaconda and PyTorch 2.1.2, and the NVIDIA GeForce RTX 3080 graphics card and CUDA 12.1 are used for computing acceleration, and it is carried out on VSCode and VS coding and debugging platforms.

[0070] 3. Experimental content: Experiment 1 uses a device that integrates a 3D point cloud acquisition device and a 2D image acquisition device on the same device to achieve 3D information scanning and 2D information acquisition. This device can obtain the original image data and original 3D point cloud data in the same space and at the same time, and these data form an image data set and a 3D point cloud data set. The collected image data is as follows Figure 7As shown in (a), the 3D point cloud data is Figure 7 (e) as shown.

[0071] Experiment 2, the deep learning training-inference method is based on the Deeplabv3+ inference algorithm implemented in pytorch. The model training uses the SGD optimizer, with the initial learning rate set to 0.01, the momentum set to 0.9, and the weight decay set to 0.0005. The training was run for 20,000 iterations, and the model was verified after every 2,000 iterations. The labelme software was used to annotate the wires in the collected images, and the annotated image data was stored in the VOC data format, divided into a training set and a test set. The processed image data was used as the input of the deep learning training-inference method to perform the training and inference process. The trained model was tested using the test set images, and the aAcc (overall accuracy) was 97.65, the mAc (average accuracy) was 87.46, and the test time was 0.1889s. The test calculation results can verify that the technology of using the DeepLabv3+ network for pixel-level segmentation of wires is feasible. Use the trained model to load, use the vscode programming environment, and use the python language to program the reasoning. Through reasoning, the binary image after binarization and the pixel position of the wire are obtained, such as Figure 7 (b) as shown.

[0072] The binary image is closed by dilating it first and then corroding it, which closes the narrow gaps and long and narrow grooves of the wire, eliminates small holes, fills the breaks, and retains the original image size. Figure 7 As shown in (c). The eight-neighborhood algorithm is used to check the grayscale difference of the eight neighbors clockwise with the first grayscale 255 pixel as the center to determine the edge point, and the edge tracking is used to connect the edge points to form a contour, and the interior of the contour is filled with a grayscale value of 255 to distinguish and identify each wire in the image, as shown in Figure 7 (d) as shown.

[0073] Experiment 3: For each wire coordinate set, The operation obtains the position index of the corresponding coordinates in the original 3D point cloud data, and uses the ExtractIndices filter to fuse the point cloud according to the wire index. The fusion result is as follows Figure 7 (f) as shown.

[0074] Experiment 4, through Experiment 3 to obtain the 3D point cloud data set of the wire, the obtained 3D point cloud data is subjected to statistical outlier filtering, a statistical analysis is performed on the area of ​​each point obtained, and those points that do not meet certain standards are trimmed, so as to achieve the effect of removing outliers in the point cloud data, and obtain the processed 3D point cloud data set. The result after filtering is as follows Figure 7 (g) as shown.

[0075] Experiment 5, select any random point as the starting point, search the wire head and tail point cloud position based on the radial characteristics of the wire, and remove the wire head and tail data points. Use the current wire point cloud to determine whether there is a larger point cloud cluster around it. The point cloud cluster search situation is as follows Figure 7 As shown in (h), the spatial location of the output point cloud cluster is as follows Figure 8 shown.

[0076] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

Claims

1. A method for identifying and locating three-dimensional objects in complex scenes by integrating point cloud clustering and image texture association, characterized in that: The steps include: Step S1, collecting image data and point cloud data through sensors to form an image data set and a three-dimensional point cloud data set; Step S2, processing the collected image data, using a deep learning training-inference method to perform annotation training and inference on the 2D image data set to obtain image position data of the area of ​​interest; Step S3, using the position of the component in the image as an index, searching for and segmenting the 3D point cloud data of the component; Step S4, filtering and denoising the obtained 3D point cloud data to eliminate noise and clutter therein; Step S5, starting from an arbitrarily selected random point, and performing corresponding processing according to different component features; Step S6, using the current component point cloud to determine whether there are any point cloud cluster components around, and outputting the spatial position of the point cloud cluster.

2. The method for complex scene three-dimensional target recognition and positioning by integrating point cloud clustering and image texture association according to claim 1, characterized in that: The step S2 specifically includes firstly marking the region of interest by using a marking software, converting the marked image file into a file in a data set format required for model training, and then using the file as input data to train it by using a deep learning training-inference method to obtain a model adapted to the data; inferring the region of interest of the two-dimensional image data based on the model, and binarizing the region of interest to obtain a binary image; The binary image is expanded and eroded. The specific representation of erosion is as follows: ； ； Where C represents the erosion operation, (x, y) is the pixel position in image I, and (i, j) is the relative position in the structural element S; The specific expression of expansion is as follows: ； ； Where E represents the dilation operation, (x, y) is the pixel position in image I, and (i, j) is the relative position in the structural element S; Traverse the processed binary image from top to bottom to find the pixel points with a value of 255, and arrange the component areas composed of the points that meet the requirements one by one according to the eight-neighborhood algorithm to obtain the specific number of components in the image and the corresponding component numbers, and obtain the position point coordinate set of each component and the corresponding position point index; remove the position point coordinate set that is less than a certain threshold, so as to obtain the total number of components in the image and the index set of the two-dimensional coordinates of the pixel points corresponding to each component.

3. The method for complex scene three-dimensional target recognition and positioning by integrating point cloud clustering and image texture association according to claim 2 is characterized in that: In step S3, the index set of the two-dimensional coordinates of the component in the image obtained in step S2 is used as an index to obtain a 3D point cloud data set for search and segmentation; there are two optional methods, one is to achieve the conversion between the image and the 3D point cloud by converting the points in the pixel coordinate system to the world coordinate system; Another method is to filter the corresponding point cloud space through the position point index pointID to obtain the point cloud data of the component.

4. The method for complex scene three-dimensional target recognition and positioning by integrating point cloud clustering and image texture association according to claim 3 is characterized in that: The method of converting the points in the pixel coordinate system to the world coordinate system to realize the conversion between the image and the 3D point cloud specifically includes: Step S3.1: converting the acquired two-dimensional coordinate set Convert from pixel coordinate system to image coordinate system, where (u i ,v i ) indicates that the i-th point in the position point coordinate set pointSet corresponds to the pixel in the v-th row and u-th column in the image, and pointNum indicates the total number of pixels; Step S3.2: Convert from the image coordinate system to the camera coordinate system; Step S3.3: Convert from the camera coordinate system to the world coordinate system; Through steps S3.1 to S3.3, the conversion between the pixel coordinate system and the world coordinate system is completed by the following formula: ; in, Represents the point on the Z axis in the camera coordinate system. is the point coordinate in the pixel coordinate system, and Indicates the physical size of each pixel in the u-axis and v-axis directions; Indicates the focal length of the camera; Indicates that the camera optical center is perpendicular to the point in the image coordinate system, and Respectively represent the focal length of the camera in the horizontal and vertical directions, R and T are the rotation matrix and translation matrix from the camera coordinate system to the world coordinate system, It is the coordinate of 3D point cloud data in the world coordinate system.

5. The method for complex scene three-dimensional target recognition and positioning by integrating point cloud clustering and image texture association according to claim 3 is characterized in that: The filtering operation on the corresponding point cloud space through the position point index pointID to obtain the point cloud data of the component specifically includes: step S3.1: placing the three-dimensional point cloud acquisition device and the two-dimensional image acquisition device in the same space and at the same time to perform data acquisition, and collecting image data and point cloud data of the area containing the component in the same space and at the same time; step S3.2: processing the image data in step S2 to obtain the position point index pointID of the component; step S3.3: using the ExtractIndices filter to extract the point cloud subset pointcloudSet according to the original point cloud collected in step S3.1 and the point index pointID obtained in step S3.

2.

6. The method for complex scene three-dimensional target recognition and positioning by integrating point cloud clustering and image texture association according to claim 5, characterized in that: The step S3.3 specifically includes: 1) downsampling using the voxel grid downsampling method; 2) extracting a point cloud data subset that conforms to the geometric model using the SAC plane parameter model; 3) extracting the remaining point clouds of the opposite point cloud set using the negative variable; 4) using the remaining point clouds as the point clouds to be processed, returning to step 2, and entering the next cycle.

7. The method for complex scene three-dimensional target recognition and positioning by integrating point cloud clustering and image texture association according to claim 1, characterized in that: In step S4, the 3D point cloud data set of the component is obtained through step S3, and the obtained 3D point cloud data is subjected to statistical outlier filtering, a statistical analysis is performed on the field of each point obtained, and those points that do not meet the standards are trimmed to obtain a processed 3D point cloud data set; the statistical outlier filtering of the obtained 3D point cloud data specifically includes: 1) traversing the point cloud data set, and finding its k nearest neighbors for each point in the point cloud data set; 2) calculating the average distance from the point to the k nearest neighbor points ; 3) Calculate the average distance The average and standard deviation ;4) Set distance threshold , whose expression is , where is the scaling factor for the distance threshold, is the standard deviation multiplier, which is a non-zero natural number; 5) Compare the set distance threshold and , points exceeding the threshold are marked as outliers and removed, while points below the threshold are marked as inliers.

8. The method for complex scene three-dimensional target recognition and positioning by integrating point cloud clustering and image texture association according to claim 1, characterized in that: In step S5, based on the 3D point cloud data set filtered in step S4, an arbitrarily selected random point is used as the starting point to perform corresponding processing according to the characteristics of different components; for those components showing obvious spherical features, starting from the starting point, a sphere fitting algorithm is used to accurately depict the outline of the sphere, and its center and radius are determined accordingly; the sphere is segmented and cropped based on this information; if the point cloud data contains components with cylindrical features, starting from the starting point, a cylinder fitting algorithm is used to accurately capture the axis and radius of the cylinder, and then determine the positions of its two ends; the cylinder is cut and adjusted based on this information; for components with more complex shapes and more diverse features, machine learning algorithms are used for feature classification and recognition, as well as processing combined with shape analysis and geometric transformation methods.

9. The method for complex scene three-dimensional target recognition and positioning by integrating point cloud clustering and image texture association according to claim 1, characterized in that: In step S6, the two points with the longest distance on the horizontal section of the current component point cloud are calculated and the distance is recorded as ,by As the diameter, a dynamic sphere is synthesized, where δ is the range to be searched near the current part point cloud. If it is greater than 0, the center point cloud coordinates of the same plane of the component are used as the center of the sphere, and the center position of the dynamic sphere is constantly changed. External There are point cloud clusters in the memory, and the spatial locations of the point cloud clusters are output.

10. The method for complex scene three-dimensional target recognition and positioning by integrating point cloud clustering and image texture association according to claim 9, characterized in that: The specific steps of step S6 are as follows: 1) Calculate the two points with the farthest distance on the horizontal section of the current component point cloud; the component point cloud is P, which contains n points, that is, ,in ; Find the two points farthest apart on the transverse section and , so that ; 2) Determine the diameter and search range of the dynamic sphere; ; ； by As the diameter, a dynamic sphere is synthesized with radius R, where It is the range to be searched near the current component point cloud; 3) Determine the center of the dynamic sphere; The center point cloud coordinates of the same plane of the component are used as the center of the sphere : ； in is the mean or a fixed value of the z coordinate; 4) Search for point clouds around the element; If the maximum distance on the horizontal section of the current component point cloud is found External There are point cloud clusters in the memory, and the spatial positions of the point cloud clusters are output: for a certain sphere center position and radius , if there exists a point cloud p that satisfies: ； Then output the spatial location where the point cloud cluster exists.

Images