Three-dimensional path planning method for full-coverage detection task of outer surface of automobile

Abstract

The invention provides a three-dimensional path planning method for a full-coverage detection task of the outer surface of an automobile. The detection plane is defined according to the visual field of the camera of the detection device, and it can be guaranteed that the visual field of the camera of the detection device completely covers the surface to be detected while the detection viewpoint is generated. Through outer surface clustering partition and viewpoint subdivision, it is ensured that a detection path avoids a perforated part, and the shape and size of each sub-region are convenient to plan. According to the invention, rapid full-coverage detection path planning is carried out through the hybrid greedy-non-dominated sorting genetic algorithm, the improved greedy search algorithm carries out preliminary rapid connection on photographing viewpoints, an initial detection path is formed as a high-quality initial solution of the non-dominated sorting genetic algorithm, search is carried out near a better solution space, and the detection efficiency is improved. And the time and the number of iterations required for searching the optimal solution are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile paint surface detection, and in particular to a three-dimensional path planning method for a full coverage detection task of an automobile outer surface. Background Art

[0002] With the intelligent upgrade of the automobile manufacturing industry, the quality inspection of the exterior surface of the vehicle body has become increasingly important. Traditional manual visual inspection relies on experienced technicians, and has problems such as low efficiency, strong subjectivity, and easy omissions. It is difficult to meet the high-precision and high-coverage inspection requirements of modern production lines. In recent years, inspection technology based on automated equipment has gradually replaced manual labor, but there are still problems in the full coverage inspection of complex three-dimensional surfaces. For example, when planning the inspection path, the existing automated methods often lead to blind spots or redundant paths due to sudden changes in the curvature of the vehicle body surface and the presence of hole structures (such as windows, sunroofs, etc.).

[0003] Currently, mainstream inspection path planning methods include manual teaching and offline programming. Manual teaching relies on an operator guiding the end-of-arm inspection device on-site to record inspection points and paths. While simple to operate, it exposes the operator to hazardous environments. Furthermore, the quality of inspection paths for complex surfaces is highly dependent on operator experience, making missed inspections and path duplication more likely. Offline programming improves planning and inspection efficiency by optimizing inspection points and paths through algorithms. However, coverage path planning for automotive exterior surface inspection tasks is characterized by large variations in surface curvature and the presence of holes (such as windows and sunroofs). Existing coverage path planning algorithms often segment a two-dimensional plane into hole-free subregions (such as cell decomposition methods), then perform path planning and path combination within each subregion. This approach struggles to adapt to the complex geometric features of three-dimensional surfaces. Furthermore, traditional greedy search or genetic algorithms for path optimization are prone to local optima or slow convergence, making it difficult to balance multiple objectives such as path length, posture smoothness, and hole avoidance, further limiting the efficiency of full coverage inspection. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems in the prior art and propose a three-dimensional path planning method for the task of full coverage detection of the outer surface of an automobile.

[0005] The present invention is implemented by the following technical solution. The present invention proposes a three-dimensional path planning method for the task of full coverage detection of the outer surface of an automobile, the method comprising the following steps:

[0006] Step 1: Preprocess the car model by extracting the car's exterior surface point cloud from multiple perspectives and performing normal vector sampling calculations to remove noise points inside the car and extract a single-layer point cloud data of the car's exterior surface.

[0007] Step 2: A two-stage clustering algorithm is used to cluster and partition the car's exterior surface. In the first stage, the point cloud distance threshold and the normal vector angle threshold are dynamically adjusted according to the local density to generate initial clusters. In the second stage, unclassified points are assigned to the nearest cluster according to the distance threshold; the car's exterior surface is divided into several main areas.

[0008] Step 3: Sample and generate detection viewpoints for each major area. Define the detection plane based on the camera field of view of the detection device. Based on the size of the detection plane, downsample each major area while ensuring complete coverage to obtain the viewpoints used for subsequent photo detection.

[0009] Step 4: Subdivide the viewpoints of each main area. By defining a 3D scanning detection plane, generate multiple subdivision sub-areas based on the existing holes, reasonably avoid the parts with holes, reduce the length of the subsequent detection path, and ensure that the shape and size of each sub-area are convenient for path planning;

[0010] Step 5: Use a hybrid greedy-non-dominated sorting genetic algorithm to connect the detection viewpoints to generate the optimal detection path, perform collision detection and transition viewpoint interpolation optimization operations on the optimal detection path to ensure the feasibility and safety of the detection path.

[0011] Furthermore, in the first stage of step 2, the points that meet both the distance threshold ε and the normal vector angle threshold θ are grouped into the same cluster. Specifically: initialize the threshold parameters, perform local density calculation, and for each point p i Calculate its local density ρ i =|KNN(p i )|, where KNN(p i ) is point p i The set of K nearest neighbor points of ;

[0012] Define the distance threshold and take the median of the Euclidean distance of neighboring points:

[0013] ε(p i )=median(||p i -p k ||2),p k ∈KNN(p i )

[0014] Define the normal vector angle threshold, according to point p i Normal vector n i and neighboring point p k Normal vector n k , calculated by combining the mean and standard deviation:

[0015] θ(p i )=mean(cos -1 (ni ·n k ))+std(cos -1 (n i ·n k )),p k =KNN(p i )

[0016] Perform core point determination, if point p i There are several points in the neighborhood of , which satisfy:

[0017] ||p i -p k ||2≤ε(p i ) and cos -1 (n i ·n k )≤θ(p i )

[0018] Then mark p i As the core point, create a new cluster C k ; Recursively merge all neighboring points that meet the above conditions to complete the region expansion and form the initial cluster.

[0019] Furthermore, in the second stage of step 2, the unclassified points are assigned to the nearest existing cluster based on the condition that only the distance threshold ε is satisfied; the clustering results of the first stage and the clustering results of the second stage are optimized as follows:

[0020] Perform morphological closing operation on the boundaries of each cluster to eliminate jagged edges; merge similar clusters. If there are adjacent clusters C a and C b The mean angle of the normal vector is less than the given global threshold θ global , then merge the two clusters:

[0021]

[0022] According to the above steps, the outer surface point cloud of the car can be preliminarily divided into various main areas.

[0023] Furthermore, the step three is specifically as follows:

[0024] Step 3.1: Rasterize the point cloud data of each main area and use the center point of each grid as the representative sampling point. The grid size viewsize is related to the image sensor size w×h, w≤h, camera focal length f, and shooting distance d. The calculation formula is:

[0025]

[0026] Step 3.2: Downsample each main area according to the grid size obtained in step 3.1 to generate viewpoints that meet the full coverage detection of each main area. The average of the point cloud normal vectors in each grid is the viewpoint's line of sight, i.e., the normal vector:

[0027]

[0028] Where m is the total number of point clouds in the grid, n j is the normal vector of each point, n i The obtained viewpoint p i The normal vector of

[0029] The center point of the grid is along the viewpoint line of sight direction n i =(n ix ,n iy ,n iz ) Move the shooting distance d to get the viewpoint p i Location information (x i ,y i ,z i ), each viewpoint p finally obtained i Contains coordinate location information and line of sight information.

[0030] Furthermore, the step 4 is specifically as follows:

[0031] Step 4.1: Define the 3D scanning plane for a set of primary region detection viewpoints where z max Detect the maximum coordinate value of the viewpoint set in the z-axis direction for a certain main area; the scanning plane is scanned step by step according to the set viewsize:

[0032]

[0033] Where k is the index of the three-dimensional scanning plane, and its value range is k=0,1,2,...; the termination condition of the scanning plane is z scan <z min , where z min Detect the minimum coordinate value of the viewpoint set in the z-axis direction for a certain main area;

[0034] Step 4.2: Perform viewpoint overlap judgment and sorting, and record the viewpoints that overlap with the scan plane as a set The overlapping geometry is defined as the z coordinate value of the viewpoint is located at the current z coordinate of the scan plane scan within a given small range around the value;

[0035] Step 4.3: Calculate and classify the distance between adjacent points. For the sorted overlapping viewpoint sequence, calculate the adjacent point p ki and p ki+1 Distance between:

[0036]

[0037] d ki Whether all the overlapping viewpoints are less than or equal to viewsize is used as the judgment condition; the overlapping viewpoints that meet the conditions are classified into category A and recorded as set P A :

[0038]

[0039] The overlapping viewpoints that do not meet the conditions are classified into category B and are recorded as set P B :

[0040]

[0041] Class A represents the classified areas without holes, and Class B represents the classified areas with holes;

[0042] Step 4.4: Use the DBSCAN algorithm to reclassify the A and B viewpoints obtained in step 4.3, and obtain the A viewpoint labeled P. A1 , P A2 ...The segmentation result is P for the viewpoint of type B. B1 , P B2 ...the segmentation results, and finally obtain the various segmented detection areas divided for a certain main area.

[0043] Furthermore, in step 4.2, The specific method of sorting is to sort all viewpoints in ascending order by x coordinates first, and then sort them in ascending order by y coordinates for the same x coordinates; the sorted overlapping point cloud sequence is expressed as where p ki =(x ki ,y ki ,z ki ,n kix ,n kiy ,n kiz ), i = 1, 2, ..., m, m is the number of overlapping point clouds on the scanning plane with the current index k.

[0044] Furthermore, in step five, a greedy search algorithm is used to preliminarily connect the determined detection viewpoints to form an initial detection path; the initial detection path obtained by the greedy search algorithm is used as a high-quality initial solution of the non-dominated sorting genetic algorithm NSGA-II, and the initial solution is optimized using the non-dominated sorting genetic algorithm; through selection, crossover and mutation operations, continuous iterative evolution is carried out, and the multi-objective fitness function is used as the evaluation criterion to find the optimal detection path.

[0045] Furthermore, a hybrid greedy-non-dominated sorting genetic algorithm is used to obtain the viewpoint traversal path for each area. The optimization path is calculated to see whether the line connecting two adjacent viewpoints overlaps with the car model. If there is an overlap, it is marked as a collision. Interpolation is performed between the two viewpoints to generate a transition viewpoint to avoid the collision. The line of sight of the transition viewpoint is the average of the line of sight of the two viewpoints, and the coordinates are the average of the two viewpoints and then moved along the line of sight by a certain safe distance until the collision is avoided. Finally, a full coverage detection path is generated.

[0046] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the three-dimensional path planning method for the task of fully covering the outer surface of an automobile.

[0047] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a three-dimensional path planning method for the task of fully covering the outer surface of an automobile.

[0048] Beneficial effects of the present invention:

[0049] The present invention defines the detection plane according to the camera field of view of the detection device, and can ensure that the camera field of view of the detection device fully covers the surface to be detected while generating the detection viewpoint. The outer surface clustering and viewpoint subdivision ensure that the detection path avoids the porous part and the shape and size of each sub-area are easy to plan. The present invention uses a hybrid greedy-non-dominated sorting genetic algorithm to perform fast full-coverage detection path planning. The improved greedy search algorithm performs preliminary and rapid connection of the photographing viewpoints to form an initial detection path as a high-quality initial solution of the non-dominated sorting genetic algorithm, and searches near a better solution space, reducing the time and number of iterations required to find the optimal solution. Compared with existing methods, the present invention effectively solves the problems of low efficiency and easy error of traditional manual detection, as well as the blind spots and lengthy paths of existing automated detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0051] Figure 1 This is a flow chart of a three-dimensional path planning method for the task of full coverage detection of the exterior surface of an automobile as described in the present invention.

[0052] Figure 2 This is the preprocessing result diagram.

[0053] Figure 3 Schematic diagram of external surface classification.

[0054] Figure 4 Generate a schematic diagram for detecting viewpoints.

[0055] Figure 5 Schematic diagram of viewpoint subdivision detection.

[0056] Figure 6 Schematic diagram of score calculation for viewpoints to be visited in the neighborhood.

[0057] Figure 7 Schematic diagram of the full coverage detection path for the entire vehicle. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] Specifically, combined Figure 1-Figure 7 The present invention proposes a three-dimensional path planning method for the task of full coverage detection of the outer surface of an automobile, the method comprising the following steps:

[0060] Step 1: Preprocess the car model by extracting the car's exterior surface point cloud from multiple perspectives and performing normal vector sampling calculations to remove noise points inside the car and extract a single-layer point cloud data of the car's exterior surface.

[0061] Step 2: Given the large variations in car surface curvature, a two-stage clustering algorithm is used to cluster and partition the car's exterior surface. In the first stage, the point cloud distance threshold and the normal vector angle threshold are dynamically adjusted based on local density to generate initial clusters. In the second stage, unclassified points are assigned to the nearest cluster based on the distance threshold. The car's exterior surface is then divided into several main areas (roof, door, rear, front, etc.).

[0062] Step 3: Sample and generate detection viewpoints for each major area. Define the detection plane based on the camera field of view of the detection device. Based on the size of the detection plane, downsample each major area while ensuring complete coverage to obtain the viewpoints used for subsequent photo detection.

[0063] Step 4: Subdivide the viewpoints of each main area. By defining a 3D scanning detection plane, generate multiple sub-areas based on the existing holes, reasonably avoid areas with holes (such as windows and sunroofs), reduce the length of the subsequent detection path, and ensure that the shape and size of each sub-area are convenient for path planning;

[0064] Step 5: Use the hybrid greedy-non-dominated sorting genetic algorithm to connect the detection viewpoints to generate the optimal detection path. Perform collision detection and transition viewpoint interpolation optimization on the optimal detection path to ensure the feasibility and safety of the detection path. The detection paths obtained in each sub-area can be allocated and integrated according to the robot workspace. Figure 1 shown.

[0065] In the first stage of step 2, points that meet both the distance threshold ε and the normal vector angle threshold θ are grouped into the same cluster. Specifically: initialize the threshold parameters, perform local density calculation, and for each point p i Calculate its local density ρ i =|KNN(p i )|, where KNN(p i ) is point p i The set of K nearest neighbor points of ;

[0066] Define the distance threshold and take the median of the Euclidean distance of neighboring points:

[0067] ε(p i )=median(||p i -p k ||2),p k ∈KNN(p i )

[0068] Define the normal vector angle threshold, according to point p i Normal vector n i and neighboring point p k Normal vector n k , calculated by combining the mean and standard deviation:

[0069] θ(p i )=mean(cos -1 (n i ·n k ))+std(cos -1 (n i ·n k )),p k =KNN(p i )

[0070] Perform core point determination, if point p i There are several points in the neighborhood of , which satisfy:

[0071] ||p i -p k ||2≤ε(p i ) and cos -1 (n i ·nk )≤θ(p i )

[0072] Then mark p i As the core point, create a new cluster C k ; Recursively merge all neighboring points that meet the above conditions to complete the region expansion and form the initial cluster.

[0073] In the second stage of step 2, the unclassified points are assigned to the nearest existing cluster based on the condition that only the distance threshold ε is satisfied. The clustering results of the first stage and the clustering results of the second stage are optimized as follows:

[0074] Perform morphological closing operation on the boundaries of each cluster to eliminate jagged edges; merge similar clusters. If there are adjacent clusters C a and C b The mean angle of the normal vector is less than the given global threshold θ global , then merge the two clusters:

[0075]

[0076] According to the above steps, the outer surface point cloud of the car can be preliminarily divided into various main areas.

[0077] The step three is specifically as follows:

[0078] Step 3.1: Rasterize the point cloud data of each main area and use the center point of each grid as the representative sampling point. The grid size viewsize is related to the image sensor size w×h, w≤h, camera focal length f, and shooting distance d. The calculation formula is:

[0079]

[0080] Step 3.2: Downsample each main area according to the grid size obtained in step 3.1 to generate viewpoints that meet the full coverage detection of each main area. The average of the point cloud normal vectors in each grid is the viewpoint's line of sight, i.e., the normal vector:

[0081]

[0082] Where m is the total number of point clouds in the grid, n j is the normal vector of each point, n i The obtained viewpoint p i The normal vector of

[0083] The center point of the grid is along the viewpoint line of sight direction n i =(n ix ,n iy ,n iz ) Move the shooting distance d to get the viewpoint pi Location information (x i ,y i ,z i ), each viewpoint p finally obtained i Contains coordinate location information and line of sight information.

[0084] The step 4 is specifically as follows:

[0085] Step 4.1: Define the 3D scanning plane for a set of primary region detection viewpoints where z max Detect the maximum coordinate value of the viewpoint set in the z-axis direction for a certain main area; the scanning plane is scanned step by step according to the set viewsize:

[0086]

[0087] Where k is the index of the three-dimensional scanning plane, and its value range is k=0,1,2,...; the termination condition of the scanning plane is z scan <z min , where z min Detect the minimum coordinate value of the viewpoint set in the z-axis direction for a certain main area;

[0088] Step 4.2: Perform viewpoint overlap judgment and sorting, and record the viewpoints that overlap with the scan plane as a set The overlapping geometry is defined as the z coordinate value of the viewpoint is located at the current z coordinate of the scan plane scan within a given small range around the value;

[0089] Step 4.3: Calculate and classify the distance between adjacent points. For the sorted overlapping viewpoint sequence, calculate the adjacent point p ki and p ki+1 Distance between:

[0090]

[0091] d ki Whether all the overlapping viewpoints are less than or equal to viewsize is used as the judgment condition; the overlapping viewpoints that meet the conditions are classified into category A and recorded as set P A :

[0092]

[0093] The overlapping viewpoints that do not meet the conditions are classified into category B and are recorded as set P B :

[0094]

[0095] Class A represents the classified areas without holes, and Class B represents the classified areas with holes;

[0096] Step 4.4: Use the DBSCAN algorithm to reclassify the A and B viewpoints obtained in step 4.3, and obtain the A viewpoint labeled P. A1 , P A2 ...The segmentation result is P for the viewpoint of type B. B1 , P B2 ...the segmentation results, and finally obtain the various segmented detection areas divided for a certain main area.

[0097] In step 4.2, The specific method of sorting is to sort all viewpoints in ascending order by x coordinates first, and then sort them in ascending order by y coordinates for the same x coordinates; the sorted overlapping point cloud sequence is expressed as where p ki =(x ki ,y ki ,z ki ,n kix ,n kiy ,n kiz ), i = 1, 2, ..., m, m is the number of overlapping point clouds on the scanning plane with the current index k.

[0098] In step five, a greedy search algorithm is used to preliminarily connect the determined detection viewpoints to form an initial detection path. The improved greedy search algorithm, based on a prescribed neighboring viewpoint scoring strategy, selects the highest-scoring, unvisited viewpoint within the neighborhood for connection, thereby rapidly generating a feasible viewpoint connection sequence. The initial detection path obtained by the greedy search algorithm is used as a high-quality initial solution for the non-dominated sorting genetic algorithm (NSGA-II). This initial solution is then optimized using the non-dominated sorting genetic algorithm. Through selection, crossover, and mutation operations, continuous iterative evolution is performed, and a multi-objective fitness function is used as the evaluation criterion to find the optimal detection path. The optimization objective comprehensively considers detection path length, camera pose variation, number of corners, and cumulative corner angles. This results in an optimized detection path that maintains full coverage while achieving a shorter path length and more stable camera pose variation.

[0099] The viewpoint traversal path of each area is obtained by hybrid greedy-non-dominated sorting genetic algorithm; the optimization path is calculated to see whether the line connecting two adjacent viewpoints overlaps with the car model. If there is an overlap, it is marked as a collision. Interpolation is required between the two viewpoints to generate a transition viewpoint to avoid the collision. The line of sight of the transition viewpoint is the average of the line of sight of the two viewpoints, and the coordinates are the average of the two viewpoints and then moved along the line of sight by a certain safe distance until the collision is avoided, finally generating a full coverage detection path.

[0100] Example

[0101] The following uses an STL format car model to explain the method of the present invention in detail: The present invention proposes a three-dimensional path planning method for the task of full coverage detection of the outer surface of a car, the method comprising the following steps:

[0102] Step 1: Process the STL car model and extract the surface point cloud for subsequent clustering and partitioning. The STL model is a mesh composed of triangular facets. Extract all vertex data and triangular facet information of the car model. Each triangular facet contains normal vector information and three vertices.

[0103] Step 1.1: Sampling is performed based on vertex data, face information and normal vectors to obtain a car point cloud P consisting of N points = {p1, p2, ..., p N}, where p i =(x i ,y i ,z i ,n ix ,n iy ,n iz ), i=1,2,...,N,p i Contains the coordinate information of the point (x i ,y i ,z i ) and normal vector information n i =(n ix ,n iy ,n iz ).

[0104] Step 1.2: Define a virtual camera in the point cloud environment to extract the outer surface point cloud from the car point cloud P obtained in step 1.1 and define the initial position c of the virtual camera. camera , set the angle threshold θ threshold , calculate each point p in the point cloud within the visible range of the virtual camera at the current position i The vector to the virtual camera and the normal vector n of the point i The angle θ i :

[0105]

[0106] Keep satisfying θ i ≤θ threshold The point cloud of the outer surface under this viewing angle is obtained.

[0107] Step 1.3: Define multiple rotation angles of the virtual camera on each axis of the 3D coordinate system to extract the complete car surface point cloud:

[0108]

[0109] Among them, θmax Indicates the maximum rotation angle, θ step Represents the step size of the rotation angle, and k is an integer index. Repeat step 1.2 to collect the point cloud indexes from all perspectives and remove duplicate indexes. Finally, extract the complete point cloud of the car's outer surface based on the index. The processing flow and results are shown in the following figure. Figure 2 shown.

[0110] Step 2: Cluster the acquired car surface point cloud to segment the main areas such as the car door and roof. Since the car surface has uneven areas with large curvature changes, a two-stage clustering algorithm is used to reasonably segment the points. Clustering is performed based on the distance threshold ε and the normal vector angle threshold θ. The core idea is as follows:

[0111] Stage 1: Points that meet both the distance threshold ε and the normal vector angle threshold θ are grouped into the same cluster.

[0112] Stage 2: Assign unclassified points to the nearest existing cluster based on the condition that only the distance threshold ε is satisfied.

[0113] Step 2.1: Initialize the threshold parameters and calculate the local density. i Calculate its local density ρ i =|KNN(p i )|, where KNN(p i ) is point p i The set of K nearest neighbors of .

[0114] Define the adaptive distance threshold and take the median of the Euclidean distance of neighboring points:

[0115] ε(p i )=median(||p i -p k ||2),p k ∈KNN(p i )

[0116] Define the adaptive normal vector angle threshold, according to point p i Normal vector n i and neighboring point p k Normal vector n k , calculated by combining the mean and standard deviation:

[0117] θ(p i )=mean(cos -1 (n i ·n k ))+std(cos -1 (n i ·n k )),p k =KNN(pi )

[0118] Step 2.2: Clustering in stage 1, determine the core points. If point p i There are several points in the neighborhood of , which satisfy:

[0119] ||p i -p k ||2≤ε(p i ) and cos -1 (n i ·n k )≤θ(p i )

[0120] Then mark p i As the core point, create a new cluster C k Recursively merge all neighboring points that meet the above conditions to complete the region expansion and form the initial cluster.

[0121] Step 2.3: In the second clustering stage, the unclassified points are assigned to the nearest existing cluster according to the condition that only the distance threshold ε is satisfied.

[0122] Step 2.4: Optimize the results of step 2.2 and step 2.3. Perform morphological closing operation on the boundaries of each cluster to eliminate jagged edges. Merge similar clusters. If there are adjacent clusters C a and C b The mean angle of the normal vector is less than the given global threshold θ global , then merge the two clusters:

[0123]

[0124] According to the above steps, the outer surface point cloud of the car can be preliminarily divided to obtain the roof P roof 、Door P door 、Rear P rear 、Car front P front The classification results are as follows: Figure 3 shown.

[0125] Step 3: Sample and generate detection viewpoints for each main area.

[0126] Step 3.1: Rasterize the point cloud data of each area and use the center point of each grid as the representative sampling point, such as Figure 4 To ensure full coverage of the detection, the grid size needs to be defined. The grid size viewsize is related to the image sensor size w×h (w≤h), the camera focal length f, and the shooting distance d. The calculation formula is:

[0127]

[0128] Step 3.2: Downsample each area according to the grid size obtained in step 3.1 to generate viewpoints that meet the full coverage detection requirements of each area. The average of the point cloud normal vectors in each grid is used to obtain the viewpoint's line of sight (normal vector):

[0129]

[0130] Where m is the total number of point clouds in the grid, n j is the normal vector of each point, n i The obtained viewpoint p i The normal vector of .

[0131] The center point of the grid is along the viewpoint line of sight direction n i =(n ix ,n iy ,n iz ) Move the shooting distance d to get the viewpoint p i Location information (x i ,y i ,z i ), each viewpoint p finally obtained i Contains coordinate position information and line of sight (normal vector) information.

[0132] The roof P obtained in step 2 roof 、Door P door 、Rear P rear 、Car front P front The main areas are operated to obtain the detection viewpoint set of each area, roof P roof_view 、Door P door_view 、Rear P rear_view 、Car front P front_view wait.

[0133] Step 4: Segment the internal viewpoints of each area to solve the problem of holes (such as car windows) in the area. Use the detection viewpoint subdivision method based on the 3D scanning plane to obtain the door viewpoint set P in step 3. door_view Take this as an example to illustrate, Figure 5 shown.

[0134] Step 4.1: Define the door_view 3D scanning plane where z max P door view The maximum coordinate value in the z-axis direction. The scanning plane is scanned step by step according to the viewsize set in step 3:

[0135]

[0136] Where k is the index of the three-dimensional scanning plane, and its value range is k=0,1,2,... The termination condition of the scanning plane is z scan <z min , where z min P door_view The minimum coordinate value in the z-axis direction.

[0137] Step 4.2: Perform viewpoint overlap judgment and sorting, and record the viewpoints that overlap with the scan plane as a set The overlapping geometry is defined as the z coordinate value of the viewpoint is located at the current z coordinate of the scan plane scan A given small range around the value.

[0138] Will The specific method of sorting is to sort by x coordinate in ascending order, and then sort by y coordinate in ascending order for the same x coordinate. The sorted overlapping point cloud sequence is represented as where p ki =(x ki ,y ki ,z ki ,n kix ,n kiy ,n kiz ), i = 1, 2, ..., m, m is the number of overlapping point clouds on the scanning plane with the current index k.

[0139] Step 4.3: Calculate and classify the distance between adjacent points. For the sorted overlapping viewpoint sequence, calculate the adjacent point p ki and p ki+1 Distance between:

[0140]

[0141] d ki Whether all the overlapping viewpoints are less than or equal to viewsize is used as the judgment condition. The overlapping viewpoints that meet the conditions are classified into category A and recorded as set P A :

[0142]

[0143] The overlapping viewpoints that do not meet the conditions are classified into category B and are recorded as set P B :

[0144]

[0145] Class A represents the classified areas without holes, and class B represents the classified areas with holes.

[0146] Step 4.4: Use the DBSCAN algorithm to reclassify the A and B viewpoints obtained in step 4.3, and obtain the A viewpoint labeled P.A1 , P A2 ...The segmentation result is P for the viewpoint of type B. B1 , P B2 ...the segmentation results, and finally the segmented detection areas for the car door are obtained, such as Figure 5 shown.

[0147] Step 5: Use a hybrid greedy-non-dominated sorting genetic algorithm to connect the detection viewpoints in the segmented area.

[0148] Step 5.1: Use the improved greedy search algorithm to preliminarily connect the detection viewpoints to form an initial detection path as a high-quality initial solution for subsequent genetic algorithm optimization.

[0149] The improved greedy search algorithm is based on a prescribed neighboring viewpoint scoring strategy. When connecting viewpoints, the algorithm selects the viewpoint with the highest score in the neighborhood that has not been visited according to the scoring strategy. Each viewpoint to be visited is scored based on the three path characteristics of the distance between the currently visited viewpoint and the viewpoint to be visited, the viewpoint's viewing direction, and the path type. The specific diagram is as follows: Figure 6 shown.

[0150] Define the current viewpoint p i and the viewpoint to be visited p i+1 Distance:

[0151]

[0152] Define the viewpoint angle and calculate the current viewpoint normal vector n i and the normal vector n of the viewpoint to be visited i+1 The smaller the angle, the smoother the transition of the detection device:

[0153]

[0154] Define the path type and calculate the previous viewpoint p i-1 To the current viewpoint p i The vector and the current viewpoint p i To the viewpoint p to be visited i+1 The angle between the vectors is:

[0155]

[0156] Define path type function based on angle:

[0157]

[0158] Construct a viewpoint scoring function:

[0159]

[0160] where λ d is the distance weight coefficient, λ α is the viewpoint viewing angle weight coefficient, λ T is the path type weight coefficient, It is an exponential decay term. When the viewpoint angle becomes smaller, the score increases, which strengthens the continuity of small angles and avoids perspective jumps.

[0161] The scores of all the viewpoints to be visited in the neighborhood are obtained according to the scoring function, and the viewpoint with the highest score is selected as the next viewpoint. Multiple edge viewpoints are selected as starting points, and the connection order is generated respectively. The generated solutions are used as high-quality initial solutions for the next step of non-dominated sorting genetic algorithm optimization.

[0162] Step 5.2: Use the non-dominated sorting genetic algorithm (NSGA-II) to optimize the initial solution and generate the Pareto solution set through non-dominated sorting, crowding distance calculation, elite retention strategy, and crossover mutation.

[0163] Determination of non-dominance relationship: stratify the individuals in the population according to the dominance relationship of the objective function, and define that the dominance of individual p to q must satisfy:

[0164] and

[0165] Where m is the number of objective functions, f k is the kth optimization objective value.

[0166] Crowding distance calculation: For individuals i in the same non-dominated layer, calculate their distribution density in the target space:

[0167]

[0168] in is the maximum and minimum value of the kth target.

[0169] Elite retention: From the combined population of parents and offspring, individuals with high non-dominant hierarchy and large crowding distance are preferentially selected.

[0170] Crossover mutation: The crossover method selects high-scoring continuous subpaths from a parent generation and inserts them into the remaining viewpoints of another parent generation to generate offspring. The mutation method is viewpoint position mutation, which randomly changes the position of each viewpoint with a certain mutation probability.

[0171] The multi-objective fitness function is defined as:

[0172]

[0173] Where f1 is the total length of the path, f2 is the smoothness of the detected line of sight, f3 is the cumulative number of turns, and f4 is the cumulative turning angle.

[0174] The viewpoint traversal path of each area is obtained by hybrid greedy-non-dominated sorting genetic algorithm. The optimization path is calculated to see if the line connecting two adjacent viewpoints overlaps with the car model. If there is an overlap, it is marked as a collision. Interpolation is required between the two viewpoints to generate a transition viewpoint to avoid the collision. The line of sight of the transition viewpoint is the average of the line of sight of the two viewpoints. The coordinates are the average of the two viewpoints and then moved along the line of sight by a certain safe distance until the collision is avoided. Finally, the transition viewpoint is generated. Figure 7 Full coverage detection path shown.

[0175] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the three-dimensional path planning method for the task of fully covering the outer surface of an automobile.

[0176] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a three-dimensional path planning method for the task of fully covering the outer surface of an automobile.

[0177] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0178] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0179] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it will not be described in detail here.

[0180] It should be noted that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0181] The above is a detailed introduction to the three-dimensional path planning method for the full coverage detection task of the outer surface of an automobile proposed in the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A three-dimensional path planning method for full coverage inspection of automobile exterior surfaces, characterized in that: The method comprises the following steps: Step 1: Preprocess the car model by extracting the car's exterior surface point cloud from multiple perspectives and performing normal vector sampling calculations to remove noise points inside the car and extract a single-layer point cloud data of the car's exterior surface. Step 2: A two-stage clustering algorithm is used to cluster and partition the car's exterior surface. In the first stage, the point cloud distance threshold and the normal vector angle threshold are dynamically adjusted according to the local density to generate initial clusters. In the second stage, unclassified points are assigned to the nearest cluster according to the distance threshold; the car's exterior surface is divided into several main areas. Step 3: Sample and generate detection viewpoints for each major area. Define the detection plane based on the camera field of view of the detection device. Based on the size of the detection plane, downsample each major area while ensuring complete coverage to obtain the viewpoints used for subsequent photo detection. Step 4: Subdivide the viewpoints of each main area. By defining a 3D scanning detection plane, generate multiple subdivision sub-areas based on the existing holes, reasonably avoid the parts with holes, reduce the length of the subsequent detection path, and ensure that the shape and size of each sub-area are convenient for path planning; Step 5: Use a hybrid greedy-non-dominated sorting genetic algorithm to connect the detection viewpoints to generate the optimal detection path, perform collision detection and transition viewpoint interpolation optimization operations on the optimal detection path to ensure the feasibility and safety of the detection path.

2. The method according to claim 1, characterized in that In the first stage of step 2, points that meet both the distance threshold ε and the normal vector angle threshold θ are grouped into the same cluster. Specifically: initialize the threshold parameters, perform local density calculation, and for each point p i Calculate its local density ρ i =|KNN(p i )|, where KNN(p i ) is point p i The set of K nearest neighbor points of ; Define the distance threshold and take the median of the Euclidean distance of neighboring points: ε(p i )(median(||p i -p k ||2),p k ∈KNN(p i ) Define the normal vector angle threshold, according to point p i Normal vector n i and neighboring point p k Normal vector n k , calculated by combining the mean and standard deviation: θ(p i )=mean(cos -1 (n i ·n k ))+std(cos -1 (n i ·n k )), p k =KNN(p i ) Perform core point determination, if point p i There are several points in the neighborhood of , which satisfy: ||p i -p k ||2≤ε(p i ) and cos -1 (n i ·n k )≤θ(p i ) Then mark p i As the core point, create a new cluster C k ; Recursively merge all neighboring points that meet the above conditions to complete the region expansion and form the initial cluster.

3. The method according to claim 2, characterized in that In the second stage of step 2, the unclassified points are assigned to the nearest existing cluster based on the condition that only the distance threshold ε is satisfied. The clustering results of the first stage and the clustering results of the second stage are optimized as follows: Perform morphological closing operation on the boundaries of each cluster to eliminate jagged edges; merge similar clusters. If there are adjacent clusters C a and C b The normal vector mean angle is less than the given global threshold θ global , then merge the two clusters: According to the above steps, the outer surface point cloud of the car can be preliminarily divided into various main areas.

4. The method according to claim 3, characterized in that The step three is specifically as follows: Step 3.1: Rasterize the point cloud data of each main area and use the center point of each grid as the representative sampling point. The grid size viewsize is related to the image sensor size w×h, w≤h, camera focal length f, and shooting distance d. The calculation formula is: Step 3.2: Downsample each main area according to the grid size obtained in step 3.1 to generate viewpoints that meet the full coverage detection of each main area. The average of the point cloud normal vectors in each grid is the viewpoint's line of sight, i.e., the normal vector: Where m is the total number of point clouds in the grid, n j is the normal vector of each point, n i The obtained viewpoint p i The normal vector of The center point of the grid is along the viewpoint line of sight direction n i =(n ix ,n iy ,n iz ) Move the shooting distance d to get the viewpoint p i Location information (x i ,y i ,z i ), each viewpoint p finally obtained i Contains coordinate location information and line of sight information.

5. The method according to claim 4, characterized in that The step 4 is specifically as follows: Step 4.1: Define the 3D scanning plane for a set of primary region detection viewpoints where z max Detect the maximum coordinate value of the viewpoint set in the z-axis direction for a certain main area; the scanning plane is scanned step by step according to the set viewsize: Where k is the index of the three-dimensional scanning plane, and its value range is k=0,1,2,...; the termination condition of the scanning plane is z scan <z min , where z min Detect the minimum coordinate value of the viewpoint set in the z-axis direction for a certain main area; Step 4.2: Perform viewpoint overlap judgment and sorting, and record the viewpoints that overlap with the scan plane as a set The overlapping geometry is defined as the z coordinate value of the viewpoint is located at the current z coordinate of the scan plane scan within a given small range around the value; Step 4.3: Calculate and classify the distance between adjacent points. For the sorted overlapping viewpoint sequence, calculate the adjacent point p ki and p ki+1 Distance between: d ki Whether all the overlapping viewpoints are less than or equal to viewsize is used as the judgment condition; the overlapping viewpoints that meet the conditions are classified into category A and recorded as set P A : The overlapping viewpoints that do not meet the conditions are classified into category B and are recorded as set P B : Class A represents the classified areas without holes, and Class B represents the classified areas with holes; Step 4.4: Use the DBSCAN algorithm to reclassify the A and B viewpoints obtained in step 4.3, and obtain the A viewpoint labeled P. A1 , P A2 ...The segmentation result is P for the viewpoint of type B. B1 , P B2 ...the segmentation results, and finally obtain the various segmented detection areas divided for a certain main area.

6. The method according to claim 5, characterized in that In step 4.2, The specific method of sorting is to sort all viewpoints in ascending order by x coordinates first, and then sort them in ascending order by y coordinates for the same x coordinates; the sorted overlapping point cloud sequence is expressed as where p ki =(x ki ,y ki ,z ki ,n kix ,n kiy ,n kiz ), i = 1, 2, ..., m, m is the number of overlapping point clouds on the scanning plane with the current index k.

7. The method according to claim 6, characterized in that In step five, a greedy search algorithm is used to preliminarily connect the determined detection viewpoints to form an initial detection path. The initial detection path obtained by the greedy search algorithm is used as a high-quality initial solution of the non-dominated sorting genetic algorithm NSGA-II, and the initial solution is optimized using the non-dominated sorting genetic algorithm. Through selection, crossover and mutation operations, continuous iterative evolution is carried out, and the multi-objective fitness function is used as the evaluation criterion to find the optimal detection path.

8. The method according to claim 7, characterized in that The viewpoint traversal path of each area is obtained by hybrid greedy-non-dominated sorting genetic algorithm; the optimization path is calculated to see whether the line connecting two adjacent viewpoints overlaps with the car model. If there is an overlap, it is marked as a collision. Interpolation is required between the two viewpoints to generate a transition viewpoint to avoid the collision. The line of sight of the transition viewpoint is the average of the line of sight of the two viewpoints, and the coordinates are the average of the two viewpoints and then moved along the line of sight by a certain safe distance until the collision is avoided, finally generating a full coverage detection path.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images