A photographing point generation method and device for appearance detection, a medium and equipment

Abstract

This application discloses a method, apparatus, medium, and device for generating photo points for appearance inspection. The method includes: acquiring an initial point cloud containing several surface points corresponding to an object to be inspected; selecting a target point cloud corresponding to the area to be inspected from the initial point cloud; performing point cloud segmentation on each surface point in the target point cloud using a region growing algorithm to obtain several clustered point clouds; and generating several target photo points corresponding to each clustered point cloud based on each surface point in the same clustered point cloud and the shooting parameters of the target camera. This application, by using a region growing algorithm to segment the surface points in the target point cloud, can quickly and accurately obtain several clustered point clouds. Subsequently, target photo points corresponding to each clustered point cloud can be reasonably and accurately generated based on each surface point in the same clustered point cloud, reducing the number of images taken and improving the efficiency of appearance inspection.

Description

Technical Field

[0001] This invention relates to the field of appearance inspection technology, and in particular to a method, apparatus, medium and equipment for generating photographic points for appearance inspection. Background Technology

[0002] In the product manufacturing process, visual inspection is a crucial step in quality control. By conducting visual inspections, surface defects can be detected promptly, ensuring product quality and performance.

[0003] In existing appearance inspection methods, multiple area scan cameras are typically used to photograph the product. The position and orientation of the area scan cameras are adjusted using the position of the area scan camera bracket to achieve multi-angle shooting. However, this method causes redundancy of line scan cameras and images, resulting in a cumbersome inspection process and low inspection efficiency. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus, medium and device for generating photo points for appearance inspection, the main purpose of which is to solve the problem of low inspection efficiency caused by redundant area scan camera settings and redundant images in the current appearance inspection process.

[0005] To address the aforementioned problems, this application provides a method for generating photographic points for appearance inspection, comprising:

[0006] Obtain an initial point cloud containing several surface points corresponding to the object to be detected;

[0007] Based on the region to be detected, a target point cloud corresponding to the region to be detected is obtained from the initial point cloud;

[0008] The target point cloud is segmented based on the region growing algorithm to obtain several clustered point clouds;

[0009] Based on the surface points in the same clustered point cloud and the shooting parameters of the target camera, several target shooting points corresponding to each clustered point cloud are generated.

[0010] Optionally, obtaining an initial point cloud containing a plurality of surface points corresponding to the object to be detected specifically includes:

[0011] Obtain the 3D model of the object to be inspected;

[0012] The three-dimensional model is converted to a new format to obtain an initial point cloud containing several surface points corresponding to the object to be detected.

[0013] Optionally, the step of segmenting the surface points in the target point cloud using a region growing algorithm to obtain several clustered point clouds specifically includes:

[0014] Step 1: Determine the initial seed point from the target point cloud;

[0015] Step 2: Based on the angle between the normals of the initial seed point and each remaining surface point, filter each remaining surface point to obtain a number of first surface points corresponding to the initial seed point;

[0016] Step 3: Determine whether there are any surface points among the first surface points that meet the predetermined update conditions. If there are surface points that meet the predetermined update conditions, use the surface points that meet the predetermined update conditions as initial seed points in sequence, and repeat Step 2 until none of the first surface points meet the predetermined update conditions. Then, use the initial seed points and each first surface point as the cluster point cloud corresponding to the initial seed points.

[0017] Step 4: Based on the number of remaining seed points after point cloud segmentation, determine whether the point cloud segmentation stopping condition is met; if the point cloud segmentation stopping condition is not met, determine the initial seed points from the remaining seed points after point cloud segmentation, and repeat Step 2 until the point cloud segmentation stopping condition is met, and obtain several clustered point clouds.

[0018] Optionally, before filtering the remaining surface points based on the angle between the normals of the initial seed point and each remaining surface point, the method further includes:

[0019] Based on the unit normal vector of the initial seed point and the unit normal vectors of each remaining surface point, calculate the angle between the normal vectors of the initial seed point and each remaining surface point.

[0020] The step of filtering the remaining surface points based on the angle between the normals of the initial seed point and each remaining surface point to obtain a number of first surface points corresponding to the initial seed point specifically includes:

[0021] The included angle of the normals at each remaining surface point is compared with a predetermined included angle threshold.

[0022] The remaining surface points corresponding to the included angles of the normals that are less than the included angle threshold are determined as the first surface points, so as to obtain a number of first surface points corresponding to the initial seed point.

[0023] Optionally, the step of generating several target image points corresponding to each cluster of point clouds based on the surface points in the same cluster of point clouds and the shooting parameters of the target camera specifically includes:

[0024] Based on the surface points in the same clustered point cloud, determine the bounding box corresponding to each clustered point cloud;

[0025] The bottom surface of each bounding box is divided to obtain a number of rectangular blocks corresponding to each bounding box;

[0026] Based on the center of each rectangular block, the shooting parameters of the target camera, and the direction vector of the bounding box height corresponding to the rectangular block, an initial shooting point corresponding to each rectangular block is generated;

[0027] The initial photo points are adjusted to obtain target photo points corresponding to each initial photo point, thereby generating several target photo points corresponding to each clustered point cloud.

[0028] Optionally, adjusting each of the initial photo-taking points to obtain a target photo-taking point corresponding to each of the initial photo-taking points specifically includes:

[0029] Based on each of the initial photo-taking points, several candidate photo-taking points corresponding to each initial photo-taking point are obtained through adjustment.

[0030] Determine the tilt angle of the area to be detected;

[0031] When the tilt angle is less than or equal to a predetermined tilt angle threshold, the distance between each candidate photographing point and the center point of the workpiece to be inspected is determined, and the candidate photographing point farthest from the center point of the workpiece to be inspected is determined from each candidate photographing point corresponding to the same initial photographing point as the target photographing point, so as to obtain the target photographing point corresponding to each initial photographing point.

[0032] When the tilt angle is greater than a predetermined tilt angle threshold, the shooting height of each candidate shooting point is determined, and the candidate shooting point with the highest shooting height is determined from the candidate shooting points corresponding to the same initial shooting point as the target shooting point, so as to obtain the target shooting point corresponding to each initial shooting point.

[0033] Optionally, after adjusting and obtaining several candidate photo-taking points corresponding to each initial photo-taking point based on each initial photo-taking point, the method further includes:

[0034] For each candidate photo location, a light projection algorithm is used to select unobstructed candidate photo locations.

[0035] To address the aforementioned problems, this application provides a device for generating photographic points for appearance inspection, comprising:

[0036] The acquisition module is used to acquire an initial point cloud containing several surface points corresponding to the workpiece object to be inspected;

[0037] The filtering module is used to filter out target point clouds corresponding to the detection area from the initial point cloud based on the detection area of ​​the workpiece to be inspected;

[0038] The point cloud segmentation module is used to segment each surface point in the target point cloud based on a region growing algorithm to obtain several clustered point clouds.

[0039] The generation module is used to generate several target image points corresponding to each cluster of point clouds based on the surface points in the same cluster of point clouds and the shooting parameters of the target camera.

[0040] To address the aforementioned problems, this application provides a storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for generating image points for appearance inspection.

[0041] To address the aforementioned problems, this application provides an electronic device, comprising at least a memory and a processor. The memory stores a computer program, and the processor, when executing the computer program in the memory, implements the steps of the above-described method for generating image points for appearance inspection.

[0042] The method, apparatus, medium, and equipment for generating photo points for appearance inspection disclosed in this application determine the area to be inspected, obtain the target point cloud corresponding to the area to be inspected, and then further segment the point cloud by using a region growing algorithm on each surface point in the target point cloud. This can quickly and accurately obtain several clustered point clouds. Subsequently, based on each surface point in the same clustered point cloud, target photo points corresponding to each clustered point cloud can be generated reasonably and accurately. Then, area scan cameras can be used directly to take pictures of the object to be inspected based on each photo point, eliminating the need to set up a large number of area scan cameras, reducing the number of pictures taken, and improving the efficiency of appearance inspection.

[0043] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0044] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0045] Figure 1This is a flowchart illustrating a method for generating photo points for appearance inspection according to an embodiment of this application;

[0046] Figure 2 This is a flowchart of the point cloud segmentation process in an embodiment of this application.

[0047] Figure 3 This is a structural block diagram of a photo point generation device for appearance inspection, according to another embodiment of this application.

[0048] Figure 4 This is a structural block diagram of an electronic device according to another embodiment of this application. Detailed Implementation

[0049] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0050] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0051] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0052] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0053] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.

[0054] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0055] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0056] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0057] This application provides a method for generating photo capture points for appearance inspection, such as... Figure 1 As shown, it includes the following steps:

[0058] Step S101: Obtain an initial point cloud containing several surface points corresponding to the object to be detected;

[0059] In this step, the object to be inspected can be a workpiece or a product. In practice, a 3D model of the object to be inspected can be obtained first; then, the 3D model is converted to a new format to obtain an initial point cloud containing several surface points corresponding to the object to be inspected. Since modern factories use 3D models to produce actual workpieces, there will be original 3D model files for both the object to be inspected and the workpiece to be inspected. Alternatively, a corresponding 3D model can be pre-created for the object to be inspected to obtain its 3D model.

[0060] In this step, when converting the 3D model, Poisson sampling can be used to convert the 3D model into a point cloud format, thereby obtaining the 3D model of the point cloud domain, that is, obtaining the initial point cloud.

[0061] Step S102: Based on the region to be detected, select the target point cloud corresponding to the region to be detected from the initial point cloud;

[0062] In this step, the target point cloud that can be captured from the initial point cloud can be selected based on the camera's shooting position. Specifically, a hidden point removal algorithm can be used to simulate the camera's shooting position to remove point cloud areas that cannot be captured from the object under inspection at that shooting position, thereby selecting the target point cloud that can be captured from that shooting position. Taking the shooting position as being directly above the object / workpiece under inspection as an example, the point cloud area below the object under inspection that cannot be captured can be removed from the initial point cloud, thus obtaining the target point cloud of the upper surface of the workpiece. The area where this target point cloud is located is the area whose appearance should be inspected by the quality inspection system.

[0063] Step S103: Based on the region growing algorithm, perform point cloud segmentation on each surface point in the target point cloud to obtain several clustered point clouds;

[0064] In the specific implementation of this step, a region growing algorithm can be used to segment the target point cloud of the area to be detected, that is, to cluster the target point cloud to group surface points with similar normal vectors into one class, thereby obtaining several clustered point clouds. In this embodiment, the number of clusters is related to the complexity of the object / workpiece to be detected; the more complex the surface of the object to be detected, the more clusters are required.

[0065] Step S104: Based on the surface points in the same clustered point cloud and the shooting parameters of the target camera, generate several target shooting points corresponding to each clustered point cloud.

[0066] In the specific implementation process of this step, based on the bounding box formed by each surface point in the same clustered point cloud, the camera field of view width, and the camera field of view height, several field of view regions corresponding to each clustered point cloud and matching the camera field of view are determined, that is, several rectangular blocks corresponding to each clustered point cloud are obtained; then, based on the predetermined shooting distance and the center coordinates of each rectangular block / field of view region, the camera pose coordinates corresponding to each rectangular block / field of view region are determined to obtain the shooting point corresponding to each rectangular block, thereby obtaining the target shooting point.

[0067] The image point generation method for appearance inspection in this embodiment determines the area to be inspected, obtains the target point cloud corresponding to the area to be inspected, and then further segments the point cloud by using a region growing algorithm on each surface point in the target point cloud. This can quickly and accurately obtain several clustered point clouds. Subsequently, based on each surface point in the same clustered point cloud, target image points corresponding to each clustered point cloud can be generated reasonably and accurately. Then, the area scan camera can be used directly to take pictures of the object to be inspected based on each image point, without the need to set up a large number of area scan cameras, reducing the number of pictures taken and improving the efficiency of appearance inspection.

[0068] Another embodiment of this application provides a method for generating image points for appearance inspection. In this embodiment, when performing step S103, which involves segmenting the surface points in the target point cloud based on a region growing algorithm to obtain several clustered point clouds, the point cloud segmentation process is as follows:

[0069] Step S1031: Determine the initial seed point from the target point cloud;

[0070] In this step, during the specific implementation process, the unit normal vector (normal) and curvature (curvature) of each surface point to be segmented in the target point cloud can be calculated first; then, an initial seed point is selected based on the curvature. That is, the surface points are sorted in ascending order based on the magnitude of curvature, and then the surface point with the lowest curvature is selected as the initial seed point. Of course, the surface point with the highest curvature can also be selected as the initial seed point.

[0071] Step S1032: Based on the angle between the normals of the initial seed point and each remaining surface point, the remaining surface points are filtered to obtain a number of first surface points corresponding to the initial seed point.

[0072] In this step, before filtering the remaining surface points, the angle between the normals of the initial seed point and each remaining surface point can be calculated based on the unit normal vector of the initial seed point and the unit normal vectors of each remaining surface point. Then, the angle between the normals of each remaining surface point is compared with a predetermined angle threshold; the remaining surface points with angles smaller than the threshold are identified as first surface points, thus obtaining several first surface points corresponding to the initial seed point. In order to reduce computation, this step can be further refined by calculating only the angle between the normals of the initial seed point and each remaining surface point in its neighborhood to determine whether each neighboring point (the remaining surface point in the neighborhood) has a similar normal vector to the initial seed point.

[0073] Step S1033: Determine whether there are surface points among the first surface points that meet the predetermined update conditions. If there are surface points that meet the predetermined update conditions, sequentially use the surface points that meet the predetermined update conditions as initial seed points, and repeat step S1032 until none of the first surface points meet the predetermined update conditions. Then, use the initial seed points and each first surface point as the cluster point cloud corresponding to the initial seed points.

[0074] In this step, after obtaining several first surface points, the initial seed points can be further updated based on each first surface point to continue region growth based on the updated seed points. That is, after obtaining several first surface points, a curvature threshold can be set based on the curvature of the initial seed points; it is determined whether the curvature of each first surface point is less than the curvature threshold, so that if the curvature of any first surface point is less than the curvature threshold, it is determined that there are surface points among the first surface points that meet the predetermined update conditions. Thus, each first surface point with a curvature less than the curvature threshold can be determined as a surface point that meets the predetermined update conditions, and each first surface point that meets the predetermined update conditions can be determined as an initial seed point in sequence, and step S1032 is re-executed to continue region growth based on the updated initial seed points. This process is repeated until each first surface point no longer meets the predetermined update conditions, at which point a region growth is completed, and the initial seed point and each first surface point are used as the cluster point cloud corresponding to the initial seed point.

[0075] Step S1034: Based on the number of remaining seed points after point cloud segmentation, determine whether the point cloud segmentation stopping condition is met; if the point cloud segmentation stopping condition is not met, determine the initial seed points from the remaining seed points after point cloud segmentation, and repeat step S1032 until the point cloud segmentation stopping condition is met, and obtain several clustered point clouds.

[0076] In this embodiment, the principle of the region growing algorithm is as follows: First, a seed point is found for each point cloud that needs to be segmented as the starting point for growth. Then, points in the neighborhood of the seed point that have the same or similar properties as the seed point are merged into the region where the seed point is located. New points continue to grow outwards as seed points until no more points meeting the conditions can be included. At this point, a region is grown. Specifically, the point cloud segmentation process in this embodiment will be further explained and illustrated below in conjunction with specific application scenarios, such as... Figure 2 As shown, the point cloud segmentation process in this embodiment includes:

[0077] Step 1: Begin by inputting the target point cloud, then calculate the normal and curvature, and sort them in ascending order of curvature.

[0078] Step 2: Select the point with the lowest curvature as the initial seed point, and compare the neighboring surface points around the initial seed point with the initial seed point;

[0079] Step 3: Set the normal angle threshold, search for the neighboring points of the current seed point, calculate the angle between the normal of the neighboring points and the normal of the current seed point, and determine whether the neighboring points (nearby surface points) are less than the predetermined angle threshold.

[0080] Step 4: Set a curvature threshold and determine whether the curvature of neighboring points is small enough (the surface is at the same degree of curvature). That is, check the curvature of each neighboring point and determine whether the curvature of the neighboring points is less than the curvature threshold.

[0081] Step 5: If a neighboring point (surface point) satisfies both Step 3 and Step 4, then classify the surface point and add it to the seed point sequence for use in the next round of region growth.

[0082] Step 6: If a surface point only satisfies the conditions of Step 3, then classify the surface point, i.e. add it to the class of seed points, instead of making it a seed point.

[0083] Step 7: Repeat the above growth process until the seed point sequence is cleared. At this point, a region has been grown and added to the clustering array to obtain a clustered point cloud.

[0084] Step 8: Repeat the above steps for the remaining points until all points have been traversed, and obtain several clustered point clouds.

[0085] In other words, during the point cloud segmentation process described above, the first criterion is whether the included angle threshold is met; the second criterion is whether the curvature threshold is met; and the third criterion is whether the seed sequence is empty. Therefore, after selecting a seed point and its neighboring points, the process first checks if the neighboring points are smaller than the included angle threshold. If the neighboring points are greater than or equal to the included angle threshold (i.e., the criterion is not met), the process further checks if the seed sequence is empty. If the neighboring points are smaller than the included angle threshold (i.e., the first criterion is met), the process further checks if the neighboring points are smaller than the curvature threshold. If they are smaller than the curvature threshold (i.e., the second criterion is met), the point is classified and added to the seed point sequence for use in the next round of region growing. If the point is greater than or equal to the curvature threshold (i.e., the second criterion is not met), the point is only added to the seed point class but is not used as a seed point.

[0086] In this embodiment, by using a region growing algorithm to segment the point cloud of each surface point, the point cloud segmentation results can be made more reasonable and accurate, laying the foundation for the subsequent reasonable calculation of the corresponding target photography points based on the segmented point clouds.

[0087] Another embodiment of this application provides a method for generating photo points for appearance inspection. In this embodiment, when performing step S104 to generate several target photo points corresponding to each cluster point cloud, the following generation method can be adopted:

[0088] Step S1041: Based on each surface point in the same clustered point cloud, determine the bounding box corresponding to each clustered point cloud;

[0089] In this step, the OBB bounding box (orientation bounding box) of each clustered point cloud can be calculated, resulting in a rectangular box with length, width, and height of Length, Width, and Height, respectively. The bounding box can encompass all surface points within the same clustered point cloud. Simultaneously, the three principal orientation boxes of each bounding box can be obtained. L box W box H These represent the direction vectors of the long side, wide side, and height of the current bounding box in the world coordinate system, respectively. Each principal direction vector is a 3*1 vector representing the components of each direction in the world coordinate system: box L =(X world_x X world_y X world_z ).

[0090] Step S1042: Divide the bottom surface of each bounding box to obtain a number of rectangular blocks corresponding to each bounding box;

[0091] In this step, the bottom surface of each bounding box can be divided using the camera's field of view width (fov_width) and height (fov_height), thus obtaining W*H rectangular blocks corresponding to each bounding box. That is, given the length and width of the square bounding box surface, the surface is divided into W*H smaller rectangular blocks, where... W represents the minimum number of camera field of view widths required to cover the width of the shooting rectangle. H represents the minimum number of camera view heights required to cover the height of the captured rectangle. The coordinates of each small rectangle can be obtained by combining the coordinates of any vertex with the three principal directions of the bounding box.

[0092] Step S1043: Based on the center of each rectangular block, the shooting parameters of the target camera, and the direction vector of the bounding box height corresponding to the rectangular block, generate the initial shooting point corresponding to each rectangular block;

[0093] In this step, for each rectangular block, the coordinates of its center can be obtained as (a...). i ,b i ,c i From this, we can obtain the initial photographing point corresponding to the rectangular block as (a i ,b i ,c i +working_distance*box H The camera is located at the center of each small rectangular block, according to the bounding box. H (Height direction) Move upwards to the specified working distance, with the lens facing the box. H The directions are opposite (in the height direction). Therefore, the complete camera pose at each position can be obtained.

[0094] Step S1044: Adjust each of the initial photo-taking points to obtain target photo-taking points corresponding to each of the initial photo-taking points, so as to generate a number of target photo-taking points corresponding to each clustered point cloud.

[0095] In this step, because shooting perpendicularly to the normal vector is not very effective for capturing certain defects, after obtaining the initial shooting points, it is necessary to adjust the shooting angle to obtain the target shooting points. The specific adjustment process is as follows:

[0096] Step S10441: Based on each of the initial photo-taking points, adjust and obtain a number of candidate photo-taking points corresponding to each initial photo-taking point;

[0097] In this step, for each initial photo-taking point, based on the input angle requirements, the initial photo-taking point can be tilted forward and backward at fixed angles around the X-axis and Y-axis, respectively, to obtain 4 candidate photo-taking points.

[0098] The rotation matrix R is the rotation of the object around the X-axis by an angle α counterclockwise. x :

[0099] The rotation matrix R is the rotation of the object around the Y-axis by an angle α counterclockwise. y :

[0100] Step S10442: Determine the tilt angle of the area to be detected;

[0101] In this step, the average normal vector (n) of all surface points within the area to be detected can be calculated. x ,n y ,n z Then calculate the average normal vector (n) x ,n y ,n z The angle θ between the vertical upward normal vectors of (0, 0, 1) and (0, 0, 1) is used to obtain the tilt angle, and the angle θ is compared with a pre-set plane threshold.

[0102] In this step, the formula for calculating the included angle θ is: θ = cos -1 ((nx×0)+(ny×0)+(nz×1))=cos -1 nz.

[0103] Step S10443: When the tilt angle is less than or equal to a predetermined tilt angle threshold, determine the distance between each candidate photographing point and the center point of the workpiece object to be inspected, and determine the candidate photographing point farthest from the center point of the workpiece object to be inspected from each candidate photographing point corresponding to the same initial photographing point as the target photographing point, so as to obtain the target photographing point corresponding to each initial photographing point.

[0104] In this step, when the tilt angle / angle θ is less than or equal to the predetermined tilt angle threshold, it indicates that the plane is relatively horizontal. Based on experience, it is preferable to take pictures from the outside of the product inward to ensure the safety of the robotic arm. Therefore, the distance between each candidate image point and the center point of the workpiece to be inspected can be calculated. Then, from the candidate image points corresponding to the same initial image point, the candidate image point with the greater distance is selected as the target image point.

[0105] In this step, the candidate photo location p i The center point μ of the workpiece object to be inspected j The formula for calculating the distance between them is: Among them, (xi y i , z i ) indicates the selected photo location p i The coordinates, (x j y j , z j ) represents the center point μ of the workpiece object. j The coordinates.

[0106] Step S10444: When the tilt angle is greater than a predetermined tilt angle threshold, determine the shooting height of each candidate shooting point, and determine the candidate shooting point with the highest shooting height from each candidate shooting point corresponding to the same initial shooting point as the target shooting point, so as to obtain the target shooting point corresponding to each initial shooting point.

[0107] In this step, when the tilt angle / angle θ is greater than the predetermined tilt angle threshold, it indicates that the plane may be tilted. Based on experience, it is preferable to photograph the product from top to bottom to ensure the safety of the robotic arm's posture. Therefore, the shooting height of each candidate shooting point can be calculated. Then, from all candidate shooting points corresponding to the same initial shooting point, the candidate shooting point with the highest shooting height is determined as the target shooting point.

[0108] In this embodiment, after adjusting and obtaining several candidate shooting points corresponding to each initial shooting point based on the initial shooting points, a ray casting algorithm can be used to filter out unobstructed candidate shooting points from each candidate shooting point. That is, for each candidate shooting point, the ray casting algorithm is used to determine whether each candidate shooting point will be obstructed, so as to retain unobstructed candidate shooting points. Then, the target shooting point is determined from the unobstructed candidate shooting points, thereby making the acquisition of the target shooting point more reasonable and accurate.

[0109] Another embodiment of this application provides a device for generating photo locations for appearance inspection, such as... Figure 3 As shown, it includes:

[0110] The acquisition module 11 is used to acquire an initial point cloud containing several surface points corresponding to the workpiece object to be inspected;

[0111] The filtering module 12 is used to filter and obtain a target point cloud corresponding to the detection area from the initial point cloud based on the detection area of ​​the workpiece to be detected.

[0112] Point cloud segmentation module 13 is used to segment each surface point in the target point cloud based on a region growing algorithm to obtain several clustered point clouds;

[0113] The generation module 14 is used to generate several target image points corresponding to each cluster point cloud based on the surface points in the same cluster point cloud and the shooting parameters of the target camera.

[0114] In this embodiment, the acquisition module is specifically used to: acquire a three-dimensional model of the object to be detected; and convert the format of the three-dimensional model to obtain the initial point cloud, which contains several surface points corresponding to the object to be detected.

[0115] In the specific implementation of this embodiment, the point cloud segmentation module is specifically used for: Step 1, determining the initial seed point from the target point cloud;

[0116] Step 2: Based on the angle between the normals of the initial seed point and each remaining surface point, filter each remaining surface point to obtain a number of first surface points corresponding to the initial seed point;

[0117] Step 3: Determine whether there are any surface points among the first surface points that meet the predetermined update conditions. If there are surface points that meet the predetermined update conditions, use the surface points that meet the predetermined update conditions as initial seed points in sequence, and repeat Step 2 until none of the first surface points meet the predetermined update conditions. Then, use the initial seed points and each first surface point as the cluster point cloud corresponding to the initial seed points.

[0118] Step 4: Based on the number of remaining seed points after point cloud segmentation, determine whether the point cloud segmentation stopping condition is met; if the point cloud segmentation stopping condition is not met, determine the initial seed points from the remaining seed points after point cloud segmentation, and repeat Step 2 until the point cloud segmentation stopping condition is met, and obtain several clustered point clouds.

[0119] In this embodiment, the photo point generation device for appearance inspection further includes a calculation module. The calculation module is used to: calculate the angle between the normal of the initial seed point and each remaining surface point based on the unit normal vector of the initial seed point and the unit normal vector of each remaining surface point before filtering each remaining surface point based on the normal angle between the initial seed point and each remaining surface point.

[0120] The point cloud segmentation module is specifically used to: compare the included angle of the normal of each remaining surface point with a predetermined included angle threshold; determine the remaining surface points corresponding to the included angle of the normal that is less than the included angle threshold as first surface points, so as to obtain a number of first surface points corresponding to the initial seed point.

[0121] In this embodiment, the generation module is specifically used to: determine the bounding box corresponding to each cluster point cloud based on each surface point in the same cluster point cloud; divide the bottom surface of each bounding box to obtain a number of rectangular blocks corresponding to each bounding box; generate initial image points corresponding to each rectangular block based on the center of each rectangular block, the shooting parameters of the target camera, and the direction vector of the height of the bounding box corresponding to the rectangular block; adjust each initial image point to obtain target image points corresponding to each initial image point, so as to generate a number of target image points corresponding to each cluster point cloud.

[0122] In this embodiment, the generation module is specifically used for: adjusting and obtaining a number of candidate photography points corresponding to each initial photography point based on each initial photography point; determining the tilt angle of the area to be detected; when the tilt angle is less than or equal to a predetermined tilt angle threshold, determining the distance between each candidate photography point and the center point of the workpiece to be detected, and determining the candidate photography point farthest from the center point of the workpiece to be detected from the candidate photography points corresponding to the same initial photography point as the target photography point, so as to obtain the target photography point corresponding to each initial photography point; when the tilt angle is greater than the predetermined tilt angle threshold, determining the shooting height of each candidate photography point, and determining the candidate photography point with the highest shooting height from the candidate photography points corresponding to the same initial photography point as the target photography point, so as to obtain the target photography point corresponding to each initial photography point.

[0123] In this embodiment, the device for generating photo points for appearance inspection further includes a determination module. The determination module is used to: determine whether each candidate photo point will be occluded using a light projection algorithm, so as to retain the unoccluded candidate photo points.

[0124] The image point generation device for appearance inspection in this embodiment determines the area to be inspected, obtains the target point cloud corresponding to the area to be inspected, and then further segments the point cloud by using a region growing algorithm on each surface point in the target point cloud. This can quickly and accurately obtain several clustered point clouds. Subsequently, based on each surface point in the same clustered point cloud, target image points corresponding to each clustered point cloud can be generated reasonably and accurately. Then, the area scan camera can be used directly to take pictures of the object to be inspected based on each image point, without the need to set up a large number of area scan cameras, reducing the number of pictures taken and improving the efficiency of appearance inspection.

[0125] Another embodiment of this application provides a storage medium storing a computer program, which, when executed by a processor, implements the following method steps:

[0126] Step 1: Obtain an initial point cloud containing several surface points corresponding to the object to be detected;

[0127] Step 2: Based on the region to be detected, select the target point cloud corresponding to the region to be detected from the initial point cloud;

[0128] Step 3: Perform point cloud segmentation on each surface point in the target point cloud based on the region growing algorithm to obtain several clustered point clouds;

[0129] Step 4: Based on the surface points in the same clustered point cloud and the shooting parameters of the target camera, generate several target shooting points corresponding to each clustered point cloud.

[0130] The specific implementation process of the above method steps can be found in the embodiments of the above-mentioned method for generating photo points for appearance inspection, which will not be repeated here.

[0131] The storage medium in this application determines the area to be detected, obtains the target point cloud corresponding to the area to be detected, and then further segments the point cloud by using a region growing algorithm on each surface point in the target point cloud. This can quickly and accurately obtain several clustered point clouds. Subsequently, based on each surface point in the same clustered point cloud, target image points corresponding to each clustered point cloud can be generated reasonably and accurately. Then, the area scan camera can be used directly to take pictures of the object to be detected based on each image point, without the need to set up a large number of area scan cameras, reducing the number of pictures taken and improving the efficiency of appearance detection.

[0132] Another embodiment of this application provides an electronic device, such as... Figure 4 As shown, it includes at least a memory 1 and a processor 2. The memory 1 stores a computer program, and the processor 2 performs the following method steps when executing the computer program in the memory 1:

[0133] Step 1: Obtain an initial point cloud containing several surface points corresponding to the object to be detected;

[0134] Step 2: Based on the region to be detected, select the target point cloud corresponding to the region to be detected from the initial point cloud;

[0135] Step 3: Perform point cloud segmentation on each surface point in the target point cloud based on the region growing algorithm to obtain several clustered point clouds;

[0136] Step 4: Based on the surface points in the same clustered point cloud and the shooting parameters of the target camera, generate several target shooting points corresponding to each clustered point cloud.

[0137] The specific implementation process of the above method steps can be found in the embodiments of the above-mentioned method for generating photo points for appearance inspection, which will not be repeated here.

[0138] The electronic device in this application determines the area to be detected, obtains the target point cloud corresponding to the area to be detected, and then further segments the point cloud by using a region growing algorithm on each surface point in the target point cloud. This can quickly and accurately obtain several clustered point clouds. Subsequently, based on each surface point in the same clustered point cloud, target image points corresponding to each clustered point cloud can be generated reasonably and accurately. Then, the area scan camera can be used directly to take pictures of the object to be detected based on each image point, eliminating the need to set up a large number of area scan cameras, reducing the number of pictures taken, and improving the efficiency of appearance detection.

[0139] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A method for generating photo points for appearance inspection, characterized in that, include: Obtain an initial point cloud containing several surface points corresponding to the object to be detected; Based on the detection area of ​​the object to be detected, a target point cloud corresponding to the detection area is obtained from the initial point cloud; The target point cloud is segmented based on the region growing algorithm to obtain several clustered point clouds; Based on the surface points in the same clustered point cloud, determine the bounding box corresponding to each clustered point cloud; The bottom surface of each bounding box is divided to obtain a number of rectangular blocks corresponding to each bounding box; Based on the center of each rectangular block, the shooting parameters of the target camera, and the direction vector of the bounding box height corresponding to the rectangular block, an initial shooting point corresponding to each rectangular block is generated; Based on each of the initial photo-taking points, several candidate photo-taking points corresponding to each initial photo-taking point are obtained through adjustment. For each candidate photo location, a light projection algorithm is used to select unobstructed candidate photo locations. Determine the tilt angle of the area to be detected; When the tilt angle is less than or equal to a predetermined tilt angle threshold, the distance between each unobstructed candidate shooting point and the center point of the object to be detected is determined, and the candidate shooting point farthest from the center point of the object to be detected is determined from each unobstructed candidate shooting point corresponding to the same initial shooting point as the target shooting point, so as to obtain the target shooting point corresponding to each initial shooting point. When the tilt angle is greater than a predetermined tilt angle threshold, the shooting height of each unobstructed candidate shooting point is determined, and the candidate shooting point with the highest shooting height is determined from each unobstructed candidate shooting point corresponding to the same initial shooting point as the target shooting point, so as to obtain the target shooting point corresponding to each initial shooting point. Based on the target shooting points corresponding to each of the initial shooting points, generate a number of target shooting points corresponding to each of the clustered point clouds.

2. The method as described in claim 1, characterized in that, The acquisition of an initial point cloud containing several surface points corresponding to the object to be detected specifically includes: Obtain the 3D model of the object to be inspected; The three-dimensional model is converted to a new format to obtain an initial point cloud containing several surface points corresponding to the object to be detected.

3. The method as described in claim 1, characterized in that, The point cloud segmentation of the target point cloud based on the region growing algorithm to obtain several clustered point clouds specifically includes: Step 1: Determine the initial seed point from the target point cloud; Step 2: Based on the angle between the normals of the initial seed point and each remaining surface point, filter each remaining surface point to obtain a number of first surface points corresponding to the initial seed point; Step 3: Determine whether there are any surface points among the first surface points that meet the predetermined update conditions. If there are surface points that meet the predetermined update conditions, use the surface points that meet the predetermined update conditions as initial seed points in sequence, and repeat Step 2 until none of the first surface points meet the predetermined update conditions. Then, use the initial seed points and each first surface point as the cluster point cloud corresponding to the initial seed points. Step 4: Based on the number of remaining surface points after point cloud segmentation, determine whether the point cloud segmentation stopping condition is met; if the point cloud segmentation stopping condition is not met, determine the initial seed points from the remaining surface points after point cloud segmentation, and repeat Step 2 until the point cloud segmentation stopping condition is met, and obtain several clustered point clouds.

4. The method as described in claim 3, characterized in that, Before filtering the remaining surface points based on the angle between the normals of the initial seed point and each remaining surface point, the method further includes: Based on the unit normal vector of the initial seed point and the unit normal vectors of each remaining surface point, calculate the angle between the normal vectors of the initial seed point and each remaining surface point. The step of filtering the remaining surface points based on the angle between the normals of the initial seed point and each remaining surface point to obtain a number of first surface points corresponding to the initial seed point specifically includes: The included angle of the normals at each remaining surface point is compared with a predetermined included angle threshold. The remaining surface points corresponding to the included angles of the normals that are less than the included angle threshold are determined as the first surface points, so as to obtain a number of first surface points corresponding to the initial seed point.

5. A device for generating photographic points for appearance inspection, characterized in that, include: The acquisition module is used to acquire an initial point cloud containing several surface points corresponding to the object to be detected; The filtering module is used to filter out target point clouds corresponding to the detection area of ​​the object to be detected from the initial point cloud; The point cloud segmentation module is used to segment each surface point in the target point cloud based on a region growing algorithm to obtain several clustered point clouds. The generation module is used to determine the bounding box corresponding to each cluster of point clouds based on each surface point in the same cluster of point clouds; to segment the bottom surface of each bounding box to obtain a number of rectangular blocks corresponding to each bounding box; and to generate the initial image position corresponding to each rectangular block based on the center of each rectangular block, the shooting parameters of the target camera, and the direction vector of the height of the bounding box corresponding to the rectangular block. Based on the initial image capture points, several candidate image capture points corresponding to each initial image capture point are obtained. For each candidate image capture point, an unobstructed candidate image capture point is selected from the candidate image capture points using a ray casting algorithm. The tilt angle of the area to be detected is determined. If the tilt angle is less than or equal to a predetermined tilt angle threshold, the distance between each unobstructed candidate image capture point and the center point of the object to be detected is determined, and the candidate image capture point farthest from the center point of the object to be detected is determined as the target image capture point from the unobstructed candidate image capture points corresponding to the same initial image capture point. If the tilt angle is greater than the predetermined tilt angle threshold, the shooting height of each unobstructed candidate image capture point is determined, and the candidate image capture point with the highest shooting height is determined as the target image capture point from the unobstructed candidate image capture points corresponding to the same initial image capture point. Based on the target image capture points corresponding to each initial image capture point, several target image capture points corresponding to each clustered point cloud are generated.

6. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the image point generation method for appearance inspection as described in any one of claims 1-4.

7. An electronic device, characterized in that, It includes at least a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program in the memory, implements the steps of the image point generation method for appearance inspection as described in any one of claims 1-4.

Images