A method for detecting a vehicle door, a computing device and a storage medium

Abstract

The present application relates to the technical field of visual inspection, and discloses a method, computing device and storage medium for inspecting a vehicle door. The method comprises: separating a reference point cloud into a plurality of first single-frame contour point clouds according to a scanning direction of a laser line, and separating a non-reference point cloud into a plurality of second single-frame contour point clouds according to a scanning direction of a laser line, determining a first feature point set based on the plurality of first single-frame contour point clouds and a fitting plane, and determining a second feature point set based on the plurality of second single-frame contour point clouds and a fitting plane, projecting a plurality of feature points in the first feature point set and the second feature point set onto a test plane to obtain a first projection point set and a second projection point set, and determining a vehicle door gap to be measured based on the first projection point set and the second projection point set. The above method of obtaining a plurality of frames of point clouds to be measured, determining a feature point set according to a fitting plane, and determining a vehicle door gap according to a projection point set improves measurement accuracy and ensures reliability of testing the vehicle door.

Description

Technical Field

[0001] The present application relates to the technical field of visual detection, and provides a method for detecting a vehicle door, a computing device, and a storage medium. Background Art

[0002] In the field of engineering vehicle manufacturing, after the vehicle doors are assembled, the installation results of the doors need to be inspected, which usually includes measuring the gap and surface difference between the vehicle body and the installed doors.

[0003] At present, in addition to the traditional manual measurement method of using mechanical measuring tools (such as gap gauges, vernier calipers, etc.) to measure gaps and flushness, non-contact measuring equipment can also be used for measurement. For example, a robotic arm carries a line scan camera to complete the measurement. That is, the line scan camera obtains a single frame of laser contour point cloud data by irradiating the single frame of laser contour on the surface of the door gap.

[0004] However, since the laser line of the line scan camera is not completely perpendicular to the door gap, the feature points of the door part in the single-frame laser contour point cloud may not be located on the same contour line as the feature points of the nearest body part, resulting in matching difficulties and affecting the measurement accuracy and reliability. Summary of the invention

[0005] The embodiments of the present application provide a method, a computing device, and a storage medium for detecting a vehicle door, so as to improve the measurement accuracy and ensure the reliability of testing the vehicle door.

[0006] The specific technical solutions provided by this application are as follows:

[0007] In a first aspect, an embodiment of the present application provides a method for detecting a vehicle door, comprising:

[0008] Separating the reference point cloud into a plurality of first single-frame contour point clouds according to the scanning direction of the laser line, and separating the non-reference point cloud into a plurality of second single-frame contour point clouds according to the scanning direction of the laser line, wherein the reference point cloud and the non-reference point cloud are obtained by dividing a plurality of frames of the point cloud to be measured, and the plurality of frames of the point cloud to be measured are obtained by a line scanning camera performing a laser line scanning on the vehicle door to be measured;

[0009] Determine a first feature point set based on a plurality of first single-frame contour point clouds and a fitting plane, and determine a second feature point set based on a plurality of second single-frame contour point clouds and a fitting plane, wherein the fitting plane is determined based on each reference vertex, each reference vertex is determined based on a bounding box determined by a reference point cloud, the first feature point set includes feature points in each first single-frame contour point cloud that are closest to the fitting plane, and the second feature point set includes feature points in each second single-frame contour point cloud that are closest to the fitting plane;

[0010] A plurality of feature points in the first feature point set and the second feature point set are projected onto a test plane to obtain a first projection point set and a second projection point set, respectively, and a door gap to be measured is determined based on the first projection point set and the second projection point set, wherein the test plane is obtained after plane fitting of the reference point cloud.

[0011] In some possible implementations provided in this application, the reference point cloud and the non-reference point cloud are determined in the following manner:

[0012] Use a line scan camera to perform laser line scanning on the vehicle door to be tested in the scanning direction of the laser line to obtain multiple frames of point clouds to be tested;

[0013] Based on the preset Euclidean search radius and point cloud resolution, Euclidean clustering is performed on multiple frames of point clouds to be tested to obtain a reference point cloud and a non-reference point cloud. When the reference point cloud represents the point cloud corresponding to the vehicle body, the non-reference point cloud represents the point cloud corresponding to the adjacent door; when the reference point cloud represents the point cloud corresponding to the door, the non-reference point cloud represents the point cloud corresponding to the adjacent vehicle body.

[0014] In some possible implementations provided in the present application, after using a line scan camera to perform laser line scanning on the vehicle door to be tested in the scanning direction of the laser line to obtain multiple frames of point clouds to be tested, the method further includes:

[0015] Compare the contours of the multiple frames of the point clouds to be tested with the standard contours of the preset standard point clouds, wherein the standard point clouds are determined based on the point clouds of the door gaps that have passed the inspection;

[0016] If the contour to be measured is larger than the standard contour, the contour to be measured is cut based on the standard contour;

[0017] If the contour to be measured is smaller than the standard contour, the contour to be measured is filled based on the standard contour.

[0018] In some possible implementations provided by the present application, after comparing the contours to be measured of the multiple frames of the point cloud to be measured with the standard contours of the preset standard point cloud, the method further includes:

[0019] Perform plane fitting on multiple frames of point clouds to be measured to obtain a reference plane;

[0020] Determine the distance value from each point in the multi-frame point cloud to be measured to the reference plane;

[0021] The points whose distance values ​​exceed the preset distance threshold are deleted from the multi-frame point cloud to be measured.

[0022] In some possible implementations provided in the present application, the reference point cloud is separated into a plurality of first single-frame contour point clouds according to the scanning direction of the laser line, including:

[0023] Determine the first scanning coordinate value of each point of the reference point cloud in the scanning direction of the laser line;

[0024] The points with equal first scanning coordinate values ​​form a first single-frame contour point cloud;

[0025] Based on the difference of the first scanning coordinate values, the reference point cloud is separated into a plurality of first single-frame contour point clouds;

[0026] Separate the non-reference point cloud into multiple second single-frame contour point clouds according to the scanning direction of the laser line, including:

[0027] Determine the second scanning coordinate value of each point of the non-reference point cloud in the scanning direction of the laser line;

[0028] The points with the same second scanning coordinate values ​​constitute a second single-frame contour point cloud;

[0029] Based on the difference in the second scanning coordinate values, the non-reference point cloud is separated into a plurality of second single-frame contour point clouds.

[0030] In some possible implementations provided in the present application, determining a first feature point set based on a plurality of first single-frame contour point clouds and a fitting plane includes:

[0031] For any first single-frame contour point cloud, the following operations are performed: determining the distance between each point in the first single-frame contour point cloud and the fitting plane, and determining the point with the closest distance as a first feature point;

[0032] Combining a plurality of first feature points determined based on a plurality of first single-frame contour point clouds into a first feature point set;

[0033] Determining a second feature point set based on a plurality of second single-frame contour point clouds and a fitting plane includes:

[0034] For any second single-frame contour point cloud, the following operations are performed: determining the distance between each point in the second single-frame contour point cloud and the fitting plane, and determining the point with the closest distance as a second feature point;

[0035] A plurality of second feature points determined based on a plurality of second single-frame contour point clouds are combined into a second feature point set.

[0036] In some possible implementations provided in this application, the fitting plane is determined in the following manner:

[0037] Convert the reference point cloud from the original coordinate system to an orthogonal coordinate system, wherein the orthogonal coordinate system is determined based on the covariance matrix of the reference point cloud;

[0038] In an orthogonal coordinate system, a bounding box is determined based on the orthogonal coordinate values ​​of each point in the converted reference point cloud, and a plurality of preselected reference vertices included in the bounding box are determined;

[0039] Convert each pre-selected reference vertex from the orthogonal coordinate system to the original coordinate system, and determine at least three pre-selected reference vertices adjacent to the non-reference point cloud as reference vertices;

[0040] The fitting plane is determined based on the coordinate values ​​corresponding to each reference vertex in the original coordinate system.

[0041] In some possible implementations provided in the present application, determining the door gap to be measured based on the first projection point set and the second projection point set includes:

[0042] For any first projection point in the first projection point set, respectively calculate the distance between the first projection point and any second projection point in the second projection point set, determine a second target projection point corresponding to the first projection point based on the closest distance, and determine the average value of the distances between each first projection point and the corresponding second target projection point as the door gap to be measured; or

[0043] For any first projection point in the first projection point set, the distance between the first projection point and the second projection line in the second projection point set is calculated respectively, and the average value of each distance is determined as the door gap to be measured, wherein the second projection line is determined by each second projection point in the second projection point set.

[0044] In a second aspect, a computing device includes:

[0045] A memory for storing executable instructions;

[0046] A processor is used to read and execute executable instructions stored in a memory to implement a method as described in any one of the items of the first aspect.

[0047] In a third aspect, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor, the processor is enabled to execute the method described in any one of the items of the first aspect.

[0048] The beneficial effects of this application are as follows:

[0049] In summary, in an embodiment of the present application, a method for detecting a vehicle door, a computing device, and a storage medium are provided, the method comprising: separating a reference point cloud into a plurality of first single-frame contour point clouds according to a scanning direction of a laser line, and separating a non-reference point cloud into a plurality of second single-frame contour point clouds according to a scanning direction of a laser line, wherein the reference point cloud and the non-reference point cloud are obtained by dividing a plurality of frames of point clouds to be tested, and the plurality of frames of point clouds to be tested are obtained by a line scan camera performing a laser line scan on a vehicle door to be tested, determining a first feature point set based on the plurality of first single-frame contour point clouds and a fitting plane, and determining a second feature point set based on the plurality of second single-frame contour point clouds and a fitting plane, wherein the fitting plane is determined based on each reference vertex, and each reference vertex is Based on the bounding box determined by the reference point cloud, the first feature point set includes the feature points in each first single-frame contour point cloud that are closest to the fitting plane, and the second feature point set includes the feature points in each second single-frame contour point cloud that are closest to the fitting plane. Multiple feature points in the first feature point set and the second feature point set are respectively projected onto the test plane to obtain the first projection point set and the second projection point set, and the door gap to be tested is determined based on the first projection point set and the second projection point set, wherein the test plane is obtained after plane fitting of the reference point cloud. The above method of obtaining multiple frames of point clouds to be tested, determining the feature point set according to the fitting plane, and determining the door gap according to the projection point set improves the measurement accuracy and ensures the reliability of testing the door.

[0050] Other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0052] Figure 1 A schematic diagram of a flow chart of a method for detecting a vehicle door in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of a process for dividing a plurality of frames of point clouds to be measured into reference point clouds and non-reference point clouds in an embodiment of the present application;

[0054] Figure 3 This is a schematic diagram of preprocessing multiple frames of point clouds to be measured in an embodiment of the present application;

[0055] Figure 4This is a schematic diagram of a process of separating a reference point cloud into a plurality of first single-frame contour point clouds according to a scanning direction of a laser line in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of a process of separating a non-reference point cloud into a plurality of second single-frame contour point clouds according to the scanning direction of a laser line in an embodiment of the present application;

[0057] Figure 6 A schematic diagram of a process of determining a first feature point set based on a plurality of first single-frame contour point clouds and a fitting plane in an embodiment of the present application;

[0058] Figure 7 A schematic diagram of a process of determining a second feature point set based on a plurality of second single-frame contour point clouds and a fitting plane in an embodiment of the present application;

[0059] Figure 8 A schematic diagram of determining a bounding box in an orthogonal coordinate system in an embodiment of the present application;

[0060] Fig. 9 A schematic diagram of the physical architecture of a computing device in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0062] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented using sequences other than those illustrated or described herein.

[0063] The preferred implementation modes of the present application are described in detail below with reference to the accompanying drawings.

[0064] In an embodiment of the present application, the system includes at least one computing device and a line scan camera. When the vehicle door is assembled, the line scan camera carried by the robotic arm emits a laser line to scan the part of the door close to the body, thereby obtaining multiple frames of point clouds to be tested, and the above multiple frames of point clouds to be tested are sent to the computing device, so that the computing device can detect whether the door gap and surface difference after the door is installed are qualified according to the multiple frames of point clouds to be tested. The specific introduction is as follows.

[0065] See also Figure 1 As shown, in the embodiment of the present application, a specific process of detecting a vehicle door is as follows:

[0066] Step 201: Separate the reference point cloud into a plurality of first single-frame contour point clouds according to the scanning direction of the laser line, and separate the non-reference point cloud into a plurality of second single-frame contour point clouds according to the scanning direction of the laser line, wherein the reference point cloud and the non-reference point cloud are obtained by dividing a plurality of frames of point clouds to be measured, and the plurality of frames of point clouds to be measured are obtained by a line scan camera performing a laser line scan on a vehicle door to be measured.

[0067] First of all, it should be explained that the reference point cloud and non-reference point cloud are determined in the following way. Figure 2 As shown:

[0068] Step 101: Use a line scan camera to perform laser line scanning on the vehicle door to be tested in the scanning direction of the laser line to obtain multiple frames of point clouds to be tested.

[0069] In an embodiment of the present application, a robotic arm carrying a line scan camera is used to perform flexibly measure the assembly results of the vehicle door. The line scan camera is usually installed at the end of the robotic arm. The robotic arm carries the line scan camera to perform uniform linear motion to complete the scanning of the laser line of the vehicle door to be measured. During the implementation process, the line scan camera is used to perform laser line scanning on the vehicle door to be measured to obtain multiple frames of point clouds to be measured. Usually, the scanning direction of the laser line is the y-axis direction of the multiple frames of point clouds to be measured, that is, the multiple frames of point clouds to be measured are obtained by translational scanning of the line scan camera in the y-axis direction.

[0070] Step 102: Perform Euclidean clustering on multiple frames of point clouds to be measured based on a preset Euclidean search radius and point cloud resolution to obtain a reference point cloud and a non-reference point cloud, wherein when the reference point cloud represents the point cloud corresponding to the vehicle body, the non-reference point cloud represents the point cloud corresponding to the adjacent door. When the reference point cloud represents the point cloud corresponding to the door, the non-reference point cloud represents the point cloud corresponding to the adjacent vehicle body.

[0071] Considering that the door gap and surface difference are determined based on the installed door and the adjacent body, after obtaining multiple frames of point cloud to be tested, the point cloud needs to be further divided into reference point cloud and non-reference point cloud.

[0072] The above-mentioned reference point cloud and non-reference point cloud are determined according to the preset Euclidean search radius and point cloud resolution. For example, the point cloud resolution is 0.1 mm, the Euclidean search radius is set to 0.3 mm, and the minimum number of points for clustering is set to 1×10 5points, the maximum number of points is set to all points of the current multi-frame point cloud to be tested, etc. During the implementation process, after the Euclidean search radius is determined, each point on the multi-frame point cloud to be tested is Euclidean clustered based on the Euclidean search radius, thereby dividing the multi-frame point cloud to be tested into two parts. Usually, there will be an obvious gap between the door and the adjacent body, so the reference point cloud and non-reference point cloud obtained during the implementation process will also correspond to the door and the body.

[0073] In the embodiment of the present application, when the reference point cloud represents the point cloud corresponding to the vehicle body, the non-reference point cloud represents the point cloud corresponding to the adjacent door. When the reference point cloud represents the point cloud corresponding to the door, the non-reference point cloud represents the point cloud corresponding to the adjacent vehicle body.

[0074] It should also be noted that the start and stop process of the robot arm will cause deformation of the head and tail areas of the multi-frame point cloud to be obtained in the scanning direction. Based on this, after the line scan camera is used to perform laser line scanning on the door to be tested in the scanning direction of the laser line to obtain the multi-frame point cloud to be tested, the following steps are also included:

[0075] (1) Compare the contours of the point clouds to be measured in multiple frames with the standard contours of the preset standard point clouds, wherein the standard point clouds are determined based on the point clouds of the door gaps that have passed the inspection.

[0076] In order to avoid the influence of point cloud deformation on measurement accuracy, the standard point cloud will be determined in advance based on the point cloud corresponding to the door gap that has been tested and qualified. The standard point cloud is usually obtained after the same line scan camera has tested the door of the same model and qualified. In order to facilitate comparison with multiple frames of point clouds to be tested, the computing device will store the standard outline of the above standard point cloud in advance. For example, when the standard point cloud is a rectangle, the length and width of the standard point cloud will be measured and stored in advance.

[0077] During the implementation process, after obtaining the standard contour of the standard point cloud, the contours to be measured of multiple frames of the point cloud to be measured are compared with the standard contour of the preset standard point cloud with reference to the door part.

[0078] (2) If the contour to be measured is larger than the standard contour, the contour to be measured is cropped based on the standard contour.

[0079] In one case, if the contour to be measured is larger than the standard contour, that is, the contour to be measured is outside the standard contour, during the implementation process, the contour to be measured is cropped according to the shape and size of the standard contour based on the standard contour. For example, a straight-through filter is used to crop the contour to be measured along the x-axis direction, and then the contour to be measured is cropped along the y-axis direction, thereby obtaining a cropped multi-frame point cloud to be measured.

[0080] (3) If the contour to be measured is smaller than the standard contour, the contour to be measured is filled based on the standard contour.

[0081] In another case, if the contour to be measured is smaller than the standard contour, that is, the contour to be measured is inside the standard contour, during the implementation process, the contour to be measured is filled in accordance with the shape and size of the standard contour based on the standard contour, thereby avoiding the situation in which the multi-frame point cloud to be collected is incomplete due to occlusion of the line scan camera.

[0082] In addition, in order to avoid the influence of noise on the measurement accuracy, after comparing the contours of the point cloud to be measured with the standard contours of the preset standard point cloud, refer to Figure 3 As shown, it also includes:

[0083] 1) Perform plane fitting on multiple frames of point clouds to obtain the reference plane.

[0084] Considering that there are also sparse point clouds in the depth direction of the gap between the door and the adjacent body during scanning, the sparse point clouds at the above positions are not used for the measurement of gap and flushness, and the sparse point clouds will interfere with the extraction of feature points. Figure 3 As shown in Figure A in the upper middle, based on this, in the implementation process, RANSAC and least squares method are first used to fit the plane of multiple frames of the point cloud to be measured to obtain the reference plane. For example, when the fitting method used is the RANSAC plane fitting method, the number of iterations is set to 200, the threshold size is 5mm, and the reference plane obtained after fitting is referred to Figure 3 As shown in Figure B in the lower middle.

[0085] 2) Determine the distance value from each point in the multi-frame point cloud to be measured to the reference plane.

[0086] In order to filter out noise points, during the implementation process, the distance between the point and the reference plane is used to measure whether each point in the multi-frame point cloud to be tested is a noise point. That is, after obtaining the reference plane, the distance value from the point to the above reference plane is calculated for each point in the multi-frame point cloud to be tested.

[0087] 3) Delete points whose distance values ​​exceed the preset distance threshold from the multi-frame point cloud to be tested.

[0088] After calculating the distance value from each point to the reference plane, each distance value is compared with the preset distance threshold. It should be noted that the preset distance threshold represents the maximum distance value for a point to be identified as a noise point. During the implementation process, when the distance value corresponding to any point exceeds the preset distance threshold, the point is determined to be a noise point, and then the point is deleted from the multi-frame point cloud to be tested, thereby achieving the purpose of filtering noise points.

[0089] Since the above-mentioned multi-frame point cloud to be measured is a collection of multi-frame point cloud data obtained by scanning the door and the adjacent body with a line scan camera, in order to obtain accurate gap and face difference, etc. during the implementation process, the reference point cloud is separated into multiple first single-frame contour point clouds according to the scanning direction of the laser line, see Figure 4 As shown, including:

[0090] Step 2011: Determine the first scanning coordinate value of each point of the reference point cloud in the scanning direction of the laser line.

[0091] After dividing the multi-frame point clouds to be measured into reference point clouds and non-reference point clouds, the reference point clouds and non-reference point clouds are also collections of multi-frame point clouds. During the implementation process, the reference point clouds and non-reference point clouds need to be separated into point clouds with single-frame contours respectively.

[0092] Considering that the multi-frame point cloud to be measured is a collection of point clouds to be measured obtained by multiple scans of the line scan camera along the scanning direction of the laser line, based on this, separation is performed based on the scanning coordinate value obtained by the line scan camera along the scanning direction of the laser line. During the implementation process, the first scanning coordinate value of each point in the reference point cloud in the scanning direction of the laser line is first determined. For example, when the scanning direction of the laser line is the y-axis direction, the first scanning coordinate value of each point in the reference point cloud in the y-axis direction is obtained respectively.

[0093] Step 2012: All points with equal first scanning coordinate values ​​form a first single-frame contour point cloud.

[0094] Since the first scanning coordinate values ​​of each point obtained by the line scan camera during a scanning process are the same, during the implementation process, after the first scanning coordinate values ​​of each point are obtained, the points with equal first scanning coordinate values ​​are formed into a first single-frame contour point cloud, that is, the points with equal first scanning coordinate values ​​among the multiple points included in the reference point cloud are determined to be a first single-frame contour point cloud.

[0095] Step 2013: Based on the difference in the first scanning coordinate values, separate the reference point cloud into a plurality of first single-frame contour point clouds.

[0096] During the implementation process, after each point with equal first scanning coordinate values ​​is determined to be a first single-frame contour point cloud, each point with different first scanning coordinate values ​​in the reference point cloud can respectively constitute multiple different first single-frame contour point clouds, thereby separating the reference point cloud into multiple different first single-frame contour point clouds.

[0097] Similarly, the non-reference point cloud is separated into multiple second single-frame contour point clouds according to the scanning direction of the laser line, see Figure 5 As shown, including:

[0098] Step 2011 ′: determining the second scanning coordinate value of each point of the non-reference point cloud in the scanning direction of the laser line.

[0099] During the implementation, the second scanning coordinate value of each point in the non-reference point cloud in the scanning direction of the laser line is first determined. For example, when the scanning direction of the laser line is the y-axis direction, the second scanning coordinate value of each point in the non-reference point cloud in the y-axis direction is obtained respectively.

[0100] Step 2012 ′: construct a second single-frame contour point cloud with each point having the same second scanning coordinate value.

[0101] Since the second scanning coordinate values ​​of each point obtained by the line scan camera during a scanning process are the same, during the implementation process, after the second scanning coordinate values ​​of each point are obtained, the points with equal second scanning coordinate values ​​are formed into a second single-frame contour point cloud, that is, the points with equal second scanning coordinate values ​​among the multiple points included in the non-reference point cloud are determined to be a second single-frame contour point cloud.

[0102] Step 2013 ′: Based on the difference in the second scanning coordinate values, separate the non-reference point cloud into a plurality of second single-frame contour point clouds.

[0103] During the implementation process, after each point with equal second scanning coordinate values ​​is determined to be a second single-frame contour point cloud, each point with different second scanning coordinate values ​​in the non-reference point cloud can respectively constitute a plurality of different second single-frame contour point clouds, thereby separating the non-reference point cloud into a plurality of different second single-frame contour point clouds.

[0104] Step 202: Determine a first feature point set based on multiple first single-frame contour point clouds and fitting planes, and determine a second feature point set based on multiple second single-frame contour point clouds and fitting planes, wherein the fitting planes are determined based on each reference vertex, each reference vertex is determined based on a bounding box determined by a reference point cloud, the first feature point set includes feature points in each first single-frame contour point cloud that are closest to the fitting plane, and the second feature point set includes feature points in each second single-frame contour point cloud that are closest to the fitting plane.

[0105] During implementation, after obtaining a plurality of first single-frame contour point clouds and a plurality of second single-frame contour point clouds, a first feature point set and a second feature point set are further determined, that is, feature points for calculating gaps and face differences are preliminarily determined.

[0106] First of all, it should be noted that the feature points are determined based on the fitting plane, and the fitting plane is determined in the following way:

[0107] [1] The reference point cloud is transformed from the original coordinate system to an orthogonal coordinate system, wherein the orthogonal coordinate system is determined based on the covariance matrix of the reference point cloud.

[0108] Considering that the laser line emitted by the line scan camera in the actual measurement scene is not completely perpendicular to the door and the adjacent body, the coordinates of the reference point cloud in the original coordinate system are not accurate. Based on this, in the implementation process, the covariance matrix is ​​further calculated after obtaining the reference point cloud. , and calculate the covariance matrix The three eigenvectors of , the three eigenvectors Orthogonalize to ensure that the three eigenvectors are perpendicular to each other.

[0109] According to the above covariance matrix Determine a rotation matrix , a translation vector a translation vector and a central point ,in, is the maximum coordinate of the X-axis in the original coordinate system and the minimum coordinates The calculated coordinates of the midpoint, is the maximum coordinate of the Y axis in the original coordinate system and the minimum coordinates The calculated coordinates of the midpoint, is the maximum coordinate of the Z axis in the original coordinate system and the minimum coordinates The calculated coordinates of the middle point. Taking the above center point P as the origin, the three eigenvectors For the coordinate axes, establish an orthogonal coordinate system.

[0110] Thus, for any point in the reference point cloud The coordinates in the orthogonal coordinate system are , the conversion relationship is shown in the following formula.

[0111] Formula (1)

[0112] Formula (2)

[0113] That is, use the above formula (1) to convert the coordinate point in the original coordinate system Convert to the orthogonal coordinate system and get the coordinate point as ; Use the above formula (2) to convert the coordinate point in the orthogonal coordinate system Convert to the original coordinate system and get the coordinate point as .

[0114] During the implementation process, the above formula (1) is used to transform each point in the reference point cloud from the original coordinate system to the orthogonal coordinate system.

[0115] [2] See Figure 8As shown, in an orthogonal coordinate system, a bounding box is determined based on the orthogonal coordinate values ​​of each point in the converted reference point cloud, and a plurality of pre-selected reference vertices included in the bounding box are determined.

[0116] During the implementation process, after converting each point in the reference point cloud from the original coordinate system to the orthogonal coordinate system, the orthogonal coordinate values ​​of each point in the converted reference point cloud in the orthogonal coordinate system are determined, and the maximum and minimum coordinate values ​​in the X-axis, Y-axis and Z-axis directions in the orthogonal coordinate system are selected, that is, the maximum coordinate value of the X-axis in the orthogonal coordinate system is determined.

[0117] and the minimum coordinate value , the maximum coordinate value of the Y axis in the orthogonal coordinate system and the minimum coordinate value , the maximum coordinate value of the Z axis in the orthogonal coordinate system and the minimum coordinate value .

[0118] Further, according to the above maximum coordinate value and minimum coordinate value, 8 coordinate points are determined, namely, coordinate point 1 、Coordinate point 2 、Coordinate point 3 、Coordinate point 4 、Coordinate point 5 、Coordinate point 6 、Coordinate point 7 and coordinate point 8 , and a cuboid is determined according to the above 8 coordinate points, and the cuboid is the above bounding box. During the implementation process, the 8 coordinate points included in the above bounding box are determined as multiple pre-selected reference vertices.

[0119] [3] Each pre-selected reference vertex is converted from the orthogonal coordinate system to the original coordinate system, and at least three pre-selected reference vertices adjacent to the non-reference point cloud are determined as reference vertices.

[0120] Considering that the reference point cloud is in an orthogonal coordinate system, while the non-reference point cloud is in an original coordinate system, in order to unify the coordinate system, after determining multiple pre-selected reference vertices, the above formula (2) is used to transform each point in the reference point cloud from the orthogonal coordinate system to the original coordinate system.

[0121] After the reference point cloud carrying the bounding box and the non-reference point cloud are both in the original coordinate system, at least three pre-selected reference vertices in the reference point cloud that are adjacent to the non-reference point cloud are determined as reference vertices.

[0122] [4] The fitting plane is determined based on the coordinate values ​​of each reference vertex in the original coordinate system.

[0123] During the implementation process, after at least three reference vertices are determined, a plane is constructed according to the coordinate values ​​corresponding to the above reference vertices in the original coordinate system, and the plane is determined as the fitting plane.

[0124] After the fitting plane is determined, the first feature point set is determined based on the plurality of first single-frame contour point clouds and the fitting plane. Figure 6 As shown, including:

[0125] Step 2021: For any first single-frame contour point cloud, perform the following operations: determine the distance from each point in the first single-frame contour point cloud to the fitting plane, and determine the point with the closest distance as a first feature point.

[0126] Since a first single-frame contour point cloud includes multiple points, during the implementation process, the following operations are performed for each point in each first single-frame contour point cloud: the distance from each point in the first single-frame contour point cloud to the fitting plane is calculated respectively, so as to obtain multiple distances corresponding to the points one by one, compare the distances, and select the nearest distance therefrom. The point corresponding to the nearest distance is a first feature point of a first single-frame contour point cloud.

[0127] Step 2022: multiple first feature points determined based on multiple first single-frame contour point clouds are combined into a first feature point set.

[0128] After comparing the above distances for each first single-frame contour point cloud respectively, multiple first feature points can be obtained, and the multiple first feature points are combined into a first feature point set, and the number of first feature points in the first feature point set is equal to the number of the first single-frame contour point cloud.

[0129] Similarly, a second feature point set is determined based on a plurality of second single-frame contour point clouds and the fitted plane, see Figure 7 As shown, including:

[0130] Step 2021': perform the following operations for any second single-frame contour point cloud: determine the distance from each point in the second single-frame contour point cloud to the fitting plane, and determine the point with the shortest distance as a second feature point.

[0131] Since a second single-frame contour point cloud includes multiple points, during the implementation process, the following operations are performed for each point in each second single-frame contour point cloud: the distance from each point in the second single-frame contour point cloud to the fitting plane is calculated respectively, so as to obtain multiple distances corresponding to the points one by one, compare the distances, and select the nearest distance therefrom. The point corresponding to the nearest distance is a second feature point of a second single-frame contour point cloud.

[0132] Step 2022': assemble a second feature point set from a plurality of second feature points determined based on a plurality of second single-frame contour point clouds.

[0133] After comparing the above distances for each second single-frame contour point cloud respectively, multiple second feature points can be obtained, and the multiple second feature points are combined into a second feature point set, and the number of second feature points in the second feature point set is equal to the number of second single-frame contour point clouds.

[0134] Step 203: Project multiple feature points in the first feature point set and the second feature point set onto the test plane to obtain the first projection point set and the second projection point set, and determine the door gap to be tested based on the first projection point set and the second projection point set, wherein the test plane is obtained after plane fitting of the reference point cloud.

[0135] Taking into account that there is inevitably positional offset in the determination process of the feature points in the first feature point set and the second feature point set, in order to correct the feature points, during the implementation process, the reference point cloud is first plane-fitted to obtain a test plane. For example, the reference point cloud is plane-fitted using the RANSAC plane fitting algorithm, and then projection points are obtained based on the test plane. In the specific implementation process, multiple feature points in the first feature point set are respectively projected onto the test plane to obtain multiple projection points, and the multiple projection points are combined into a first projection point set; and multiple feature points in the second feature point set are respectively projected onto the test plane to obtain multiple projection points, and the multiple projection points are combined into a second projection point set. The above projection process will not be repeated here.

[0136] After obtaining the first projection point set and the second projection point set, the above-mentioned method of determining the door gap to be measured based on the first projection point set and the second projection point set includes:

[0137] The first case: for any first projection point in the first projection point set, the distance between the first projection point and any second projection point in the second projection point set is calculated respectively, a second target projection point corresponding to the first projection point is determined based on the closest distance, and the average value of the distance between each first projection point and the corresponding second target projection point is determined as the door clearance to be measured. Or

[0138] In one embodiment, the distance between two points is used to determine the door gap to be measured. During the implementation, the first projection point in the first projection point set is used as a reference to determine the distance between the second projection point set and the first projection point. That is, firstly, a first projection point in the first projection point set is used as a reference to calculate the distance between each second projection point in the second projection point set and the first projection point. After calculating multiple distances, the closest distance is selected, and the second projection point corresponding to the closest distance is determined as the second target projection point corresponding to the first projection point. In the process of repeating the above steps, multiple second target projection points can be obtained.

[0139] After obtaining multiple second target projection points, the distances between each first projection point and the corresponding second target projection point are calculated respectively, thereby obtaining multiple distances, and the average value of the multiple distances is calculated, and the average value is determined as the door gap to be measured.

[0140] The second case: for any first projection point in the first projection point set, the distance between the first projection point and the second projection line in the second projection point set is calculated respectively, and the average value of each distance is determined as the door gap to be measured, wherein the second projection line is determined by each second projection point in the second projection point set.

[0141] In another embodiment, the distance from a point to a straight line is used to determine the door gap to be measured. During the implementation process, the second projection points in the second projection point set are first connected to obtain the second projection line, and then, with the first projection point in the first projection point set as a reference, the distance between each first projection point and the second projection line in the second projection point set is calculated, that is, the vertical distance from the first projection point to the second projection line is calculated. After performing the above operation on each first projection point, multiple distances are obtained, and the average value of each distance is further calculated. The above average value is determined as the door gap to be measured.

[0142] It should be noted that, in the process of detecting the vehicle door, after the gap of the vehicle door to be tested is determined, the surface difference of the vehicle door to be tested is further determined. During the implementation process, the reference point cloud is plane fitted, for example, the RANSAC algorithm is used to perform plane fitting, so as to obtain an auxiliary plane, and then the distance from each point in the non-reference point cloud to the above auxiliary plane is calculated, and a preset number of larger distances are selected therefrom, and the average value of the above selected distances is used as the surface difference of the vehicle door to be tested.

[0143] Based on the same inventive concept, refer to Fig. 9 As shown, an embodiment of the present application provides a computing device, including: a memory 901, used to store executable instructions; a processor 902, used to read and execute the executable instructions stored in the memory, and execute any one of the methods of the first aspect above.

[0144] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor, the processor is enabled to execute the method described in any one of the above-mentioned first aspects.

[0145] In summary, in an embodiment of the present application, a method for detecting a vehicle door, a computing device, and a storage medium are provided, the method comprising: separating a reference point cloud into a plurality of first single-frame contour point clouds according to a scanning direction of a laser line, and separating a non-reference point cloud into a plurality of second single-frame contour point clouds according to a scanning direction of a laser line, wherein the reference point cloud and the non-reference point cloud are obtained by dividing a plurality of frames of point clouds to be tested, and the plurality of frames of point clouds to be tested are obtained by a line scan camera performing a laser line scan on a vehicle door to be tested, determining a first feature point set based on the plurality of first single-frame contour point clouds and a fitting plane, and determining a second feature point set based on the plurality of second single-frame contour point clouds and a fitting plane, wherein the fitting plane is determined based on each reference vertex, and each reference vertex is Based on the bounding box determined by the reference point cloud, the first feature point set includes the feature points in each first single-frame contour point cloud that are closest to the fitting plane, and the second feature point set includes the feature points in each second single-frame contour point cloud that are closest to the fitting plane. Multiple feature points in the first feature point set and the second feature point set are respectively projected onto the test plane to obtain the first projection point set and the second projection point set, and the door gap to be tested is determined based on the first projection point set and the second projection point set, wherein the test plane is obtained after plane fitting of the reference point cloud. The above method of obtaining multiple frames of point clouds to be tested, determining the feature point set according to the fitting plane, and determining the door gap according to the projection point set improves the measurement accuracy and ensures the reliability of testing the door.

[0146] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program product systems. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product system implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0147] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program product systems according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0150] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting a vehicle door, characterized in that: The method comprises: Determine the first scanning coordinate value of each point of the reference point cloud in the scanning direction of the laser line, form a first single-frame contour point cloud with each point having the same first scanning coordinate value, separate the reference point cloud into a plurality of the first single-frame contour point clouds based on the difference of the first scanning coordinate value, and determine the second scanning coordinate value of each point of the non-reference point cloud in the scanning direction of the laser line, form a second single-frame contour point cloud with each point having the same second scanning coordinate value, separate the non-reference point cloud into a plurality of the second single-frame contour point clouds based on the difference of the second scanning coordinate value, wherein the reference point cloud and the non-reference point cloud are obtained by dividing a plurality of frames of point clouds to be measured, and the plurality of frames of point clouds to be measured are obtained by performing laser line scanning on the vehicle door to be measured by a line scanning camera; Determine a first feature point set based on the plurality of first single-frame contour point clouds and the fitting plane, and determine a second feature point set based on the plurality of second single-frame contour point clouds and the fitting plane, wherein the fitting plane is determined based on each reference vertex, and each reference vertex is determined based on a bounding box determined by the reference point cloud, the first feature point set includes feature points in each first single-frame contour point cloud that are closest to the fitting plane, and the second feature point set includes feature points in each second single-frame contour point cloud that are closest to the fitting plane; Projecting multiple feature points in the first feature point set and the second feature point set onto a test plane respectively to obtain a first projection point set and a second projection point set, and determining a door gap to be measured based on the first projection point set and the second projection point set, wherein the test plane is obtained after plane fitting of the reference point cloud.

2. The method according to claim 1, characterized in that The reference point cloud and the non-reference point cloud are determined by: Using the line scan camera to perform laser line scanning on the vehicle door to be tested according to the scanning direction of the laser line to obtain multiple frames of point clouds to be tested; Based on the preset Euclidean search radius and point cloud resolution, Euclidean clustering is performed on the multiple frames of point clouds to be measured to obtain the reference point cloud and the non-reference point cloud, wherein when the reference point cloud represents the point cloud corresponding to the vehicle body, the non-reference point cloud represents the point cloud corresponding to the adjacent door; when the reference point cloud represents the point cloud corresponding to the door, the non-reference point cloud represents the point cloud corresponding to the adjacent vehicle body.

3. The method according to claim 2, characterized in that After using the line scan camera to perform laser line scanning on the vehicle door to be tested according to the scanning direction of the laser line to obtain multiple frames of point clouds to be tested, the method further includes: Comparing the contours to be measured of the multiple frames of the point clouds to be measured with the standard contours of a preset standard point cloud, wherein the standard point cloud is determined based on the point cloud of the door gap that has passed the inspection; If the contour to be measured is larger than the standard contour, the contour to be measured is cut based on the standard contour; If the contour to be measured is smaller than the standard contour, the contour to be measured is filled based on the standard contour.

4. The method according to claim 3, characterized in that After comparing the contours to be measured of the multiple frames of the point cloud to be measured with the standard contours of the preset standard point cloud, the method further includes: Performing plane fitting on the multiple frames of point clouds to be measured to obtain a reference plane; Determine the distance value from each point in the multi-frame point cloud to be measured to the reference plane; The points whose distance values ​​exceed a preset distance threshold are deleted from the multi-frame point cloud to be measured.

5. The method according to claim 1, characterized in that The determining of a first feature point set based on the plurality of first single-frame contour point clouds and the fitting plane comprises: For any one of the first single-frame contour point clouds, the following operations are performed: determining the distance between each point in the first single-frame contour point cloud and the fitting plane, and determining the point with the closest distance as one of the first feature points; Combining the first feature point set with the plurality of first feature points determined based on the plurality of first single-frame contour point clouds; The determining of a second feature point set based on the plurality of second single-frame contour point clouds and the fitting plane comprises: For any one of the second single-frame contour point clouds, the following operations are performed: determining the distance between each point in the second single-frame contour point cloud and the fitting plane, and determining the point with the shortest distance as one of the second feature points; The plurality of second feature points determined based on the plurality of second single-frame contour point clouds form the second feature point set.

6. The method according to claim 1, characterized in that The fitting plane is determined by: Converting the reference point cloud from an original coordinate system to an orthogonal coordinate system, wherein the orthogonal coordinate system is determined based on a covariance matrix of the reference point cloud; In the orthogonal coordinate system, determining the bounding box based on the orthogonal coordinate values ​​of each point in the converted reference point cloud, and determining a plurality of preselected reference vertices included in the bounding box; Convert each of the preselected reference vertices from the orthogonal coordinate system to the original coordinate system, and determine at least three preselected reference vertices adjacent to the non-reference point cloud as the reference vertices; The fitting plane is determined based on the coordinate values ​​corresponding to each of the reference vertices in the original coordinate system.

7. The method according to any one of claims 1 to 6, characterized in that The determining of the door gap to be measured based on the first projection point set and the second projection point set includes: For any first projection point in the first projection point set, respectively calculate the distance between the first projection point and any second projection point in the second projection point set, determine a second target projection point corresponding to the first projection point based on the closest distance, and determine the average value of the distances between each first projection point and the corresponding second target projection point as the door gap to be measured; or For any first projection point in the first projection point set, the distance between the first projection point and the second projection line in the second projection point set is calculated respectively, and the average value of each distance is determined as the door gap to be measured, wherein the second projection line is determined by each of the second projection points in the second projection point set.

8. A computing device, characterized in that include: A memory for storing executable instructions; A processor, configured to read and execute the executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor, the processor is enabled to execute the method according to any one of claims 1 to 7.

Images