Sled flatness detection method and device, computer device and storage medium

Abstract

This invention discloses a method, apparatus, computer equipment, and storage medium for detecting the flatness of a sled. The method includes: acquiring target point cloud data corresponding to each plane to be detected in each of the crossbeams using a camera, wherein each crossbeam includes multiple planes to be detected; determining a first plane equation for each of the target point cloud data based on a preset plane algorithm and the target point cloud data; transforming each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain a second plane equation for each of the target point cloud data in the preset coordinate system; determining the Z-axis vector of the reference coordinate system based on the normal vector of each of the second plane equations; and determining the flatness coefficient of the sled based on the angle difference between each of the normal vectors and the Z-axis vector. This method achieves non-contact measurement of sled flatness, improves the testing accuracy and efficiency of sled flatness, and enhances the safety and reliability of sled transportation.

Description

Technical Field

[0001] This invention relates to the field of production line control technology, and in particular to a method, apparatus, computer equipment, and storage medium for detecting the flatness of a skid. Background Technology

[0002] A skid is a conveyor system device used in automobile production lines to support and transport car bodies. Multiple crossbeams are mounted on top of the skid, with their upper surfaces in direct contact with the underside of the car body; meaning, the crossbeams directly support the car body. Therefore, the flatness of the crossbeam surfaces affects the overall flatness of the skid, and inspecting the flatness of the crossbeams is crucial for ensuring transportation safety and reliability. However, existing flatness inspection methods mainly rely on contact measurement or manual visual inspection. Contact measurement requires using measuring tools to touch the crossbeams, which can easily damage them, is complex, and has low efficiency. Furthermore, it struggles to detect subtle flatness differences. Visual inspection is time-consuming and inaccurate, and also fails to detect minute flatness variations. This results in the skid itself being easily damaged during overall flatness testing, along with low measurement accuracy and efficiency. Summary of the Invention

[0003] This invention provides a method, apparatus, computer equipment, and storage medium for detecting the flatness of a skid, aiming to solve the problems in existing methods that easily damage the skid when detecting the overall flatness of a skid, and whose detection accuracy and efficiency are both at a low level.

[0004] In a first aspect, embodiments of the present invention provide a method for detecting the flatness of a skid, the method being used to detect the flatness of a skid, the skid comprising multiple crossbeams, the method comprising:

[0005] The target point cloud data corresponding to each plane to be detected in each of the crossbeams are collected by a camera, and each of the crossbeams includes multiple planes to be detected.

[0006] The first plane equation of each target point cloud data is determined according to the preset plane algorithm and the target point cloud data respectively;

[0007] Transform each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain the second plane equations of each of the target point cloud data in the preset coordinate system;

[0008] Based on the normal vectors of each of the second plane equations, determine the Z-axis vector of the preset coordinate system;

[0009] The flatness coefficient of the skid is determined based on the angle difference between each of the normal vectors and the Z-axis vector.

[0010] In a second aspect, embodiments of the present invention provide a skid flatness detection device, comprising:

[0011] The acquisition unit is used to acquire target point cloud data corresponding to each plane to be detected in each of the crossbeams through a camera. Each of the crossbeams includes multiple planes to be detected.

[0012] The first determining unit is used to determine the first plane equation of each target point cloud data according to the preset plane algorithm and each target point cloud data respectively;

[0013] A transformation unit is used to transform each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain the second plane equations of each of the target point cloud data in the preset coordinate system.

[0014] The second determining unit is used to determine the Z-axis vector of the preset coordinate system based on the normal vector of each of the second plane equations;

[0015] The third determining unit is used to determine the flatness coefficient of the skid based on the angle difference between each of the normal vectors and the Z-axis vector.

[0016] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the skid flatness detection method as described in the first aspect.

[0017] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the skid flatness detection method as described in the first aspect.

[0018] This invention provides a method, apparatus, computer device, and storage medium for detecting the flatness of a skid. The method includes: acquiring target point cloud data corresponding to each plane to be detected in each of the crossbeams using a camera, wherein each crossbeam includes multiple planes to be detected; determining a first plane equation for each of the target point cloud data based on a preset plane algorithm and the target point cloud data; transforming each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain a second plane equation for each of the target point cloud data in the preset coordinate system; determining the Z-axis vector of the preset coordinate system based on the normal vector of each of the second plane equations; and determining the flatness coefficient of the skid based on the angle difference between each of the normal vectors and the Z-axis vector. This invention utilizes a camera to take non-contact photographs of the crossbeam in a skid and generate a 3D point cloud. Subsequently, the 3D point cloud is used to fit a plane equation to the plane to be tested. Finally, the skid flatness coefficient is calculated based on the fitted plane equation and the offset of its normal vector in the point cloud. This achieves non-contact measurement of skid flatness, improves the testing accuracy and efficiency of skid flatness, and enhances the safety and reliability of skid transportation in automobile production lines. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram illustrating an application scenario of the skid flatness detection method provided in an embodiment of the present invention;

[0021] Figure 2 A schematic flowchart illustrating the skid flatness detection method provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic flowchart of the first sub-process in the skid flatness detection method provided in the embodiment of the present invention;

[0023] Figure 4 This is a schematic flowchart of the second sub-process in the skid flatness detection method provided in the embodiment of the present invention;

[0024] Figure 5 This is a schematic flowchart of the third sub-process in the skid flatness detection method provided in the embodiment of the present invention;

[0025] Figure 6 This is a schematic flowchart of the fourth sub-process in the skid flatness detection method provided in the embodiment of the present invention;

[0026] Figure 7 This is a schematic block diagram of the skid flatness detection device provided in an embodiment of the present invention;

[0027] Figure 8 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0030] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0031] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0032] Please see Figure 1 and Figure 2 ,in Figure 1 This is a schematic diagram illustrating an application scenario of the skid flatness detection method provided in this embodiment of the invention. Figure 2This is a schematic flowchart illustrating the skid flatness detection method provided in an embodiment of the present invention. The skid flatness detection method is applied in a terminal or server. The terminal can be a PCL (Point Cloud Library) system connected to a camera or robotic arm. This terminal stores a series of point cloud algorithms and computational strategies, enabling it to acquire point cloud data of the plane to be detected on the crossbeam of the skid, and further process and compute the point cloud data. The camera here is a camera equipped with a three-dimensional imaging sensor, specifically a 3D scanner or an RGB-D (RGB-Depth) camera. This type of camera can project laser light onto the target object and convert the reflected laser beam into three-dimensional coordinate points to form a dataset. Each point in the dataset represents a set of X, Y, and Z geometric coordinates and an intensity value, which records the intensity of the returned signal based on the reflectivity of the object's surface. When these points are combined, a point cloud is formed, i.e., a set of data points representing a 3D shape or object in space. The point cloud data reflects the current shape of the target object. The camera is mounted at the end of the robotic arm, which is a "eye on hand" installation method. The camera can change its shooting position by moving the robotic arm.

[0033] It should be noted that, Figure 1 The image only shows one camera connected to the terminal. In actual operation, multiple cameras can be set up to collect point cloud data from different sleds or different planes to be detected on the crossbeams of the sleds.

[0034] Please see Figure 2 , Figure 2 This is a schematic flowchart of the skid flatness detection method provided in an embodiment of the present invention. Figure 2 As shown, the method is used to detect the flatness of a sled, which includes multiple crossbeams arranged parallel to each other at the top of the sled. Figure 2 As shown, the method includes the following steps S110-S150.

[0035] S110. Collect target point cloud data corresponding to each plane to be detected in each of the crossbeams using a camera. Each crossbeam includes multiple planes to be detected.

[0036] In this embodiment, each crossbeam is pre-divided into multiple detection areas, and the upper surface of each detection area (the surface in direct contact with the vehicle body) corresponds to the detection plane. The camera can collect point cloud data corresponding to each detection plane, preprocess the point cloud data, and extract features to obtain the target point cloud data of the detection plane for subsequent processing. The camera can sequentially collect the target point cloud data of each detection plane, or multiple cameras can simultaneously collect the target point cloud data of each detection plane.

[0037] In one embodiment, please refer to Figure 3 Step S110 may specifically include steps S111-S113:

[0038] S111. Collect the first point cloud data of each of the planes to be detected through the camera.

[0039] When a camera acquires point cloud data of the plane to be detected, it actually obtains the point cloud of all objects within the entire shooting area. Therefore, the first point cloud data acquired will contain not only the point cloud data corresponding to the plane to be detected, but also background point cloud data and redundant noise points. These background point cloud data and noise points are redundant. Preprocessing of the first point cloud data is necessary to remove these redundant points in order to obtain accurate point cloud data of the plane to be detected.

[0040] In one embodiment, the camera is mounted on a robotic arm that can move in response to control commands from a terminal. Please refer to [link to relevant documentation]. Figure 4 Step S111 includes:

[0041] S1111. Send a first motion command to the robotic arm, the first motion command instructing the robotic arm to move the camera to the target position.

[0042] The camera is mounted on a robotic arm using an "eye-on-hand" approach, meaning the camera is fixed relative to the robotic arm, while the movement of the robotic arm causes the camera to change position. The first motion command includes the robotic arm's endpoint and planned path. After receiving the first motion command, the robotic arm moves according to the endpoint and planned path until the camera's position is changed to be above the target position.

[0043] S1112. When the robotic arm moves the camera to the target position, it collects first point cloud data corresponding to the target detection plane through the camera. The target detection plane is the detection plane corresponding to the first point cloud data collected by the camera at the target position.

[0044] The target location is the location where point cloud data can be collected for a specific plane to be detected. At the target location, the camera can collect the first point cloud data for the target plane to be detected. Specifically, a target location with the same relative position can be set for each plane to be detected; that is, the camera's shooting angle and shooting distance relative to its corresponding target plane are the same at each target location. This improves the data correlation between the collected first point cloud data and accelerates the efficiency of subsequent data processing.

[0045] S1113. Determine whether the first point cloud data of each of the planes to be detected has been collected; if yes, proceed to step S1115; if no, proceed to step S1114.

[0046] The camera stops acquiring the first point cloud data only after all the target point cloud data for each plane to be detected has been acquired. Specifically, each plane to be detected can be assigned a number. When the first point cloud data of a target plane to be detected is acquired, the first point cloud data is stored as data with a corresponding number. The number can then be checked to determine whether all the first point cloud data have been acquired.

[0047] S1114. Generate a second motion command and use the second motion command as the first motion command, then return to step S1111.

[0048] If there are still uncollected point cloud data of the target plane, a second motion command can be generated based on the information of the target plane whose first point cloud data has not yet been collected. Specifically, another target plane closest to the previous target target plane whose target point cloud data has been collected can be designated as the new target target target plane. The endpoint of the robotic arm's movement is set as the target position corresponding to the new target target target plane. Simultaneously, a new path is planned, and a second motion command is generated based on the new endpoint and the path plan. After the second motion command is generated, it overwrites the original first motion command, thereby enabling the robotic arm to drive the camera to collect the first point cloud data of the other target plane.

[0049] S1115. Confirm that the collection of the first point cloud data has been completed.

[0050] Once the first point cloud data of each plane to be detected has been collected, it can be determined that the collection of the first point cloud data of each plane to be detected has been completed.

[0051] In this embodiment, when it is determined that the first point cloud data of each of the planes to be detected has been collected, the collection of the first point cloud data is considered complete.

[0052] In this embodiment, the robotic arm drives the camera to move between various target positions, collects target point cloud data of each plane to be detected, and determines whether the collection has been completed, which can ensure the comprehensiveness and orderliness of point cloud data collection.

[0053] S112. Based on a preset redundancy removal strategy, remove redundant points from each of the first point cloud data to obtain the second point cloud data corresponding to each of the first point cloud data.

[0054] The preset redundancy removal strategy is a pre-stored algorithmic strategy that can remove redundant points. Specifically, the preset redundancy removal strategy can be Gaussian bilateral filtering, pass-through filtering, or region of interest methods.

[0055] Gaussian bilateral filtering is a nonlinear filtering method used to smooth point cloud data while preserving edge information. It achieves smoothing of image or point cloud data by combining spatial distance and color difference to calculate the weight of each point in the initial point cloud data. Its basic operation is to calculate the weighted average of all points in the neighborhood of each pixel or point, using this as the new value for that pixel or point. The weight of a point consists of two parts: one is a Gaussian function based on spatial distance, and the other is a Gaussian function based on color difference. Points that are closer in spatial distance and have smaller color differences have greater weights. Gaussian bilateral filtering can effectively remove noise from point cloud data while preserving the edge information of the point cloud.

[0056] Pass-through filtering preprocesses the first point cloud data by setting a threshold range and removing points outside that range. It is typically used to limit the range of point cloud data along a specific axis. For example, pass-through filtering can be used to remove points in the first point cloud data whose Z-axis coordinates are outside a specified range. This effectively removes points far from the region of interest, thereby improving the quality of the first point cloud data.

[0057] Region of Interest (ROI) is a method used to extract a specific region from a point cloud dataset. It preprocesses image or point cloud data by defining an ROI and then retaining only the data within that region. An ROI refers to a specific area in an image or point cloud dataset that contains information of interest, i.e., the specific information needed. In the preprocessing of first-order point cloud data, the ROI can be used to extract specific parts of the data for further analysis and processing. ROIs can be defined in various ways, such as through manual selection, thresholding, or edge detection. Once the ROI is determined, only the data within it can be retained for further analysis and processing. Using ROIs can effectively reduce the amount of data and improve data processing efficiency. It allows for greater focus on the target or feature of interest, thereby improving the accuracy of point cloud data analysis.

[0058] In summary, by using a preset redundancy removal strategy, redundant points in the first point cloud data can be removed to obtain preprocessed second point cloud data, which mainly contains the point cloud data corresponding to the plane to be detected.

[0059] S113. Based on a preset region growth segmentation strategy, extract the target point cloud clusters from each of the second point cloud data, and determine the target point cloud clusters as the target point cloud data.

[0060] In this embodiment, the second point cloud data includes the target point cloud data corresponding to the plane to be detected, but also includes some unnecessary noise data, such as noise data not within the range of the plane to be detected. To remove the remaining noise data and obtain the final target point cloud data, a preset region growth segmentation strategy needs to be invoked. The preset region growth segmentation strategy is a point cloud data processing strategy based on region growth segmentation. The region growth segmentation algorithm can group similar point clouds together to form regions. First, a seed point is found for each region to be segmented as the starting point for growth. Then, points in the neighborhood of the seed point that have the same or similar properties as the seed point are merged into the region where the seed point is located. The new points continue to grow outwards as seeds until no more points that meet the conditions can be included, and a point cloud region is then grown.

[0061] The most prominent point cloud region grown from the second point cloud data is the point cloud region corresponding to the plane to be detected. Among unnecessary noise, some cannot grow into valid point cloud regions, while others grow into smaller point cloud regions compared to the most prominent one. The largest point cloud region grown from the second point cloud data is the target point cloud cluster, which corresponds to the target point cloud data of the plane to be detected on the upper surface of the beam. The target point cloud data includes only the point cloud data corresponding to the beam surface, removing background point cloud data and other existing noise data. Assuming each beam is uniformly divided into three planes to be detected, the area enclosed by the edges of the target point cloud data from the three planes should be the same. However, the X, Y, and Z geometric coordinates and intensity values ​​of each point data point are not identical. These X, Y, and Z geometric coordinates and intensity values ​​also determine the plane equation corresponding to the target point cloud data.

[0062] S120. Determine the first plane equation for each of the target point cloud data according to the preset plane algorithm and the target point cloud data respectively.

[0063] In this embodiment, the preset plane algorithm can be RANSAC (Random Sample Consensus). RANSAC is generally used to estimate the parameters of a mathematical model from observation data containing outliers. It can fit a model by randomly selecting a subset of data and determine the best-fit model through iteration. When applied to target point cloud data, the basic operation of RANSAC is as follows: For a given target point cloud, a set of points is randomly selected as inliers. For example, three points can be selected from the target point cloud (three points can define a plane) as inliers, and an initial plane equation is fitted using these inliers. Specifically, the coordinate data of the three points can be used to fit the initial plane equation. Subsequently, the remaining points in the target point cloud are used to test the initial plane equation, and the points that conform to the initial plane equation are included in the consensus set. Specifically, a consensus set threshold can be set. If the error between the point data and the preset plane equation is less than the consensus set threshold, the point data can be included in the consensus set. If the number of points in the consensus set exceeds the preset threshold, the initial plane equation is considered to have a high degree of fit. Subsequently, the plane equation can be refitted using all point data in the consensus set to improve it. This process is repeated multiple times, generating a new plane equation each time. The plane equation with the best fit is then selected as the first plane equation. Based on the final fitted first plane equation, the shape of the plane to be detected corresponding to the current target point cloud data can be determined, enabling subsequent flatness detection of the plane. Each piece of target point cloud data corresponds to its own first plane equation, which best reflects the planar shape of the plane to be detected.

[0064] In one embodiment, please refer to Figure 5 After determining the first plane equation for each target point cloud data, it is necessary to determine whether the acquired target point cloud data meets the preset standard based on the first plane equation. Only when the acquired target point cloud data meets the preset standard can the target point cloud data be used for flatness detection (execute step 115). When the acquired target point cloud data does not meet the preset standard, it is necessary to reacquire the target point cloud data through the camera (return to execute step S110). The judgment process specifically includes the following steps S310-S360:

[0065] S310. Determine the first centroid of the target point cloud data according to the first plane equation.

[0066] Based on the first plane equation, the first centroid of the plane corresponding to this first plane equation can be determined. In this process, all point data in the target point cloud data must first be projected onto the plane corresponding to the first plane equation. The specific projection operation is as follows: determine the X-axis and Y-axis information corresponding to each point, then substitute the X-axis and Y-axis information corresponding to each point into the first plane equation to calculate the Z-axis data of the projection point corresponding to each point in the first plane equation. Then, replace the Z-axis data in the original point data with the Z-axis data of the projection point to obtain the projection point coordinates of each point in the target point cloud data.

[0067] Subsequently, based on the projection point data of the target point cloud data in the first plane equation, the average coordinates of each projection point are calculated to obtain the coordinates of the first centroid. Specifically, assume there are N projection point data, and the coordinates of each projection point are (X... i Y i Z i ), where i = 1, 2, ..., N. Then the coordinates of the first centroid of the target point cloud data are (X... c Y c Z c ), where X c =X1+X2+…+X N / N, Y c =Y1+Y2+…+Y N / N, Z c =Z1+Z2+…+Z N / N. That is, the X of the first mass. c The coordinates are X for each projection point. i The average value of the coordinates, the Y-axis of the first centroid c The coordinates are Y for each projection point i The average value of the coordinates, Z of the first centroid c The coordinates are Z for each projection point i The average value of the coordinates.

[0068] S320. Transform each of the first centroids from the camera coordinate system to a preset coordinate system to obtain the second centroids of each of the target point cloud data in the preset coordinate system.

[0069] Because the camera acquires target point cloud data from different locations on different planes to be detected, the point cloud data obtained at different locations are all different. Therefore, to synchronize the operation on the first centroid of each plane to be detected, it is necessary to transform the first centroid from its respective camera coordinate system to a preset coordinate system. This transformation can be performed using a hand-eye matrix. This preset coordinate system is a unified, pre-defined coordinate system, whose origin can be set on a fixed reference object, such as a beam. Transforming the first centroid of the target point cloud data corresponding to each plane to be detected into the preset coordinate system allows for transformation and calculation within the same coordinate system. After the first centroid with camera coordinates is transformed into the preset coordinate system, it becomes a second centroid with beam coordinates.

[0070] S330. Determine the line connecting the second centroids in each of the aforementioned beams.

[0071] Since the area covered by each target point cloud is the same, and each target point cloud on a single beam actually corresponds to a region uniformly divided along the axis of the single beam, and the regions corresponding to each target point cloud are collinear, the second centroids corresponding to each target point cloud in a single beam are also collinear. Therefore, the lines connecting the second centroids in each beam can be obtained, resulting in the connecting line corresponding to each beam. The connecting line corresponding to each beam represents the extension direction of each beam.

[0072] S340. Determine whether the maximum included angle between the connecting lines is greater than a preset included angle threshold; if yes, proceed to step S350; if no, proceed to step S360.

[0073] The preset angle threshold is a pre-defined threshold representing a fixed angle between the beams. Optionally, the preset angle threshold can be set to a small threshold such as 0 degrees. Since there can be multiple beams, there may be multiple angles between the connecting lines. By selecting the largest angle for comparison, the maximum difference between the selected angle and the preset angle threshold can be obtained, thus ensuring that the comparison result is the most accurate.

[0074] S350: Reacquire target point cloud data via camera.

[0075] In this embodiment, when it is determined that the maximum included angle between the connecting lines is greater than the preset included angle threshold, the target point cloud data is reacquired by the camera, that is, the execution step S110 is returned.

[0076] Specifically, if the maximum angle between the connecting lines is greater than the preset angle threshold, it indicates that there is an error point within the second centroid obtained at this time, which cannot truly reflect the angular relationship between the beams. In this case, it is necessary to re-acquire the target point cloud data of each plane to be detected in each beam to remove the error.

[0077] S360 uses target point cloud data for flatness detection.

[0078] That is, the flatness coefficient of the skid is determined only when the maximum included angle is less than or equal to the preset included angle threshold.

[0079] Specifically, if the maximum included angle of the connecting lines is less than or equal to the preset included angle threshold, it can be determined that the second centroid at this time can truly reflect the included angle relationship between the measurements. It is the second centroid corresponding to the accurately acquired target point cloud data. Therefore, the acquired target point cloud data can be used for subsequent flatness calculation operations.

[0080] In this embodiment, by comparing the angle between the lines connecting the second centroids to determine errors in the acquisition of target point cloud data, it is possible to ensure that the acquired target point cloud data is correct and to avoid errors that occur when transforming coordinate systems.

[0081] S130. Transform each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain the second plane equations of each of the target point cloud data in the preset coordinate system.

[0082] In this embodiment, because the camera acquires target point cloud data at different locations for different detection planes, the point cloud data obtained at different locations are all different. Therefore, to synchronize the operation of the first plane equation for each detection plane, the first plane equation needs to be transformed from its respective camera coordinate system to a preset coordinate system. Specifically, a hand-eye matrix can be used for this transformation. This preset coordinate system is a unified preset coordinate system, whose origin can be set on a fixed reference object, such as a beam or a robotic arm. Transforming the first plane equation of the target point cloud data corresponding to each detection plane into the preset coordinate system allows the first plane equation to be transformed and calculated within the same coordinate system. After the first plane equation with camera coordinates is transformed into the preset coordinate system, it is transformed into a second plane equation with beam coordinates.

[0083] In one embodiment, please refer to Figure 6 Step S130 may specifically include steps S131-S134:

[0084] S131. Obtain the hand-eye matrix between the camera coordinate system and the preset coordinate system.

[0085] The hand-eye matrix describes the relationship between the camera coordinate system and a preset coordinate system. The preset coordinate system can be the robotic arm coordinate system, representing a fixed coordinate system corresponding to the robotic arm's movement, rather than the constantly changing camera coordinate system. Since the camera and robotic arm are connected via an "eye-on-hand" connection, the hand-eye matrix can be determined through calibration methods. The calibration process typically includes the following steps: First, prepare a calibration board, a flat plate with a special pattern, used to determine the spatial relationship between the camera and the robotic arm. Second, take calibration images using the camera. Before taking the images, place the calibration board at the end of the robotic arm and then take multiple calibration images from different angles. Next, extract feature points from the calibration images. Specifically, computer vision algorithms can be used to extract feature points from the calibration images; these feature points are typically corner points or centers of circles on the calibration board. Finally, the hand-eye matrix can be calculated based on the feature points. Specifically, based on the extracted feature points and the robotic arm's motion information (including translation and rotation), optimization algorithms such as the Tsai-Lenz algorithm and the Horaud algorithm can be used to calculate the hand-eye matrix. This process can determine the relationship between the camera coordinate system and the robotic arm coordinate system, thereby achieving coordinate system transformation of the target point cloud data. In addition, there are several other proven and effective methods for determining the hand-eye matrix, which are not limited in this invention.

[0086] S132. For each of the first plane equations, take at least three non-collinear point data in the first plane equation as the transformation point data group of the first plane equation.

[0087] For each first plane equation, at least three non-collinear point data points can be selected. These point data points are then used as a transformation point data set, with each first plane equation corresponding to its own transformation point data set. The point data points in these transformation point data sets can directly determine a unique first plane equation. Therefore, if coordinate transformations are subsequently performed on the point data points in these transformation point data sets, the transformed transformation point data sets can still determine a unique plane equation.

[0088] S133. Based on the hand-eye matrix, perform matrix multiplication on the camera coordinates of each conversion point data group in the camera coordinate system to obtain the beam coordinates of each conversion point data group in the preset coordinate system.

[0089] Based on the hand-eye matrix, matrix multiplication is performed on the camera coordinates of the point data in each transformation point data group to obtain the beam coordinates of the point data in each transformation point data group. Specifically, the hand-eye matrix is ​​a 4*4 homogeneous transformation matrix. For each point data, assuming the camera coordinates of that point data are (X1, Y1, Z1), during the transformation, it needs to be converted to homogeneous coordinate form, i.e., (X1, Y1, Z1, 1). Subsequently, matrix multiplication is performed on the homogeneous coordinates of the point data to obtain the transformed homogeneous coordinates (X1, Y1, Z1, 1). w Y w Z w Then, dividing the transformed homogeneous coordinates by W yields the transformed beam coordinates (X, W). w / W, Y w / W, Z w / W). By transforming the coordinates of each point in the transformation point data set, the coordinate transformation of the first plane equation can be achieved.

[0090] S134. Based on the coordinates of each beam, determine the second plane equations corresponding to each first plane equation.

[0091] Based on the beam coordinates of each point in the transformed point data group, the second plane equation can be determined in the preset coordinate system. Each second plane equation has the same plane shape as the first plane equation, the only difference being the coordinate system in which it is located.

[0092] In this embodiment, the second plane equations corresponding to multiple target point cloud data are set in the same coordinate system, which enables the comparison of multiple second plane equations and provides a basis for subsequent flatness measurement.

[0093] S140. Determine the Z-axis vector of the beam based on the normal vectors of the equations of each of the second planes.

[0094] In this embodiment, the Z-axis vector corresponding to the beam can be determined using the normal vector of the second plane equation. The Z-axis vector of the reference coordinate system is the Z-axis vector of the coordinate system of the plane containing the upper surface of the beam. The plane determined by the X and Y axes in the reference coordinate system is also the approximate plane containing each plane to be detected; it cannot accurately encompass all the planes to be detected, but only serves as a reference plane. The Z-axis vector is also the normal vector of the plane determined by the X and Y axes in the reference coordinate system.

[0095] In one embodiment, step S140 specifically includes: calculating the normal vector of each second plane equation according to each second plane equation; and averaging the normal vectors of each second plane equation to obtain the Z-axis vector.

[0096] In this embodiment, the normal vectors of each second plane equation are first calculated, then the average value of each normal vector is calculated, and this average value is used as the Z-axis vector of the reference coordinate system. Since the upper surface of the beam is not a definite plane, its plane equation cannot be calculated. Therefore, the Z-axis vector of the reference coordinate system can be determined by averaging the normal vectors of each second plane equation. This Z-axis vector is only a reference value, used to represent the approximate normal vector of the plane on which each plane to be detected lies, and serves as reference data for flatness detection. That is, determining the Z-axis vector also determines the reference coordinate system.

[0097] S150. Determine the flatness coefficient of the skid based on the angle difference between each of the normal vectors and the Z-axis vector.

[0098] In this embodiment, the reference plane defined by the X and Y axes in the reference coordinate system represented by the Z-axis vector may be coplanar or non-coplanar with the planes represented by the respective second plane equations. Therefore, there may be different angular differences between the Z-axis vector and the normal vector of the second plane equation. These angular differences can be used to characterize the degree of deviation between the plane represented by the second plane equation and the reference plane. The larger the angular differences, the greater the deviation between the plane represented by the second plane equation and the reference plane, and the worse the flatness. Conversely, the smaller the angular differences, the smaller the deviation between the plane represented by the second plane equation and the reference plane, and the better the flatness.

[0099] In one embodiment, step S150 specifically includes: obtaining the angle difference between each of the normal vectors and the Z-axis vector; determining the average angle difference between each of the normal vectors and the Z-axis vector based on the angle difference, and using the average angle difference as the flatness coefficient.

[0100] In this embodiment, the average angle difference is calculated by averaging the various angle differences to obtain an average angle difference value. This average angle difference value can be used as a flatness coefficient, which reflects the overall flatness of the crossbeam and thus the flatness of the skid. The magnitude of the flatness coefficient is directly proportional to the flatness of the skid.

[0101] As can be seen, the embodiments of the present invention use a camera to take non-contact photos of the crossbeam in the skid and generate a three-dimensional point cloud. Subsequently, the plane equation of the plane to be tested is fitted using the three-dimensional point cloud. Finally, the skid flatness coefficient is calculated based on the plane equation fitted in the point cloud and the offset of its normal vector. This achieves non-contact measurement of skid flatness, improves the testing accuracy and efficiency of skid flatness, and enhances the safety and reliability of skid transportation in automobile production lines.

[0102] Please see Figure 7 , Figure 7This is a schematic block diagram of a skid flatness detection device 70 provided in an embodiment of the present invention. Figure 7 As shown, corresponding to the above-described skid flatness detection method, the present invention also provides a skid flatness detection device 70. This device includes a unit for performing the above-described skid flatness detection method. This device can be configured in a terminal or server, where the terminal can be a desktop computer, tablet computer, laptop computer, PLC system operating terminal, etc. Specifically, please refer to... Figure 7 The skid flatness detection device 70 includes a data acquisition unit 71, a first determination unit 72, a conversion unit 73, a second determination unit 74, and a third determination unit 75.

[0103] Acquisition unit 71 is used to acquire target point cloud data corresponding to each plane to be detected in each of the beams through a camera;

[0104] The first determining unit 72 is used to determine the first plane equation of each target point cloud data according to the preset plane algorithm and each target point cloud data respectively;

[0105] The transformation unit 73 is used to transform each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain the second plane equations of each of the target point cloud data in the preset coordinate system.

[0106] The second determining unit 74 is used to determine the Z-axis vector of the reference coordinate system based on the normal vector of each of the second plane equations;

[0107] The third determining unit 75 is used to determine the flatness coefficient of the skid based on the angle difference between each of the normal vectors and the Z-axis vector.

[0108] In one embodiment, the acquisition unit 71 is further configured to:

[0109] The camera acquires first point cloud data for each of the planes to be detected.

[0110] Based on a preset redundancy removal strategy, redundant points in each of the first point cloud data are removed to obtain the second point cloud data corresponding to each of the first point cloud data.

[0111] Based on a preset region growth segmentation strategy, target point cloud clusters are extracted from each of the second point cloud data, and the target point cloud clusters are determined as the target point cloud data.

[0112] In one embodiment, the conversion unit 73 is further configured to:

[0113] Calibrate the hand-eye matrix between the camera coordinate system and the preset coordinate system;

[0114] For each of the first plane equations, at least three non-collinear point data points in the first plane equation are respectively used as the transformation point data group of the first plane equation;

[0115] Based on the hand-eye matrix, matrix multiplication is performed on the camera coordinates of each conversion point data group in the camera coordinate system to obtain the beam coordinates of each conversion point data group in the preset coordinate system.

[0116] Based on the coordinates of each beam, determine the second plane equation corresponding to each first plane equation.

[0117] In one embodiment, the second determining unit 74 is further configured to:

[0118] Based on the equations of each second plane, the normal vectors of each second plane equation are calculated.

[0119] The Z-axis vector is obtained by averaging the normal vectors of each second plane equation.

[0120] In one embodiment, the third determining unit 75 is further configured to:

[0121] Obtain the angle differences between each of the normal vectors and the Z-axis vector;

[0122] The average angle difference between each normal vector and the Z-axis vector is determined based on the angle difference, and the average angle difference is used as the flatness coefficient.

[0123] In one embodiment, the skid flatness detection device 70 further includes a monitoring unit, which is used for:

[0124] The first centroid is determined based on the first plane equation;

[0125] Transform each of the first centroids from the camera coordinate system to a preset coordinate system to obtain the second centroids of each of the target point cloud data in the preset coordinate system;

[0126] Determine the line connecting the second centroids in each of the aforementioned beams;

[0127] If the maximum included angle between the connecting lines is greater than the preset included angle threshold, then return to start the acquisition unit 71;

[0128] If the maximum included angle is less than or equal to the preset included angle threshold, then the third determination unit 75 is activated.

[0129] In one embodiment, the skid flatness detection device 70 further includes a control unit, which is used for:

[0130] Send a first motion command to the robotic arm, the first motion command instructing the robotic arm to move the camera to the target position;

[0131] When the robotic arm moves the camera to the target position, it collects target point cloud data corresponding to the target detection plane through the camera. The target detection plane is the detection plane corresponding to the target point cloud data collected by the camera at the target position.

[0132] Determine whether the target point cloud data of each of the planes to be detected has been collected;

[0133] If the target point cloud data of any of the planes to be detected has not been fully collected, a second motion command is generated and used as the first motion command. The process then returns to the step of sending the first motion command to the robotic arm until the target point cloud data of each of the planes to be detected has been fully collected.

[0134] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned skid flatness detection device 70 and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0135] The aforementioned skid flatness detection device can be implemented as a computer program, which can perform tasks such as... Figure 8 It runs on the computer device shown.

[0136] Please see Figure 8 , Figure 8 This is a schematic block diagram of a computer device 800 provided in an embodiment of the present invention. The computer device can be a terminal or a server. The terminal can be an electronic device with image display capabilities, such as a smartphone, tablet computer, laptop computer, desktop computer, personal digital assistant, or wearable device. The server can be a standalone server or a server cluster composed of multiple servers.

[0137] Please see Figure 8 The computer device 800 includes a processor 802 connected via a system bus, as well as a memory and a network interface 805, wherein the memory may include a storage medium 803 and internal memory 804.

[0138] The storage medium 803 may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause the processor to perform a skid flatness detection method.

[0139] The processor 802 provides computing and control capabilities to support the operation of the entire computer device 800.

[0140] The internal memory 804 provides an environment for the operation of the computer program 8032 in the storage medium 803. When the computer program is executed by the processor 802, the processor 802 can execute a skid flatness detection method.

[0141] This network interface 805 is used for network communication with other devices. Those skilled in the art will understand that... Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 800 to which the present application is applied. The specific computer device 800 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] The processor 802 is used to run a computer program 8032 stored in the memory to perform the following steps:

[0143] The target point cloud data corresponding to each plane to be detected in each of the crossbeams are collected by a camera, and each of the crossbeams includes multiple planes to be detected.

[0144] The first plane equation of each target point cloud data is determined according to the preset plane algorithm and the target point cloud data respectively;

[0145] Transform each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain the second plane equations of each of the target point cloud data in the preset coordinate system;

[0146] Based on the normal vectors of each of the second plane equations, determine the Z-axis vector of the reference coordinate system;

[0147] The flatness coefficient of the skid is determined based on the angle difference between each of the normal vectors and the Z-axis vector.

[0148] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0149] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0150] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the following steps:

[0151] The target point cloud data corresponding to each plane to be detected in each of the crossbeams are collected by a camera, and each of the crossbeams includes multiple planes to be detected.

[0152] The first plane equation of each target point cloud data is determined according to the preset plane algorithm and the target point cloud data respectively;

[0153] Transform each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain the second plane equations of each of the target point cloud data in the preset coordinate system;

[0154] Based on the normal vectors of each of the second plane equations, determine the Z-axis vector of the reference coordinate system;

[0155] The flatness coefficient of the skid is determined based on the angle difference between each of the normal vectors and the Z-axis vector.

[0156] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0157] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0158] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0159] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0160] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0161] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method of detecting flatness of a skid, characterized by, The method is used to detect the flatness of a skid, the skid comprising multiple crossbeams, and the method includes: The target point cloud data corresponding to each plane to be detected in each of the crossbeams are collected by a camera, and each of the crossbeams includes multiple planes to be detected. The first plane equation of each target point cloud data is determined according to the preset plane algorithm and the target point cloud data respectively; Transform each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain the second plane equations of each of the target point cloud data in the preset coordinate system; Based on the normal vectors of each of the second plane equations, the Z-axis vector of the reference coordinate system is determined; wherein, the Z-axis vector of the reference coordinate system is the Z-axis vector of the coordinate system of the plane containing the upper surface of the beam. The flatness coefficient of the skid is determined based on the angle difference between each of the normal vectors and the Z-axis vector; Determining the Z-axis vector of the reference coordinate system based on the normal vectors of each of the second plane equations includes: Based on the equations of each second plane, the normal vectors of each second plane equation are calculated. The Z-axis vector is obtained by averaging the normal vectors of each second plane equation. The step of determining the flatness coefficient of the skid based on the angle difference between each of the normal vectors and the Z-axis vector includes: Obtain the angle differences between each of the normal vectors and the Z-axis vector; The average angle difference between each normal vector and the Z-axis vector is determined based on the angle difference, and the average angle difference is used as the flatness coefficient.

2. The sled flatness detection method of claim 1, wherein, The step of acquiring target point cloud data corresponding to each plane to be detected in each of the beams via a camera includes: The camera acquires first point cloud data for each of the planes to be detected. Based on a preset redundancy removal strategy, redundant points in each of the first point cloud data are removed to obtain the second point cloud data corresponding to each of the first point cloud data. Based on a preset region growth segmentation strategy, target point cloud clusters are extracted from each of the second point cloud data, and the target point cloud clusters are determined as the target point cloud data.

3. The method for detecting the flatness of a skid according to claim 2, characterized in that, The camera is mounted on a robotic arm; the acquisition of first point cloud data for each of the planes to be detected via the camera includes: Send a first motion command to the robotic arm, the first motion command instructing the robotic arm to move the camera to the target position; When the robotic arm moves the camera to the target position, it collects first point cloud data corresponding to the target detection plane through the camera. The target detection plane is the detection plane corresponding to the first point cloud data collected by the camera at the target position. Determine whether the first point cloud data of each of the planes to be detected has been collected; If it is determined that the first point cloud data of the plane to be detected has not been fully collected, a second motion command is generated and used as the first motion command. The process then returns to the step of sending the first motion command to the robotic arm until the first point cloud data of each plane to be detected has been fully collected.

4. The method for detecting the flatness of a skid according to claim 1, characterized in that, The step of transforming each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain the second plane equations of each of the target point cloud data in the preset coordinate system includes: Calibrate the hand-eye matrix between the camera coordinate system and the preset coordinate system; For each of the first plane equations, at least three non-collinear point data points in the first plane equation are respectively used as the transformation point data group of the first plane equation; Based on the hand-eye matrix, matrix multiplication is performed on the camera coordinates of each conversion point data group in the camera coordinate system to obtain the beam coordinates of each conversion point data group in the preset coordinate system. Based on the coordinates of each beam, determine the second plane equation corresponding to each first plane equation.

5. The sled flatness detection method of any one of claims 1 to 4, wherein, After calculating the first plane equation for each of the target point cloud data based on each of the target point cloud data, the method further includes: The first centroid of the target point cloud data is determined based on the first plane equation; Transform each of the first centroids from the camera coordinate system to a preset coordinate system to obtain the second centroids of each of the target point cloud data in the preset coordinate system; Determine the line connecting the second centroids in each of the aforementioned beams; If the maximum included angle between the connecting lines is greater than the preset included angle threshold, then return to the step of collecting the target point cloud data corresponding to each plane to be detected in each of the beams by means of the camera; The step of determining the flatness coefficient of the skid based on the angle difference between each of the normal vectors and the Z-axis vector includes: If the maximum included angle is less than or equal to the preset included angle threshold, the flatness coefficient of the skid is determined based on the angle difference between each normal vector and the Z-axis vector.

6. A sled flatness detection device, characterized by, The skid flatness detection device is used to detect the flatness of the skid. The skid includes multiple crossbeams. The skid flatness detection device includes: The acquisition unit is used to acquire target point cloud data corresponding to each plane to be detected in each of the crossbeams through a camera. Each of the crossbeams includes multiple planes to be detected. The first determining unit is used to determine the first plane equation of each target point cloud data according to the preset plane algorithm and each target point cloud data respectively; A transformation unit is used to transform each of the first plane equations from the camera coordinate system to a preset coordinate system to obtain the second plane equations of each of the target point cloud data in the preset coordinate system. The second determining unit is used to determine the Z-axis vector of the reference coordinate system based on the normal vector of each of the second plane equations; wherein the Z-axis vector of the reference coordinate system is the Z-axis vector of the coordinate system of the plane on which the upper surface of the beam is located. The third determining unit is used to determine the flatness coefficient of the skid based on the angle difference between each of the normal vectors and the Z-axis vector; The second determining unit is specifically used for: Based on the equations of each second plane, the normal vectors of each second plane equation are calculated. The Z-axis vector is obtained by averaging the normal vectors of each second plane equation. The third determining unit is specifically used for: Obtain the angle differences between each of the normal vectors and the Z-axis vector; The average angle difference between each normal vector and the Z-axis vector is determined based on the angle difference, and the average angle difference is used as the flatness coefficient.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the skid flatness detection method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, cause the processor to perform the skid flatness detection method as described in any one of claims 1-5.

Images