Paint surface defect detection method, medium and equipment

Abstract

According to the paint surface defect detection method, the medium and the equipment, the initial virtual scene is created by acquiring the internal reference matrixes of the acquisition equipment and the first conversion matrixes among the acquisition equipment, and the virtual acquisition equipment and the automobile model are imported according to the internal reference matrixes of the acquisition equipment and the first conversion matrixes among the acquisition equipment, so that the paint surface defects are detected. Obtaining a relative pose between the calibrated automobile and each acquisition device according to the prior engineering data, correcting a relative pose between each virtual acquisition device and the automobile model according to the relative pose between the calibrated automobile and each acquisition device, obtaining a final virtual scene, obtaining a plurality of groups of current automobile 2D images, and obtaining the current automobile 2D images. And obtaining the coordinates of the defect points in each group of 2D images, and mapping the coordinates to an automobile model to obtain the coordinates and normal directions of the defect points. The problems of low efficiency, poor accuracy, insufficient compatibility, limited precision and the like of automobile paint surface defect detection in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a paint surface defect detection method, medium and equipment. Background Art

[0002] In the field of automobile manufacturing, the painting process is a key step to ensure the beauty and performance of the car body. The quality of the paint surface is not only related to the appearance of the vehicle, but also directly affects its performance indicators such as wear resistance and corrosion resistance. However, traditional manual inspection methods have many shortcomings, such as visual fatigue, subjective misjudgment, low efficiency and poor working environment, which limit the improvement of production quality and efficiency. The existing automated inspection methods for detecting defects in automobile paint, such as the method of using a robotic arm to carry a camera light source for inspection, still have the following problems:

[0003] Long testing time: The car body needs to remain still at the testing station, which results in a long testing time and affects the overall rhythm of the production line.

[0004] Poor compatibility: The tunnel-type body paint defect detection method is cumbersome during the calibration process and has poor compatibility with new models, which is not conducive to quickly adapting to changes in the production line.

[0005] Insufficient accuracy: The existing tunnel method only relies on the encoder information of the body rail when calculating the position change of the body, resulting in poor accuracy and unable to meet the needs of high-precision detection.

[0006] Therefore, how to improve the existing technology for detecting automobile paint defects, such as low efficiency, poor accuracy, insufficient compatibility and limited precision, is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0007] Based on this, the purpose of this application is to provide a paint defect detection method, medium and equipment to solve at least one technical problem mentioned in the above background technology.

[0008] In a first aspect, the present application provides a paint surface defect detection method, comprising:

[0009] Collecting several groups of calibrated automobile surface images to obtain the internal parameter matrix of each acquisition device and the first conversion matrix between each acquisition device;

[0010] Create an initial virtual scene, and import virtual acquisition devices and a car model according to the internal parameter matrix of each acquisition device and the first conversion matrix between each acquisition device;

[0011] The 3D surface point cloud of the calibration car is obtained according to the prior engineering data, and several 2D image points in any surface image and the corresponding 3D surface point cloud on the calibration car are selected to construct feature point pairs to obtain the relative position and posture between the calibration car and each acquisition device;

[0012] Correct the relative postures between each virtual acquisition device and the car model according to the relative postures between the calibrated car and each acquisition device to obtain the final virtual scene;

[0013] Acquire several groups of current car 2D images, obtain the coordinates of the defect points in each group of 2D images, and map them to the car model to obtain the coordinates and normal of the defect points.

[0014] Furthermore, the step of obtaining the internal parameter matrix of each acquisition device and the first conversion matrix between each acquisition device includes:

[0015] Each acquisition device is installed at a set position, and the calibration car is controlled to move in a set direction. Each acquisition device scans the surface of the calibration car at a preset sampling distance to obtain several groups of surface images of the calibration car;

[0016] According to the surface images of all calibrated cars, several groups of relative postures between the calibrated cars and the acquisition devices are obtained, and each acquisition device is converted to the same coordinate system to obtain several relative acquisition devices;

[0017] Performing image analysis on the surface images of each group of calibrated cars to obtain image feature points in each group of surface images;

[0018] The image feature points in the coordinate systems of the relative acquisition devices are three-dimensionally reconstructed to obtain the first conversion matrix between the coordinate systems of the relative acquisition devices and the intrinsic parameter matrix of each relative acquisition device.

[0019] Furthermore, the steps of creating an initial virtual scene and importing virtual acquisition devices and a car model according to the internal parameter matrix of each acquisition device and the first conversion matrix between the acquisition devices include:

[0020] Import the acquisition equipment model of the same model as the actual scene and the design digital model of the calibrated car into the virtual scene to obtain the virtual acquisition equipment and car model;

[0021] The virtual acquisition device parameters are configured according to the internal parameter matrix, and the relative positions and postures between the virtual acquisition devices in the virtual scene are obtained according to the first conversion matrix;

[0022] According to the position of the calibrated car in the real scene, the corresponding car model is placed in the virtual scene.

[0023] Furthermore, the step of obtaining the relative position and posture between the calibrated vehicle and each acquisition device includes:

[0024] The 3D surface point cloud of the calibrated car is obtained based on prior engineering data;

[0025] Select several 2D image points and corresponding 3D surface point clouds in any surface image to construct several feature point pairs;

[0026] The feature point pairs are reprojected to obtain a second transformation matrix between the acquisition device coordinate system corresponding to the surface image and the calibration vehicle coordinate system, and the relative posture between the calibration vehicle and each acquisition device is obtained according to the first transformation matrix between each acquisition device.

[0027] Furthermore, the step of selecting a number of 2D image points in any surface image and the corresponding 3D surface point cloud to construct a number of feature point pairs includes:

[0028] Obtain the coordinates of 2D feature points in the surface image according to the graphic detection algorithm;

[0029] Convert the 3D surface point cloud into a triangular mesh to obtain the characteristic hole area points;

[0030] Fit a plane to the characteristic hole area points and extract the plane contour;

[0031] The center of the circle is fitted to the plane contour to obtain 3D feature points, so as to construct a number of feature point pairs based on each 2D feature point and the corresponding 3D feature point.

[0032] Further, the step of obtaining the coordinates and normal of the defect point includes:

[0033] Build and train a neural network model that takes 2D images as input and defect point coordinates as output;

[0034] Obtain several sets of current car 2D images, and input all 2D images into the neural network model to obtain the coordinates of all defect points in each 2D image;

[0035] Project the coordinates of each defect point onto the car model to obtain the corresponding 3D coordinates, which are the defect model points;

[0036] The coordinates of several point clouds closest to each defect model point are obtained, and the normal direction of each defect model point is calculated.

[0037] Furthermore, the step of obtaining the normal direction of each defect model point may optionally include:

[0038] Set the neighborhood size and obtain the coordinates of all points within the set neighborhood of each defect model point to construct a covariance matrix;

[0039] Perform eigenvalue decomposition on the covariance matrix corresponding to each defect model point to obtain several eigenvalues ​​corresponding to each defect model point and eigenvectors corresponding to the eigenvalues;

[0040] Obtain the minimum eigenvalue corresponding to each defect model point, and use the eigenvector corresponding to the minimum eigenvalue as the normal of each defect model point.

[0041] Further, the covariance matrix is:

[0042]

[0043] where C is the covariance matrix, k is the set neighborhood size, N is the number of points within the k-neighborhood of each defect model point, p i is the coordinate of the i-th point, 0 < i ≤ N, u is the average coordinate of all points within the k-neighborhood, and T is the matrix transpose symbol.

[0044] In a second aspect, the present application also provides a computer storage medium storing executable program code; the executable program code is used to execute the paint defect detection method described in any item of the first aspect.

[0045] In a third aspect, the present application also provides a terminal device including a memory and a processor; the memory stores program code executable by the processor; the program code is used to execute the paint defect detection method described in any item of the first aspect.

[0046] A paint defect detection method, medium, and device provided by the present invention collect several groups of calibrated automobile surface images to obtain the internal parameter matrix of each acquisition device and the first transformation matrix between each acquisition device, create an initial virtual scene, import virtual acquisition devices and automobile models according to the internal parameter matrix of each acquisition device and the first transformation matrix between each acquisition device, and improve the efficiency of automobile paint defect detection by constructing an automobile model in the virtual scene. It is not limited to a certain type of vehicle, improving compatibility. Obtain the 3D surface point cloud of the calibrated automobile according to prior engineering data, select several 2D image points in any one surface image and the corresponding 3D surface point cloud on the calibrated automobile to construct feature point pairs, obtain the relative pose between the calibrated automobile and each acquisition device, correct the relative pose between each virtual acquisition device and the automobile model according to the relative pose between the calibrated automobile and each acquisition device to obtain the final virtual scene, obtain several groups of current automobile 2D images, obtain the coordinates of defect points in each group of 2D images, and map them to the automobile model to obtain the coordinates and normal directions of the defect points. Mapping the defect points in the 2D image to the 3D model can more accurately locate the defects and improve the accuracy of detection. It solves the problems of low efficiency, poor accuracy, insufficient compatibility, and limited precision in detecting automobile paint defects in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a flowchart of the paint defect detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] It should be noted that if the embodiments of the present invention involve directional indications, such as up, down, left, right, front, back, etc., then the directional indication is only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly. In addition, if the embodiments of the present invention involve descriptions of "first, second", "S1, S2", "step one, step two", etc., then such descriptions are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of indicated technical features or indicating the execution order of the method, etc. Those skilled in the art can understand that anything that does not violate the gist of the invention under the technical concept of the invention should be included in the protection scope of the present invention.

[0050] like Figure 1 As shown, the present invention provides a paint surface defect detection method:

[0051] S1: Collecting several groups of surface images of calibrated vehicles to obtain the internal parameter matrix of each acquisition device and the first conversion matrix between each acquisition device;

[0052] Specifically, it is optional but not limited to setting a sampling distance, installing a coding pattern on the surface of a car, and optionally using the center point or corner point of the coding pattern as an image feature point, controlling the car to move within the scanning range of a collection device, and each time the car moves the sampling distance, the collection device scans the surface of the car once to obtain a set of car surface images, until the car leaves the scanning range of the collection device to obtain several sets of car surface images, performing image analysis on all car surface images according to any existing technical solution, and obtaining the coordinates and corresponding numbers of the image feature points in each surface image in the coordinate system of each collection device, so as to obtain the internal parameter matrix of each collection device and the first conversion matrix between each collection device; the sampling distance is arbitrarily set by a person skilled in the art; the coding pattern includes but is not limited to structured light coding patterns such as two-dimensional codes, stepping stripe patterns, Gray code stripe patterns, sinusoidal stripe patterns, binary stripe patterns, and phase-shifted stripe patterns that can be used for pixel-by-pixel encoding; the collection device may optionally include commonly used collection devices such as digital cameras, video cameras, 3D line scan cameras, 3D structured light cameras, and TOF cameras.

[0053] Preferably, the step of collecting several groups of surface images of the calibrated vehicle to obtain the intrinsic parameter matrix of each acquisition device and the first conversion matrix between each acquisition device may optionally include:

[0054] S11: installing each acquisition device at a set position, controlling the calibration car to move in a set direction, and each acquisition device scans the surface of the calibration car at a preset sampling distance to obtain several groups of surface images of the calibration car;

[0055] Specifically, it is optional but not limited to setting the position, direction and preset sampling distance, installing the acquisition equipment at the set position, and then controlling the calibration car to move along the set direction, performing the first scan on the calibration car when it enters the tunnel, and then scanning the surface of the calibration car at intervals of the preset sampling distance. Each scan can obtain a set of corresponding images until the calibration car leaves the tunnel, and then several sets of surface image data of the calibration car can be obtained; the set position, direction and preset sampling distance are arbitrarily set by technical personnel in this field.

[0056] By way of example, a calibration system may be optionally provided, including a tunnel, several acquisition devices, a light source, a vehicle body, and an encoder. The acquisition device is installed in a cross-section in the middle of the tunnel, the light source is evenly installed on the tunnel surface, and a preset coding pattern is placed on the vehicle body so that the vehicle body moves along the tunnel. The moving distance of the vehicle body is obtained by the encoder so that the acquisition device scans the vehicle body at set distances to obtain a group of surface images of the vehicle. Each acquisition device scans the surface of the vehicle body several times to obtain several groups of surface images of the vehicle in the coordinate system of each acquisition device.

[0057] S12: obtaining a plurality of sets of relative postures between calibration vehicles and acquisition devices according to the surface images of all calibration vehicles, and converting each acquisition device into the same coordinate system to obtain a plurality of relative acquisition devices;

[0058] Specifically, according to step S11, several groups of surface images of the calibrated car in the acquisition device coordinate system can be obtained. The relative posture of the calibrated car and the corresponding acquisition device in each surface image is different, so several pairs of relative postures between the calibrated car and each acquisition device can be obtained. A calibration coordinate system is constructed with any point on the calibrated car as the origin. The acquisition devices corresponding to the calibrated car in all surface images are converted to the calibration coordinate system, and several relative acquisition devices can be obtained.

[0059] For example, taking an acquisition device as an example, the calibration car is controlled to move along the set direction, and the acquisition device collects surface images of the calibration car three times respectively. Each surface image corresponds to a different relative posture between the calibration car and the acquisition device. If the relative posture between the acquisition device and the calibration car is converted to the same coordinate system each time the calibration car surface is scanned, three acquisition devices in the same coordinate system can be obtained, which are relative acquisition devices.

[0060] S13: performing image analysis on the surface images of each group of calibrated vehicles to obtain image feature points in each group of surface images;

[0061] Specifically, it is optional but not limited to using the corner points or center points of the coding pattern as feature points, and performing image analysis on the surface images of each group of calibrated vehicles according to any existing image analysis method to obtain image feature points in each surface image.

[0062] S14: Perform three-dimensional reconstruction on the image feature points in the coordinate systems of the relative acquisition devices to obtain a first conversion matrix between the coordinate systems of the relative acquisition devices and an intrinsic parameter matrix of each relative acquisition device.

[0063] Specifically, it is optional but not limited to performing three-dimensional reconstruction of the image feature points in the coordinate system of each relative acquisition device by any existing three-dimensional reconstruction method, so as to obtain the first transformation matrix between the coordinate systems of each relative acquisition device and the intrinsic parameter matrix of each relative acquisition device; the three-dimensional reconstruction method may optionally include colmap-based methods, OpenMVG, OpenCV's SfM module, Meshroom and other commonly used methods and software for three-dimensional reconstruction.

[0064] S2: Create an initial virtual scene, import virtual acquisition devices and car models according to the internal parameter matrix of each acquisition device and the first conversion matrix between each acquisition device;

[0065] Specifically, it is optional but not limited to that technical personnel in this field place corresponding virtual acquisition devices and car models in the virtual scene according to the positions of each relative acquisition device and the calibrated car in step S1, and correct the relative posture of each virtual acquisition device according to the first transformation matrix between the coordinate systems of each relative acquisition device in step S1, and configure the corresponding virtual acquisition device parameters according to the intrinsic parameter matrix of each relative acquisition device, so that the image information collected by the acquisition device in the virtual scene is the same as that in the real scene.

[0066] Preferably, the step of creating an initial virtual scene and importing a virtual acquisition device and a car model according to an internal parameter matrix of each acquisition device and a first conversion matrix between each acquisition device may optionally include:

[0067] S21: importing a collection device model of the same model as the actual scene and a design digital model of a calibrated car into the virtual scene to obtain a virtual collection device and car model;

[0068] Specifically, it is optional but not limited to importing the design digital model of the acquisition device and the calibrated car of the same model as the actual scene from the virtual software platform library into the initial virtual scene to obtain the virtual acquisition device and the car model; the virtual software platform library may optionally include common virtualization platforms such as Unity, Unreal Engine, Godot, CryEngine, etc., and the order of setting may be arbitrarily set by technical personnel in this field.

[0069] S22: configuring the virtual acquisition device parameters according to the internal parameter matrix, and obtaining the relative positions and postures between the virtual acquisition devices in the virtual scene according to the first conversion matrix;

[0070] Specifically, it is optional but not limited to setting corresponding virtual acquisition device parameters according to the internal parameter matrix of each relative acquisition device calibrated in step S1, so that when the virtual acquisition device shoots in the virtual scene, it can achieve the same shooting effect as the acquisition device in the real scene, so as to simulate various operations of the acquisition device in the real scene and reduce costs. At the same time, since the first transformation matrix between the relative acquisition devices is also obtained in step S1, the relative posture between the virtual acquisition devices can be obtained according to the first transformation matrix.

[0071] S23: placing a corresponding car model in the virtual scene according to the position of the car calibrated in the real scene.

[0072] Specifically, it is optional but not limited to that a person skilled in the art places a car model in the virtual scene according to the position of the calibrated car in the real scene. It is worth noting that since the subsequent steps also include posture correction of the car model in the virtual scene, when a person skilled in the art places the car model, the position of the car model does not deviate too much from the initial position of the calibrated car in the real scene.

[0073] S3: Obtain the 3D surface point cloud of the calibration car according to the prior engineering data, select a number of 2D image points in any surface image and the corresponding 3D surface point cloud on the calibration car to construct feature point pairs, and obtain the relative position and posture between the calibration car and each acquisition device;

[0074] Specifically, it is optional but not limited to obtaining a 3D surface point cloud of the calibrated car based on prior engineering data, and then selecting any number of 2D image points in any one of the surface images obtained in step S1, and obtaining a corresponding number of 3D point clouds in the 3D surface point cloud based on the one-to-one correspondence between the pixel points in the surface image of the calibrated car and the 3D surface point cloud, so as to construct feature point pairs based on the one-to-one correspondence between the 2D image points and the 3D surface point cloud, so as to obtain a second transformation matrix between the calibrated car and the acquisition device corresponding to the selected surface image, and obtain the current posture of the calibrated car in the coordinate system of each acquisition device based on the first transformation matrix between the acquisition devices; the prior engineering data may optionally include engineering data such as automobile body digital model design and assembly information.

[0075] Preferably, since there are several characteristic holes on the car, and the characteristic holes are usually relatively stable features and can still maintain good recognition capabilities under the influence of noise and interference, thereby improving the robustness of the system, when constructing feature point pairs based on the one-to-one correspondence between each 2D image point and the 3D model point, it is preferred to use the characteristic holes in the surface image as the selected 2D image points.

[0076] Further preferably, the step of obtaining a 3D surface point cloud of the calibrated car according to the prior engineering data, selecting a number of 2D image points in any surface image and the corresponding 3D surface point cloud on the calibrated car to construct feature point pairs, and obtaining the relative position and posture between the calibrated car and each acquisition device may optionally include:

[0077] S31: Obtain the 3D surface point cloud of the calibrated car according to the prior engineering data, and construct the calibrated car coordinate system;

[0078] Specifically, it is possible to directly obtain the 3D surface point cloud of the calibrated car based on prior engineering data such as known car body digital model design and assembly information, and construct a calibrated car coordinate system with any point in the 3D surface point cloud as the origin.

[0079] S32: Select a number of 2D image points and corresponding 3D surface point clouds in any surface image to construct a number of feature point pairs;

[0080] Specifically, since there are several feature areas on the car body, such as feature holes, the center of the feature hole can be selected as the feature point, and several feature points in any surface image and corresponding feature points in the 3D surface point cloud can be obtained to construct several feature point pairs.

[0081] Preferably, when constructing feature point pairs using feature points in the surface image and the 3D surface point cloud, step S32 may optionally include:

[0082] S321: Obtaining coordinates of 2D feature points in the surface image according to a graphic detection algorithm;

[0083] Specifically, since there are several characteristic holes of different shapes on the vehicle body, the center point of each characteristic hole in the surface image can be obtained by a graphic detection algorithm, which is the 2D characteristic point coordinate. Preferably, an ellipse detection algorithm is used to detect the circular characteristic holes in the surface image to obtain the center point of the circular characteristic hole as the 2D characteristic point.

[0084] S322: Convert the 3D surface point cloud into a triangular mesh to obtain feature hole area points;

[0085] S323: fitting a plane to the characteristic hole area points and extracting the plane contour;

[0086] S324: Fitting the center of the circle to the plane contour to obtain 3D feature points, and constructing a plurality of feature point pairs based on each 2D feature point and the corresponding 3D feature point.

[0087] Specifically, Delaunay triangulation can be used to convert the 3D surface point cloud into a triangular mesh to ensure that the mesh can cover the surrounding area of ​​the characteristic hole. Then, a technician in this field can select points around the characteristic hole in the triangular mesh as characteristic hole area points, and then use a plane fitting method such as the least squares method or the RANSAC algorithm to perform plane fitting on the selected characteristic hole area points to obtain the local space plane equation of the characteristic hole. Optionally, the boundary points of the characteristic hole can be identified through boundary extraction algorithms such as normal calculation and Canny edge detection method to connect the boundary points to form a 3D contour of the characteristic hole. Finally, the least squares method is used to fit the circular model, and the 3D center of the characteristic hole is calculated based on the extracted 3D contour points to obtain the 3D feature points.

[0088] S33: Reprojecting the feature point pairs to obtain a second transformation matrix between the acquisition device coordinate system corresponding to the surface image and the calibration vehicle coordinate system, and obtaining the relative posture between the calibration vehicle and each acquisition device according to the first transformation matrix between each acquisition device.

[0089] Specifically, it is optional but not limited to reprojecting the feature point pairs obtained in step S32 to minimize the distance between corresponding points, thereby obtaining a second transformation matrix between the acquisition device coordinate system corresponding to the surface image and the calibration vehicle coordinate system, that is, obtaining the relative pose between the calibration vehicle and the acquisition device. Since step S1 has obtained the relative pose between each acquisition device, the relative pose between the calibration vehicle and each acquisition device can be obtained based on the first transformation matrix between each acquisition device.

[0090] S4: Correcting the relative postures between each virtual acquisition device and the car model according to the relative postures between the calibrated car and each acquisition device to obtain the final virtual scene;

[0091] Specifically, it is optional but not limited to correcting the posture of the car model in the virtual scene based on the relative posture between the calibrated car and the acquisition device obtained in step S3, so that the relative posture between the devices in the virtual scene is the same as the relative posture between the devices in the real scene, so that when performing simulation operations in the virtual scene, the same results as in the real scene can be obtained, which greatly saves operating costs, improves operating accuracy, and improves operating efficiency.

[0092] S5: Acquire several groups of current car 2D images, obtain the coordinates of the defect points in each group of 2D images, and map them to the car model to obtain the coordinates and normal direction of the defect points.

[0093] Specifically, after obtaining the final virtual scene, it is optional but not limited to controlling the current car to be collected to move in a set direction, collecting several groups of surface images of the current car according to step S1, and then identifying the coordinates of the defect points in all surface images, mapping them to the car model, and obtaining the 3D coordinates of the defect points. Then, the coordinates of several point clouds closest to each defect model point are obtained, and the covariance equation is constructed and eigenvalue decomposition is performed on it, so as to calculate the normal of each defect model point.

[0094] Preferably, the step of acquiring several groups of current car 2D images, obtaining the coordinates of defect points in each group of 2D images, and mapping them to the car model to obtain the coordinates and normal of the defect points in the car body coordinate system may optionally include:

[0095] S51: Build and train a neural network model with 2D images as input and defect point coordinates as output;

[0096] Specifically, it is optional but not limited to constructing and training a neural network model with a 2D image as input and defect point coordinates as output according to any existing neural network model and prior art solution. It is worth noting that this step is a preparatory work, which can be constructed and trained in advance, as long as it is constructed and trained before the neural network model is used for prediction in step S52.

[0097] S52: Acquire several groups of current automobile 2D images, and input all 2D images into the neural network model to obtain the coordinates of all defect points in each 2D image;

[0098] Specifically, it is optional but not limited to obtaining several groups of current car 2D images according to step S1, and then inputting all the obtained 2D images one by one into the neural network model obtained in step S51 to obtain the coordinates of the defect point in each 2D image in the surface image coordinate system.

[0099] S53: Projecting the coordinates of each defect point onto the car model to obtain corresponding 3D coordinates, which are defect model points;

[0100] Specifically, it is possible to convert it into a ray according to the internal parameters of the acquisition device through ray tracing, obtain the intersection point between the ray and the vehicle model, and obtain the 3D model point corresponding to the defect point, that is, the defect model point.

[0101] S54: Obtain the coordinates of several point clouds closest to each defect model point, and calculate the normal vectors of each defect model point.

[0102] Specifically, step S54 may include:

[0103] S541: Set the neighborhood size, obtain the coordinates of all points within the set neighborhood of each defect model point, and construct a covariance matrix;

[0104] Specifically, it is possible but not limited to setting the neighborhood size to k, obtaining the coordinates of all points within the k-neighborhood of each defect model point, and calculating the average coordinate u to construct the covariance matrix corresponding to each defect model point.

[0105] Preferably, the covariance matrix can be expressed as Equation 5-1:

[0106]

[0107] where C is the covariance matrix, N is the number of points within the k-neighborhood of each defect model point, p i is the coordinate of the i-th point, 0 < i ≤ N, u is the average coordinate of all points within the k-neighborhood, and T is the matrix transpose symbol.

[0108] S542: Perform eigenvalue decomposition on the covariance matrix corresponding to each defect model point to obtain several eigenvalues corresponding to each defect model point and the eigenvectors corresponding to the eigenvalues;

[0109] S543: Obtain the minimum eigenvalue corresponding to each defect model point, and use the eigenvector corresponding to the minimum eigenvalue as the normal vector of each defect model point.

[0110] Specifically, it is possible to perform eigenvalue decomposition on each covariance matrix obtained in step S541 to obtain several eigenvalues and the eigenvectors corresponding to the eigenvalues, sort the several eigenvalues corresponding to each defect point model point in a set order according to the eigenvalue magnitudes, to obtain the minimum eigenvalue of each defect point model point, and use the eigenvector corresponding to the minimum eigenvalue as the normal vector corresponding to the corresponding defect model point.

[0111] In this embodiment, a paint defect detection method of the present invention is provided. Several groups of calibrated automobile surface images are collected to obtain the internal parameter matrix of each acquisition device and the first conversion matrix between each acquisition device, and an initial virtual scene is created. According to the internal parameter matrix of each acquisition device and the first conversion matrix between each acquisition device, a virtual acquisition device and a car model are imported. By constructing a car model in the virtual scene, the efficiency of automobile paint defect detection is improved, and it is not limited to a certain car model, and the compatibility is improved. According to prior engineering data, a 3D surface point cloud of the calibrated car is obtained, and several 2D image points in any surface image and the corresponding 3D surface point cloud on the calibrated car are selected to construct feature point pairs, and the relative posture between the calibrated car and each acquisition device is obtained. According to the relative posture between the calibrated car and each acquisition device, the relative posture between each virtual acquisition device and the car model is corrected to obtain the final virtual scene, and several groups of current car 2D images are obtained. The coordinates of the defect points in each group of 2D images are obtained, and they are mapped to the car model to obtain the coordinates and normal of the defect points. The defect points in the 2D image are mapped to the 3D model, which can more accurately locate the defects and improve the accuracy of detection. The invention solves the problems of low efficiency, poor accuracy, insufficient compatibility and limited precision in the existing technology for detecting automobile paint defects.

[0112] On the other hand, the present invention further provides a computer storage medium storing executable program code; the executable program code is used to execute any of the above-mentioned paint defect detection methods.

[0113] On the other hand, the present invention also provides a terminal device, including a memory and a processor; the memory stores a program code that can be executed by the processor; the program code is used to execute any of the above-mentioned paint defect detection methods.

[0114] Exemplarily, the program code may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the program code in the terminal device.

[0115] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the terminal device may also include an input / output device, a network access device, a bus, etc.

[0116] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0117] The memory may be an internal storage unit of the terminal device, such as a hard disk or a memory. The memory may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device. Further, the memory may also include both an internal storage unit of the terminal device and an external storage device. The memory is used to store the program code and other programs and data required by the terminal device. The memory may also be used to temporarily store data that has been output or is to be output.

[0118] The above-mentioned computer storage medium and terminal device are created based on the above-mentioned paint defect detection method, and their technical functions and beneficial effects are no longer repeated here. The various technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the various technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A paint surface defect detection method, characterized in that: include: Collecting several groups of calibrated automobile surface images to obtain the internal parameter matrix of each acquisition device and the first conversion matrix between each acquisition device; Create an initial virtual scene, and import virtual acquisition devices and a car model according to the internal parameter matrix of each acquisition device and the first conversion matrix between each acquisition device; The 3D surface point cloud of the calibration car is obtained according to the prior engineering data, and several 2D image points in any surface image and the corresponding 3D surface point cloud on the calibration car are selected to construct feature point pairs to obtain the relative position and posture between the calibration car and each acquisition device; Correct the relative postures between each virtual acquisition device and the car model according to the relative postures between the calibrated car and each acquisition device to obtain the final virtual scene; Acquire several groups of current car 2D images, obtain the coordinates of the defect points in each group of 2D images, and map them to the car model to obtain the coordinates and normal of the defect points.

2. The method according to claim 1, characterized in that The step of obtaining the internal parameter matrix of each acquisition device and the first conversion matrix between each acquisition device includes: Each acquisition device is installed at a set position, and the calibration car is controlled to move in a set direction. Each acquisition device scans the surface of the calibration car at a preset sampling distance to obtain several groups of surface images of the calibration car; According to the surface images of all calibrated cars, several groups of relative postures between the calibrated cars and the acquisition devices are obtained, and each acquisition device is converted to the same coordinate system to obtain several relative acquisition devices; Performing image analysis on the surface images of each group of calibrated cars to obtain image feature points in each group of surface images; The image feature points in the coordinate systems of the relative acquisition devices are three-dimensionally reconstructed to obtain the first conversion matrix between the coordinate systems of the relative acquisition devices and the intrinsic parameter matrix of each relative acquisition device.

3. The method according to claim 1, characterized in that The steps of creating an initial virtual scene and importing virtual acquisition devices and a car model according to the internal reference matrix of each acquisition device and the first conversion matrix between the acquisition devices include: Import the acquisition equipment model of the same model as the actual scene and the design digital model of the calibrated car into the virtual scene to obtain the virtual acquisition equipment and car model; The virtual acquisition device parameters are configured according to the internal parameter matrix, and the relative positions and postures between the virtual acquisition devices in the virtual scene are obtained according to the first conversion matrix; According to the position of the calibrated car in the real scene, the corresponding car model is placed in the virtual scene.

4. The method according to claim 1, characterized in that The steps of obtaining the relative position and posture between the calibrated vehicle and each acquisition device include: The 3D surface point cloud of the calibrated car is obtained based on prior engineering data; Select several 2D image points and corresponding 3D surface point clouds in any surface image to construct several feature point pairs; The feature point pairs are reprojected to obtain a second transformation matrix between the acquisition device coordinate system corresponding to the surface image and the calibration vehicle coordinate system, and the relative posture between the calibration vehicle and each acquisition device is obtained according to the first transformation matrix between each acquisition device.

5. The method according to claim 4, characterized in that The steps of selecting a number of 2D image points and corresponding 3D surface point clouds in any surface image and constructing a number of feature point pairs include: Obtain the coordinates of 2D feature points in the surface image according to the graphic detection algorithm; Convert the 3D surface point cloud into a triangular mesh to obtain the characteristic hole area points; Fit a plane to the characteristic hole area points and extract the plane contour; The center of the circle is fitted to the plane contour to obtain 3D feature points, so as to construct a number of feature point pairs based on each 2D feature point and the corresponding 3D feature point.

6. The method according to claim 1, characterized in that The steps of obtaining the coordinates and normal direction of the defect point include: Build and train a neural network model that takes 2D images as input and defect point coordinates as output; Obtain several sets of current car 2D images, and input all 2D images into the neural network model to obtain the coordinates of all defect points in each 2D image; Project the coordinates of each defect point onto the car model to obtain the corresponding 3D coordinates, which are the defect model points; The coordinates of several point clouds closest to each defect model point are obtained, and the normal direction of each defect model point is calculated.

7. The method according to claim 6, characterized in that The steps of obtaining the normal direction of each defect model point may optionally include: Set the neighborhood size and obtain the coordinates of all points within the set neighborhood of each defect model point to construct a covariance matrix; Perform eigenvalue decomposition on the covariance matrix corresponding to each defect model point to obtain several eigenvalues ​​corresponding to each defect model point and eigenvectors corresponding to the eigenvalues; The minimum eigenvalue corresponding to each defect model point is obtained, and the eigenvector corresponding to the minimum eigenvalue is used as the normal direction of each defect model point.

8. The method according to claim 7, characterized in that The covariance matrix is: Among them, C is the covariance matrix, k is the set neighborhood size, N is the number of points within the k-neighborhood of each defect model point, p i is the coordinate of the i-th point, where 0 < i ≤ N, u is the average coordinate of all points within the k-neighborhood, and T is the matrix transpose symbol.

9. A computer storage medium, characterized in that An executable program code is stored; the executable program code is used to execute the paint surface defect detection method described in any one of claims 1-8.

10. A terminal device, characterized in that: It comprises a memory and a processor; the memory stores a program code executable by the processor; the program code is used to execute the paint defect detection method described in any one of claims 1-8.

Images