Plastic-coated template surface defect detection method and system based on three-dimensional modeling

Abstract

The invention relates to the technical field of material detection, in particular to a plastic-coated template surface defect detection method and system based on three-dimensional modeling, and the method comprises the following steps: scanning the surface of a plastic-coated template to be detected from different angles by adopting three-dimensional measurement equipment, and controlling the measurement equipment to emit structured light with specific frequency and pattern, an industrial camera is used for synchronously shooting reflected light images, and three-dimensional measurement data of the surface of the plastic-coated template are obtained; collecting a surface image of the plastic-coated template by using an industrial camera, and carrying out registration fusion on the surface image and the three-dimensional measurement data according to a corresponding position to obtain fusion data; importing the fused data into three-dimensional modeling software, and constructing a three-dimensional model of the surface of the plastic-coated template; and inputting the obtained three-dimensional model into a pre-trained plastic-coated template surface defect detection model based on deep learning to judge whether the surface of the plastic-coated template to be detected has defects or not. According to the method, the surface defects of the plastic-coated template are accurately detected through a method of combining three-dimensional modeling with deep learning, and the detection efficiency and the automation level are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of material detection, and particularly relates to a method and system for detecting surface defects of plastic-coated templates based on three-dimensional modeling. Background Art

[0002] As a commonly used material in industries such as construction and packaging, the surface quality of plastic-coated templates is related to product performance and service life. At present, the detection methods of plastic-coated templates mainly include manual visual inspection and traditional two-dimensional image detection. Manual visual inspection has low efficiency, strong subjectivity, and is prone to missed and misdetected inspections; two-dimensional image detection is greatly affected by factors such as lighting and angle, and it is difficult to detect defects with complex shapes and small sizes, unable to meet the high-precision and high-efficiency requirements of modern production, and there is an urgent need to develop new detection methods. Summary of the Invention

[0003] In view of the above-mentioned technical deficiencies, the present invention provides a method and system for detecting surface defects of plastic-coated templates based on three-dimensional modeling.

[0004] The present invention is realized through the following technical solutions:

[0005] A method for detecting surface defects of plastic-coated templates based on three-dimensional modeling is provided, and the method includes the following steps:

[0006] Step S10: A structured light three-dimensional measurement device is arranged around the plastic-coated template, and the surface of the plastic-coated template to be detected is scanned from different angles. The structured light three-dimensional measurement device includes a projector and an industrial camera. The projector projects structured light with a set frequency and pattern, and the industrial camera synchronously captures the reflected light image to obtain the three-dimensional measurement data of the surface of the plastic-coated template to be detected;

[0007] Step S20: After obtaining the three-dimensional measurement data of the surface of the plastic-coated template to be detected, the industrial camera in the structured light three-dimensional measurement device is used to capture the surface image of the plastic-coated template to be detected to record the color and texture features, and the three-dimensional measurement data is registered and fused according to the corresponding positions to obtain fusion data including geometric and appearance information;

[0008] Step S30: The fusion data is imported into three-dimensional modeling software to construct a three-dimensional model of the surface of the plastic-coated template to be detected;

[0009] Step S40: The obtained three-dimensional model is input into a pre-trained deep learning-based surface defect detection model of the plastic-coated template to determine whether there are defects on the surface of the plastic-coated template to be detected;

[0010] Among them, the structured light 3D measurement device used in step S10 utilizes the optical principle to obtain the 3D measurement data of the surface of the plastic-coated template. The projector projects stripes or Gray codes onto the surface of the plastic-coated template; the industrial camera takes reflective images from different angles and calculates the 3D coordinates of each point on the surface of the plastic-coated template based on the triangulation principle. The resolution of the industrial camera used is not less than 2k;

[0011] Among them, the pre-trained deep learning-based surface defect detection model for plastic-coated templates in step S40 adopts a multi-level convolutional network architecture, including an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer.

[0012] Preferably, the steps of obtaining the 3D measurement data of the surface of the plastic-coated template to be detected in step S10 include:

[0013] Device deployment and calibration: According to the size, shape, and detection accuracy requirements of the plastic-coated template to be measured, a certain number of groups of structured light 3D measurement devices are arranged in the detection area; after the deployment is completed, the projectors and industrial cameras with a resolution of 2k in each group of structured light 3D measurement devices are calibrated. Using standard calibration tools, such as a checkerboard calibration board, the internal parameters of the projectors and industrial cameras are obtained, including focal length and principal point coordinates, etc., as well as the relative position relationship between the projectors and industrial cameras, that is, the external parameters. These parameters are the basis for accurately calculating the 3D coordinates based on the triangulation principle later;

[0014] Structured light projection and image acquisition: After the device is turned on, control the projector to emit stripe structured light with a period of 10mm or Gray code structured light containing 10-bit information, and the industrial camera takes pictures at a speed of 5 frames per second to ensure that stable and clear reflected light images can be captured, providing accurate data for subsequent 3D coordinate calculation;

[0015] Feature point extraction and matching: Process the captured reflected light images, use image algorithms to extract the feature points in the reflected light images, such as the center of the stripes, the boundaries of the Gray codes, etc. Determine the corresponding positions of the same physical point in different images according to the images taken from all angles through the feature point matching algorithm. For example, use the SIFT algorithm or the ORB algorithm to achieve fast and accurate feature point matching and establish the connection between the two-dimensional image coordinates and the three-dimensional space coordinates;

[0016] 3D coordinate calculation and data acquisition: According to the triangulation principle and the corresponding positions of the same physical point in different images determined, calculate the 3D coordinates of each point on the surface of the plastic-coated template. The calculation formula is shown in Equation (1):

[0017]

[0018] Where X is the x-axis coordinate in the calculated three-dimensional coordinates, B is the baseline distance between the projector and the industrial camera, f is the focal length of the industrial camera, x and x' are the abscissas of the feature point in the image captured by the industrial camera and the abscissa in the projector image, and Z is the depth value of the feature point in the camera coordinate system, which is used to determine the position of the feature point in the direction of the camera optical axis. In the calculation, an initial value is first assumed, and then it is accurately solved through the multi-view constraint method. For example, when performing multi-angle scanning on the plastic-coated template, the association between the data obtained by cameras at different angles is used to determine the value of Z, calculate the accurate three-dimensional coordinates, and provide a reliable data basis for subsequent three-dimensional modeling and defect detection; the y-axis and z-axis coordinates in the three-dimensional coordinates are calculated by the same method to obtain the three-dimensional coordinates of each point on the surface of the plastic-coated template.

[0019] Preferably, the step of using the industrial camera to collect the surface image of the plastic-coated template, record the color and texture features, and register and fuse the three-dimensional measurement data according to the corresponding positions to obtain the fused data including geometric and appearance information in step S20 includes:

[0020] Matching algorithm selection and implementation: The iterative closest point algorithm is adopted. By continuously iterating, the optimal transformation matrix between the surface image features of the plastic-coated template and the three-dimensional measurement data is found, so that the coordinates in the three-dimensional measurement data are matched with the surface image features of the plastic-coated template in terms of spatial position;

[0021] Data fusion: After the spatial position is matched, the three-dimensional measurement data and the surface image features of the plastic-coated template are fused according to the corresponding positions to obtain the fused data including geometric and appearance information.

[0022] Preferably, the step of constructing the three-dimensional model of the surface of the plastic-coated template to be detected in step S30 includes:

[0023] Data import: Import the obtained fused data including geometric and appearance information into three-dimensional modeling software, such as Geomagic, 3ds Max, etc., select the coordinate system and unit that match the imported data, and each data in the fused data corresponds to a point in the coordinate system;

[0024] Noise removal: Set the standard deviation multiple, calculate the standard deviation of the distance between each point in the coordinate system and the neighboring points. When the distance between the point in the coordinate system and the neighboring points exceeds 2 times the standard deviation, it is determined as noise and removed;

[0025] Filtering process: Gaussian filtering is used to smooth the fused data. Select an appropriate Gaussian kernel size and standard deviation according to the data characteristics. The Gaussian kernel function performs weighted averaging on the coordinate values of each point in the coordinate system of the fused data according to the distance between the point in the coordinate system and the neighboring points. The closer the points are, the greater the weight. The high-frequency noise in the fused data is smoothed out to make the point cloud surface smoother and more continuous, while retaining the main geometric features;

[0026] Construct a 3D model of the surface of the plastic-coated template: Adopt the moving least squares surface construction method. By performing weighted least squares fitting on the neighborhood points of each point in the fused data containing geometric and appearance information after filtering, construct local surface patches, and then splice the constructed local surface patches into a complete surface to gradually construct a 3D model of the surface of the plastic-coated template;

[0027] Evaluate the 3D model of the surface of the plastic-coated template: Evaluate the 3D model by calculating the error between the constructed 3D model and the fused data containing geometric and appearance information after filtering. When the error is greater than 10%, readjust the weight function of the moving least squares surface construction method and then reconstruct the 3D model of the surface of the plastic-coated template until the error is less than or equal to 10%.

[0028] Preferably, the construction and training steps of the pre-trained deep learning-based surface defect detection model for plastic-coated templates in step S40 include:

[0029] Dataset construction: Obtain a large number of 3D model samples of the surfaces of plastic-coated templates with different types and degrees of defects, and divide them into a training set, a validation set, and a test set according to the ratio of 70%:20%:10%;

[0030] Construction of the surface defect detection model for the plastic-coated formwork: The entire surface defect detection model for the plastic-coated formwork includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. When the model is being trained, the input to the input layer is the training set. When the model is applied after training is completed, the input to the input layer is the three-dimensional model of the surface of the plastic-coated formwork to be detected. The number of convolutional layers is 5. The first convolutional layer includes 32 3×3 convolutional kernels for extracting low-level features of the three-dimensional model of the surface of the plastic-coated formwork. The second convolutional layer includes 64 5×5 convolutional kernels for extracting texture and pattern features of the three-dimensional model of the surface of the plastic-coated formwork. The third convolutional layer includes 128 7×7 convolutional kernels for extracting structural features of the three-dimensional model of the surface of the plastic-coated formwork, such as holes and large-area texture anomalies. The fourth convolutional layer includes 128 5×5 convolutional kernels for refining and supplementing the features extracted by the previous convolutional layer, enhancing the model's ability to identify different types of defects. The fifth convolutional layer includes 64 3×3 convolutional kernels for integrating and optimizing the features extracted by the previous convolutional layers, enabling the model to more accurately identify various defect features on the surface of the plastic-coated formwork. The pooling layer is an adaptive pooling layer that automatically adjusts the pooling area according to the size of the input feature map. The fully connected layer includes an attention mechanism module that calculates the weight of each feature, enabling the model to pay more attention to features related to the defects of the plastic-coated formwork and suppressing irrelevant information, thereby improving the accuracy of defect detection. Multiple fully connected layers are used to gradually integrate and classify the extracted features, achieving the conversion from low-level features to high-level defect category judgments. The output layer uses the Softmax function combined with regression to classify and judge different types of defects such as scratches, holes, and protrusions. At the same time, information such as the location and size of each defect is output through regression, realizing the comprehensive detection and positioning of the surface defects of the plastic-coated formwork.

[0031] Training and verification of the surface defect detection model for the plastic-coated formwork: After the model is constructed, set the model parameters. Use stochastic gradient descent and the Adam optimizer to adjust the model parameters. Use a combination of the cross-entropy loss function and the mean squared error loss function as the loss function for model training, which are used for error calculation in classification tasks and regression tasks respectively. After each round of training, use the divided validation set to verify the trained model, and update the parameters of the loss function according to the verification results to gradually reduce the loss of the model.

[0032] Evaluation of the surface defect detection model for the plastic-coated formwork: When the calculation result of the loss function cannot be reduced, the training and verification of the model are completed, and the divided test set is used for model testing.

[0033] Model optimization: Adjust the model parameters according to the model test results and retrain until the optimal combination of model parameters is obtained, and determine the corresponding version of the surface defect detection model for the plastic-coated formwork.

[0034] Preferably, the output results of the surface defect detection model for the plastic-coated formwork in step S40 include: whether there are defects on the surface of the plastic-coated formwork to be detected. When the model determines that there are defects on the surface of the plastic-coated formwork, it outputs the specific types and information of the defects. The types of defects include scratches, holes, protrusions, etc. The information of the defects includes the size and severity of the defects. For scratch defects, the model identifies them by analyzing the edge contours in the texture image and the curvature changes at the corresponding positions in the three-dimensional model, and gives data such as length and width. For hole and protrusion defects, the model judges according to geometric features such as depth and volume in the three-dimensional model. The hole defect will provide parameters such as hole area and depth, and the protrusion defect will provide parameters such as protrusion height and bottom area. These size information helps to evaluate the influence degree of the defects on the performance of the plastic-coated formwork and is also an important basis for judging the severity of the defects. At the same time, it gives the specific position information of the determined defect type on the surface of the plastic-coated formwork, and accurately gives the spatial position of the defect on the surface of the plastic-coated formwork based on the coordinate system of the three-dimensional model. For example, through the three-dimensional coordinates, it can be clear which specific area of the plastic-coated formwork the defect is located in, whether it is near the edge or the center position, etc.

[0035] In addition, to achieve the above object, the present invention also proposes a surface defect detection system for plastic-coated formwork based on three-dimensional modeling. The surface defect detection system for plastic-coated formwork based on three-dimensional modeling includes:

[0036] Plastic-coated formwork surface data acquisition module: A multi-group structured light three-dimensional measurement device is arranged around the plastic-coated formwork, and the surface of the plastic-coated formwork to be detected is scanned from different angles. The measurement device is controlled to emit structured light with a specific frequency and pattern, and a high-resolution industrial camera is used to synchronously capture the reflected light image to obtain the three-dimensional measurement data of the surface of the plastic-coated formwork to be detected;

[0037] Plastic-coated formwork surface data fusion module: A high-resolution industrial camera is used to collect the surface image of the plastic-coated formwork to be detected to record color and texture features, and is registered and fused with the three-dimensional measurement data according to the corresponding positions to obtain fusion data containing geometric and appearance information;

[0038] Plastic-coated formwork surface three-dimensional model construction module: The fusion data is imported into three-dimensional modeling software to construct a three-dimensional model of the surface of the plastic-coated formwork to be detected;

[0039] Plastic-coated formwork surface defect detection module: The obtained three-dimensional model is input into a pre-trained surface defect detection model for plastic-coated formwork based on deep learning to judge whether there are defects on the surface of the plastic-coated formwork to be detected;

[0040] The structured light 3D measurement device used in the surface data acquisition module of the plastic-coated template obtains 3D measurement data of the plastic-coated template surface using optical principles. The projector projects stripes or Gray codes onto the surface of the plastic-coated template; the industrial camera takes reflective images from different angles and calculates the 3D coordinates of each point on the surface of the plastic-coated template based on the triangulation principle. The resolution of the industrial camera used is not less than 2k;

[0041] The pre-trained deep learning-based plastic-coated template surface defect detection model in the plastic-coated template surface defect detection module adopts a multi-level convolutional network architecture, including an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer.

[0042] In addition, to achieve the above object, the present invention also proposes a plastic-coated template surface defect detection device based on 3D modeling. The device includes: a memory, a processor, and programs such as a plastic-coated template surface defect detection algorithm based on 3D modeling stored on the memory and executable on the processor. The programs such as the plastic-coated template surface defect detection algorithm based on 3D modeling are to implement the steps of a plastic-coated template surface defect detection method as described above.

[0043] In addition, to achieve the above object, the present invention also provides a computer program product. The computer program product includes programs such as a plastic-coated template surface defect detection algorithm based on 3D modeling. When the programs such as the plastic-coated template surface defect detection algorithm based on 3D modeling are executed by a processor, they implement a plastic-coated template surface defect detection method as described above.

[0044] The advantages and effects of the present invention are:

[0045] A plastic-coated template surface defect detection method and system proposed by the present invention uses a method combining 3D modeling and deep learning to accurately detect surface defects of plastic-coated templates. Through data fusion, the surface defect features of plastic-coated templates are registered and fused with 3D measurement data to obtain fusion data containing geometric and appearance information, improving the detection accuracy. The automated defect detection eliminates the need for manual participation, greatly saving time and labor costs, shortening the time for detecting surface defects of plastic-coated templates, and improving the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1It is a flowchart of a method for detecting surface defects of plastic - coated templates based on 3D modeling according to the present invention.

[0048] Figure 2 It is a schematic structural diagram of a system for detecting surface defects of plastic - coated templates based on 3D modeling according to the present invention.

[0049] Figure 3 It is a schematic block diagram of the structure of an electronic device for detecting surface defects of plastic - coated templates based on 3D modeling according to the present invention. Specific embodiments

[0050] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0051] The present invention provides a method for detecting surface defects of plastic - coated templates based on 3D modeling, as Figure 1 shown, including the following steps:

[0052] Step S10: Arrange a structured - light 3D measurement device around the plastic - coated template, scan the surface of the plastic - coated template to be detected from different angles. The structured - light 3D measurement device includes a projector and an industrial camera. The projector projects structured light with a set frequency and pattern, and the industrial camera synchronously captures the reflected - light image to obtain the 3D measurement data of the surface of the plastic - coated template to be detected.

[0053] Among them, the structured - light 3D measurement device used in step S10 obtains the 3D measurement data of the surface of the plastic - coated template using the optical principle. The projector projects stripes or Gray codes onto the surface of the plastic - coated template; the industrial camera takes reflected - light images from different angles and calculates the 3D coordinates of each point on the surface of the plastic - coated template based on the triangulation principle. The resolution of the industrial camera used is not less than 2k.

[0054] Specifically, the steps of obtaining the 3D measurement data of the surface of the plastic - coated template to be detected in step S10 include:

[0055] Device Deployment and Calibration: According to the size, shape, and detection accuracy requirements of the plastic-coated template to be measured, a certain number of structured light 3D measurement devices are arranged in the detection area. For example, for a rectangular plastic-coated template with dimensions of 2m × 1m, 4 sets of measurement devices can be arranged in a circular or square shape around it, with the device spacing maintained at about 1m to ensure full coverage scanning without detection blind spots. After deployment, calibrate the projectors and industrial cameras with 2k resolution in each set of structured light 3D measurement devices. Use standard calibration tools such as checkerboard calibration plates to obtain the internal parameters of the projectors and industrial cameras, including focal lengths and principal point coordinates, etc., as well as the relative position relationship between the projectors and industrial cameras, that is, the external parameters. These parameters are the basis for accurately calculating 3D coordinates based on the triangulation principle in the follow-up;

[0056] Structured Light Projection and Image Acquisition: After turning on the device, control the projector to emit stripe structured light with a period of 10mm or Gray code structured light containing 10-bit information, and the industrial camera takes pictures at a speed of 5 frames per second to ensure that stable and clear reflected light images can be captured, providing accurate data for subsequent 3D coordinate calculations;

[0057] Feature Point Extraction and Matching: Process the captured reflected light images, use image algorithms to extract feature points in the reflected light images, such as the centers of stripes, boundaries of Gray codes, etc. Determine the corresponding positions of the same physical point in different images according to the images taken from all shooting angles through feature point matching algorithms, such as using the SIFT algorithm or ORB algorithm, to achieve fast and accurate feature point matching and establish the connection between 2D image coordinates and 3D space coordinates;

[0058] 3D Coordinate Calculation and Data Acquisition: According to the triangulation principle and the determined corresponding positions of the same physical point in different images, calculate the 3D coordinates of each point on the surface of the plastic-coated template. The calculation formula is shown in Equation (1):

[0059]

[0060] Where X is the x-axis coordinate in the calculated 3D coordinates, B is the baseline distance between the projector and the industrial camera, f is the focal length of the industrial camera, x and x’ are the abscissas of the feature point in the image taken by the industrial camera and the abscissa in the projector image, Z is the depth value of the feature point in the camera coordinate system, used to determine the position of the feature point in the direction of the camera optical axis. Assume an initial value first in the calculation, and then accurately solve it through the multi-view constraint method. For example, when scanning the plastic-coated template from multiple angles, use the correlation between the data obtained by cameras at different angles to determine the Z value and calculate the accurate 3D coordinates, providing a reliable data basis for subsequent 3D modeling and defect detection; Calculate the y-axis and z-axis coordinates in the 3D coordinates in the same way to obtain the 3D coordinates of each point on the surface of the plastic-coated template.

[0061] Step S20: After obtaining the three-dimensional measurement data of the surface of the plastic-coated template to be detected, use the industrial camera in the structured light three-dimensional measurement device to capture the surface image of the plastic-coated template to be detected, record the color and texture features, and register and fuse them with the three-dimensional measurement data according to the corresponding positions to obtain the fused data containing geometric and appearance information.

[0062] Specifically, the steps of using the industrial camera in step S20 to collect the surface image of the plastic-coated template, record the color and texture features, and register and fuse them with the three-dimensional measurement data according to the corresponding positions to obtain the fused data containing geometric and appearance information include:

[0063] Matching algorithm selection and implementation: Adopt the iterative closest point algorithm, and continuously iterate to find the optimal transformation matrix between the surface image features of the plastic-coated template and the three-dimensional measurement data, so that the coordinates in the three-dimensional measurement data match the surface image features of the plastic-coated template in spatial position;

[0064] Data fusion: After the spatial position matching, fuse the three-dimensional measurement data and the surface image features of the plastic-coated template according to the corresponding positions to obtain the fused data containing geometric and appearance information.

[0065] Step S30: Import the fused data into three-dimensional modeling software to construct a three-dimensional model of the surface of the plastic-coated template to be detected.

[0066] Specifically, the steps of constructing a three-dimensional model of the surface of the plastic-coated template to be detected in step S30 include:

[0067] Data import: Import the obtained fused data containing geometric and appearance information into three-dimensional modeling software, such as Geomagic, 3ds Max, etc., select the coordinate system and unit that match the imported data, and each data in the fused data corresponds to a point in the coordinate system;

[0068] Noise removal: Set the standard deviation multiple, calculate the standard deviation of the distance between each point in the coordinate system and its neighborhood points. When the distance between a point in the coordinate system and its neighborhood points exceeds 2 times the standard deviation, it is determined as noise and removed;

[0069] Filtering process: Use Gaussian filtering to smooth the fused data, select appropriate Gaussian kernel size and standard deviation according to the data characteristics. The Gaussian kernel function performs weighted averaging on the coordinate values of each point in the coordinate system of the fused data according to the distance between the point in the coordinate system and its neighborhood points. The closer the points are, the greater the weight, smoothing out the high-frequency noise in the data to make the point cloud surface smoother and more continuous, while retaining the main geometric features;

[0070] Construct a 3D model of the surface of the plastic-coated formwork: Using the moving least squares surface construction method, perform weighted least squares fitting on the neighborhood points of each point in the fused data containing geometric and appearance information after filtering to construct local surface patches, and then splice the constructed local surface patches into a complete surface to gradually construct a 3D model of the surface of the plastic-coated formwork;

[0071] Evaluate the 3D model of the surface of the plastic-coated formwork: Evaluate the 3D model by calculating the error between the constructed 3D model and the fused data containing geometric and appearance information after filtering. When the error is greater than 10%, readjust the weight function of the moving least squares surface construction method and then reconstruct the 3D model of the surface of the plastic-coated formwork until the error is less than or equal to 10%.

[0072] Step S40: Input the obtained 3D model into a pre-trained deep learning-based plastic-coated formwork surface defect detection model to determine whether there are defects on the surface of the plastic-coated formwork to be detected.

[0073] Among them, the pre-trained deep learning-based plastic-coated formwork surface defect detection model in step S40 adopts a multi-level convolutional network architecture, including an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer.

[0074] Specifically, the construction and training steps of the pre-trained deep learning-based plastic-coated formwork surface defect detection model in step S40 include:

[0075] Dataset construction: Obtain a large number of 3D model samples of the surfaces of plastic-coated formworks with different types and degrees of defects, and divide them into a training set, a validation set, and a test set according to the ratio of 70%:20%:10%;

[0076] Construction of the Detection Model for Surface Defects of Plastic-Coated Templates: The entire detection model for surface defects of plastic-coated templates includes an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. When the model is being trained, the input to the input layer is the training set. When the model is applied after training is completed, the input to the input layer is the 3D model of the surface of the plastic-coated template to be detected. There are 5 convolutional layers. The first convolutional layer contains 32 3×3 convolutional kernels for extracting low-level features of the 3D model of the surface of the plastic-coated template. The second convolutional layer contains 64 5×5 convolutional kernels for extracting texture and pattern features of the 3D model of the surface of the plastic-coated template. The third convolutional layer contains 128 7×7 convolutional kernels for extracting structural features of the 3D model of the surface of the plastic-coated template, such as holes and large-area texture anomalies. The fourth convolutional layer contains 128 5×5 convolutional kernels for refining and supplementing the features extracted by the previous convolutional layer, enhancing the model's ability to identify different types of defects. The fifth convolutional layer contains 64 3×3 convolutional kernels for integrating and optimizing the features extracted by the previous convolutional layers, enabling the model to more accurately identify various defect features on the surface of the plastic-coated template. The pooling layer is an adaptive pooling layer that automatically adjusts the pooling area according to the size of the input feature map. The fully connected layer includes an attention mechanism module that calculates the weight of each feature, enabling the model to pay more attention to features related to plastic-coated template defects and suppressing irrelevant information, thereby improving the accuracy of defect detection. Multiple fully connected layers are used to gradually integrate and classify the extracted features, realizing the conversion from low-level features to high-level defect category judgments. The output layer uses the Softmax function combined with regression to classify and judge different types of defects such as scratches, holes, and protrusions. At the same time, information such as the position and size of each defect is output through regression, achieving comprehensive detection and positioning of surface defects of plastic-coated templates.

[0077] Training and Validation of the Detection Model for Surface Defects of Plastic-Coated Templates: After the model is constructed, set the model parameters. Use stochastic gradient descent and the Adam optimizer to adjust the model parameters. Use a combination of the cross-entropy loss function and the mean squared error loss function as the loss function for model training, which are used for error calculation in classification tasks and regression tasks respectively. After each round of training, use the divided validation set to verify the trained model. Update the parameters of the loss function according to the verification results to gradually reduce the loss of the model.

[0078] Evaluation of the Detection Model for Surface Defects of Plastic-Coated Templates: When the calculation result of the loss function cannot be reduced, the training and validation of the model are completed, and the divided test set is used for model testing.

[0079] Model Optimization: Adjust the model parameters according to the model test results and retrain until the optimal combination of model parameters is obtained, and determine the corresponding version of the detection model for surface defects of plastic-coated templates.

[0080] Specifically, the output results of the surface defect detection model for the plastic-coated formwork in step S40 include: whether there are defects on the surface of the plastic-coated formwork to be detected. When the model determines that there are defects on the surface of the plastic-coated formwork, it outputs the specific types and information of the defects. The types of defects include scratches, holes, protrusions, etc. The information of the defects includes the size and severity of the defects. For scratch defects, the model identifies them by analyzing the edge contours in the texture image and the curvature changes at the corresponding positions in the 3D model, and gives data such as length and width. For hole and protrusion defects, the model judges according to geometric features such as depth and volume in the 3D model. The hole defect will provide parameters such as hole area and depth, and the protrusion defect will provide parameters such as protrusion height and bottom area. These size information helps to evaluate the impact degree of the defects on the performance of the plastic-coated formwork and is also an important basis for judging the severity of the defects. For example, for scratch defects, those with shorter length, narrower width and located in non-critical parts of the formwork may be judged as minor defects; while scratches with longer length, larger width or located in the key stress area of the formwork may be judged as severe defects. At the same time, it gives the specific position information of the judged defect type on the surface of the plastic-coated formwork, and accurately gives the spatial position of the defect on the surface of the plastic-coated formwork based on the coordinate system of the 3D model. For example, through the 3D coordinates, it can be clear which specific area of the plastic-coated formwork the defect is located in, close to the edge or the center position, etc.

[0081] In addition, the present invention also proposes a surface defect detection system for plastic-coated formwork based on 3D modeling. Please refer to Figure 2 The surface defect detection system for plastic-coated formwork based on 3D modeling includes:

[0082] Plastic-coated formwork surface data acquisition module: A multi-group structured light 3D measurement device is arranged around the plastic-coated formwork, and the surface of the plastic-coated formwork to be detected is scanned from different angles. The measurement device is controlled to emit structured light with a specific frequency and pattern, and a high-resolution industrial camera is used to synchronously capture the reflected light image to obtain the 3D measurement data of the surface of the plastic-coated formwork to be detected.

[0083] Plastic-coated formwork surface data fusion module: Use a high-resolution industrial camera to collect the surface image of the plastic-coated formwork to be detected to record color and texture features, and register and fuse them with the 3D measurement data according to the corresponding positions to obtain fusion data containing geometric and appearance information.

[0084] Plastic-coated formwork surface 3D model construction module: Import the fusion data into 3D modeling software to construct the 3D model of the surface of the plastic-coated formwork to be detected.

[0085] Plastic-coated formwork surface defect detection module: Input the obtained 3D model into a pre-trained deep learning-based surface defect detection model for the plastic-coated formwork to judge whether there are defects on the surface of the plastic-coated formwork to be detected.

[0086] The structured light 3D measurement device used in the surface data acquisition module of the plastic-coated template obtains the 3D measurement data of the plastic-coated template surface by using optical principles. The projector projects stripes or Gray codes onto the plastic-coated template surface; the industrial camera takes reflective images from different angles and calculates the 3D coordinates of each point on the plastic-coated template surface based on the triangulation principle. The resolution of the industrial camera used is not less than 2k.

[0087] The pre-trained deep learning-based plastic-coated template surface defect detection model in the plastic-coated template surface defect detection module adopts a multi-level convolutional network architecture, including an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer.

[0088] A plastic-coated template surface defect detection system based on 3D modeling provided by the present application adopts a plastic-coated template surface defect detection method in the above embodiment, which can solve the technical problems of low efficiency and low accuracy of the traditional plastic-coated template surface defect detection method. Compared with the prior art, the beneficial effects of the plastic-coated template surface defect detection system based on 3D modeling provided by the present application are the same as those of the plastic-coated template surface defect detection method based on 3D modeling provided by the above embodiment, and other technical features in the plastic-coated template surface defect detection system based on 3D modeling are the same as those disclosed in the above embodiment method, which will not be elaborated here.

[0089] The present application provides a plastic-coated template surface defect detection device based on 3D modeling. The plastic-coated template surface defect detection device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute a plastic-coated template surface defect detection method in the first embodiment above.

[0090] Refer to the following Figure 3 , which shows a schematic structural diagram of a plastic-coated template surface defect detection device suitable for implementing the embodiments of the present application. A plastic-coated template surface defect detection device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3The shown surface defect detection device for plastic - coated templates based on 3D modeling is merely an example and should not impose any limitations on the functions and application scope of the embodiments of this application.

[0091] Figure 3 The shown surface defect detection device for plastic - coated templates based on 3D modeling may include a processing system 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read - only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage system 1003 into the random - access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the surface defect detection device for plastic - coated templates based on 3D modeling are also stored. The processing system 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input system 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output system 1008 including, for example, a liquid - crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage system 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication system 1009. The communication system 1009 can allow the surface defect detection device for plastic - coated templates based on 3D modeling to communicate with other devices wirelessly or wire - line to exchange data. Although the figure shows a surface defect detection device for plastic - coated templates based on 3D modeling with various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems can be alternatively implemented or had.

[0092] Particularly, according to the disclosed embodiments of this application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the disclosed embodiments of this application include a computer program product, which includes a computer program carried on a computer - readable medium, and the computer program contains program codes for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication system, or installed from the storage system 1003, or installed from the ROM 1002. When the computer program is executed by the processing system 1001, the above - defined functions in the method of the disclosed embodiments of this application are executed.

[0093] A surface defect detection device for plastic-coated templates based on 3D modeling provided by the present application, which adopts a surface defect detection method for plastic-coated templates based on 3D modeling in the above-mentioned embodiment, can solve the technical problems of low efficiency and low accuracy in the traditional surface defect detection method for plastic-coated templates. Compared with the prior art, the beneficial effects of the surface defect detection device for plastic-coated templates based on 3D modeling provided by the present application are the same as those of the surface defect detection method for plastic-coated templates based on 3D modeling provided by the above-mentioned embodiment, and other technical features in the surface defect detection device for plastic-coated templates based on 3D modeling are the same as those disclosed in the method of the previous embodiment, which will not be elaborated here.

[0094] Each part disclosed in the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0095] The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it realizes the steps of a surface defect detection method for plastic-coated templates based on 3D modeling as described above.

[0096] The computer program product provided by the present application can solve the technical problems of low efficiency and low accuracy in the traditional surface defect detection method for plastic-coated templates. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the surface defect detection method for plastic-coated templates based on 3D modeling provided by the above-mentioned embodiment, which will not be elaborated here.

[0097] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims

1. A method for detecting surface defects of a plastic-coated template based on three-dimensional modeling, characterized in that: The method comprises the following steps: Step S10: using a structured light three-dimensional measurement device to surround the plastic-coated template and scan the surface of the plastic-coated template to be inspected from different angles. The structured light three-dimensional measurement device includes a projector and an industrial camera. The projector projects structured light of a set frequency and pattern, and the industrial camera synchronously captures a reflected light image to obtain three-dimensional measurement data of the surface of the plastic-coated template to be inspected. Step S20: After obtaining the three-dimensional measurement data of the surface of the overmolded template to be inspected, the industrial camera in the structured light three-dimensional measurement device is used to shoot the surface image of the overmolded template to be inspected to record the color and texture features, and the image is fused with the three-dimensional measurement data according to the corresponding position to obtain fused data containing geometric and appearance information; Step S30: importing the fused data into a three-dimensional modeling software to construct a three-dimensional model of the surface of the plastic-coated template to be inspected; Step S40: inputting the obtained three-dimensional model into a pre-trained surface defect detection model of the overmolding template based on deep learning to determine whether there are defects on the surface of the overmolding template to be detected; The structured light 3D measurement device used in step S10 uses optical principles to obtain 3D measurement data of the surface of the plastic-coated template, and the projector projects stripes or Gray codes onto the surface of the plastic-coated template; the industrial camera takes reflective images from different angles, and calculates the 3D coordinates of each point on the surface of the plastic-coated template based on the triangulation principle, and the resolution of the industrial camera used is not less than 2k; The pre-trained deep learning-based plastic coating template surface defect detection model in step S40 adopts a multi-level convolutional network architecture, including an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer.

2. A method for detecting surface defects of a plastic-coated template based on three-dimensional modeling according to claim 1, characterized in that: The step of obtaining three-dimensional measurement data of the surface of the plastic-coated template to be detected in step S10 includes: Equipment deployment and calibration: According to the size, shape and detection accuracy requirements of the plastic-covered template to be tested, a certain number of structured light 3D measurement equipment are arranged in the detection area; after the deployment is completed, the projector and industrial camera in each group of structured light 3D measurement equipment are calibrated to obtain the internal parameters of the projector and industrial camera, including the focal length and principal point coordinates, as well as the relative position relationship between the projector and the industrial camera; Structured light projection and image acquisition: After turning on the device, the projector is controlled to emit stripe structured light with a period of 10mm or Gray code structured light containing 10 bits of information, and the industrial camera shoots at a speed of 5 frames per second; Feature point extraction and matching: Process the captured reflected light image, use the image algorithm to extract the feature points in the reflected light image, and determine the corresponding position of the same physical point in different images through the feature point matching algorithm based on the images of all shooting angles; Three-dimensional coordinate calculation and data acquisition: According to the triangulation principle and the corresponding positions of the same physical point in different images, the three-dimensional coordinates of each point on the surface of the plastic-coated template are calculated. The calculation formula is shown in formula (1): Where X is the x-axis coordinate in the calculated three-dimensional coordinates, B is the baseline distance between the projector and the industrial camera, f is the focal length of the industrial camera, x and x' are the horizontal coordinates of the feature point in the image taken by the industrial camera and the horizontal coordinates in the projector image, and Z is the depth value of the feature point in the camera coordinate system, which is used to determine the position of the feature point in the direction of the camera optical axis; the y-axis and z-axis coordinates in the three-dimensional coordinates are calculated by the same method to obtain the three-dimensional coordinates of each point on the surface of the plastic-coated template.

3. The method for detecting surface defects of a plastic-coated template based on three-dimensional modeling according to claim 1, characterized in that: In step S20, the industrial camera is used to collect the surface image of the plastic-covered template to record the color and texture features, and the image is fused with the three-dimensional measurement data according to the corresponding position to obtain the fused data containing the geometric and appearance information, which includes: Matching algorithm selection and implementation: Adopt the iterative closest point algorithm to find the optimal transformation matrix between the surface image features of the plastic template and the three-dimensional measurement data through continuous iteration, so that the coordinates in the three-dimensional measurement data match the surface image features of the plastic template in spatial position; Data fusion: After spatial position matching, the three-dimensional measurement data is fused with the surface image features of the plastic-covered template according to the corresponding positions to obtain fused data containing geometric and appearance information.

4. The method for detecting surface defects of a plastic-coated template based on three-dimensional modeling according to claim 1, characterized in that: The step of constructing a three-dimensional model of the surface of the plastic-coated template to be detected in step S30 includes: Data import: Import the obtained fused data containing geometry and appearance information into the 3D modeling software, select the coordinate system and unit that match the imported data, and each data in the fused data corresponds to a point in the coordinate system; Noise removal: Set the standard deviation multiple and calculate the standard deviation of the distance between each point in the coordinate system and the domain point. When the distance between a point in the coordinate system and the neighboring point exceeds 2 times the standard deviation, it is determined to be a noise point and removed. Filtering: Gaussian filtering is used to smooth the fused data. The appropriate Gaussian kernel size and standard deviation are selected according to the data characteristics. The Gaussian kernel function performs weighted average of the coordinate values ​​of each point in the fused data in the coordinate system according to the distance between the point in the coordinate system and the neighboring points. The closer the distance, the greater the weight of the point. Constructing a three-dimensional model of the plastic-covered template surface: Using the moving least squares surface construction method, a weighted least squares fitting is performed on the neighborhood points of each point in the filtered fusion data containing geometric and appearance information to construct a local surface patch, and then the constructed local surface patches are spliced ​​into a complete surface, gradually constructing a three-dimensional model of the plastic-covered template surface; Evaluation of the three-dimensional model of the plastic-coated template surface: The three-dimensional model is evaluated by calculating the error between the constructed three-dimensional model and the fused data containing geometric and appearance information after filtering. When the error is greater than 10%, the weight function of the moving least squares surface construction method is readjusted and the three-dimensional model of the plastic-coated template surface is reconstructed until the error is less than or equal to 10%.

5. The method for detecting surface defects of a plastic-coated template based on three-dimensional modeling according to claim 1, characterized in that: The steps of constructing and training the pre-trained deep learning-based plastic coating template surface defect detection model in step S40 include: Dataset construction: Obtain 3D model samples of the surface of plastic-coated templates with different types and degrees of defects, and divide them into training set, validation set and test set in a ratio of 70%:20%:10%; Construction of the surface defect detection model for the overmolded template: The entire surface defect detection model for the overmolded template includes an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer; when the model is trained, the input of the input layer is the training set, and when the model is applied after training, the input of the input layer is the three-dimensional model of the surface of the overmolded template to be detected; the number of convolution layers is 5, the first convolution layer includes 32 3×3 convolution kernels for extracting low-level features of the three-dimensional model of the overmolded template surface, the second convolution layer includes 64 5×5 convolution kernels for extracting texture and pattern features of the three-dimensional model of the overmolded template surface, and the third convolution layer includes 128 7×7 convolution kernels for extracting The structural features of the three-dimensional model of the plastic template surface are obtained. The fourth convolution layer includes 128 5×5 convolution kernels for refining and supplementing the features extracted by the previous convolution layer. The fifth convolution layer includes 64 3×3 convolution kernels for integrating and optimizing the features extracted by the previous convolution layer. The pooling layer is an adaptive pooling layer, which automatically adjusts the pooling area according to the size of the input feature map. The fully connected layer includes an attention mechanism module, which calculates the weight of each feature to make the model focus on the features related to the defects of the plastic template. The output layer uses the Softmax function combined with regression to classify and judge different types of defects, and outputs the position and size of each defect by regression. Training and verification of the plastic-coated template surface defect detection model: After the model is built, the model parameters are set, and the model parameters are adjusted using stochastic gradient descent and Adam optimizer. The cross entropy loss function and the mean square error loss function are combined as the loss function for model training, which are used for error calculation of classification tasks and regression tasks respectively. After each round of training, the divided verification set is used to verify the trained model, and the parameters of the loss function are updated according to the verification results; Evaluation of the model for surface defect detection of plastic-coated templates: When the calculation result of the loss function cannot be reduced, complete the training and verification of the model, and use the divided test set to test the model; Model optimization: Adjust model parameters and retrain according to model test results until the optimal model parameter combination is obtained, and determine the corresponding model version for surface defect detection of overmolded templates.

6. The method for detecting surface defects of a plastic-coated template based on three-dimensional modeling according to claim 1, characterized in that: The output result of the overmolding template surface defect detection model in step S40 includes: whether there are defects on the surface of the overmolding template to be detected; when the model determines that there are defects on the surface of the overmolding template, the specific type and information of the defects are output, the types of defects include scratches, holes and protrusions, and the information of the defects includes the size of the defects and the severity of the defects; at the same time, the specific position information of the determined defect type on the surface of the overmolding template is given, and the spatial position of the defect on the surface of the overmolding template is accurately given based on the coordinate system of the three-dimensional model.

7. A surface defect detection system for plastic-coated templates based on three-dimensional modeling, characterized in that: The three-dimensional modeling-based surface defect detection system for plastic coating templates comprises: Plastic template surface data acquisition module: Use structured light three-dimensional measurement equipment to surround the plastic template, scan the surface of the plastic template to be inspected from different angles, control the measurement equipment to emit structured light of specific frequency and pattern, use industrial camera to synchronously shoot reflected light image, and obtain three-dimensional measurement data of the surface of the plastic template to be inspected; Plastic overlay template surface data fusion module: Use industrial cameras to collect images of the plastic overlay template surface to be inspected, record color and texture features, and perform registration and fusion with the 3D measurement data according to corresponding positions to obtain fused data containing geometric and appearance information; Module for constructing a three-dimensional model of the surface of the plastic-coated template: importing the fused data into the three-dimensional modeling software to construct a three-dimensional model of the surface of the plastic-coated template to be inspected; Plastic overlay template surface defect detection module: input the obtained three-dimensional model into a pre-trained plastic overlay template surface defect detection model based on deep learning to determine whether there are defects on the surface of the plastic overlay template to be detected; The structured light 3D measurement device used in the plastic template surface data acquisition module uses optical principles to obtain 3D measurement data of the plastic template surface, and the projector projects stripes or Gray codes onto the plastic template surface; the industrial camera takes reflective images from different angles and calculates the 3D coordinates of each point on the plastic template surface based on the triangulation principle, and the resolution of the industrial camera used is not less than 2k; The pre-trained deep learning-based plastic coating template surface defect detection model in the plastic coating template surface defect detection module adopts a multi-level convolutional network architecture, including an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer.

8. A surface defect detection device for plastic-coated templates based on three-dimensional modeling, characterized in that: The surface defect detection device for plastic coating template based on three-dimensional modeling includes: A memory, a processor, and a plastic template surface defect detection program based on three-dimensional modeling stored in the memory and executable on the processor, wherein the plastic template surface defect detection program based on three-dimensional modeling, when executed by the processor, implements a plastic template surface defect detection method based on three-dimensional modeling as described in any one of claims 1 to 6.

9. A computer program product, characterized in that The computer program product includes a three-dimensional modeling-based overmolding template surface defect detection program, and when the three-dimensional modeling-based overmolding template surface defect detection program is executed by a processor, it implements a three-dimensional modeling-based overmolding template surface defect detection method as described in any one of claims 1 to 6.

Images