Precise plastic mold detection method, device and equipment and storage medium

Abstract

The invention relates to the technical field of mold detection, and discloses a precision plastic mold detection method, device and equipment and a storage medium, the method comprises the following steps: carrying out multispectral imaging and three-dimensional laser scanning on the surface of a precision plastic mold to obtain multispectral image data and three-dimensional point cloud data; performing multi-scale adaptive threshold segmentation to obtain a first three-dimensional model; performing tensor voting surface reconstruction and texture recovery to obtain a second three-dimensional model; performing global optimization to obtain a target three-dimensional model; extracting multi-view features and carrying out defect classification and positioning to obtain defect category and distribution information; according to the defect category and distribution information and the geometrical characteristics of the precise plastic mold, an optimal scanning detection path is calculated by using a sequence convex optimization algorithm guided by ray tracing, and a self-adaptive mold detection strategy is generated, so that the coverage rate and the precision of detection are improved, the detection time and the resource consumption are balanced, and the detection efficiency is improved. And efficient, comprehensive and accurate mold quality control is realized.

Description

Technical Field

[0001] The present application relates to the field of mold detection technology, and in particular to a precision plastic mold detection method, device, equipment and storage medium. Background Art

[0002] Traditional precision plastic mold inspection methods mainly rely on manual visual inspection and simple machine vision systems, which often have problems such as strong subjectivity, low efficiency and limited accuracy.

[0003] In recent years, the development of multispectral imaging technology and 3D laser scanning technology has provided new possibilities for mold inspection. However, how to effectively integrate these two technologies and accurately identify and locate tiny defects from massive multimodal data remains a huge challenge. In addition, the complex geometry and material properties of the mold surface also bring additional difficulties to detection, especially when dealing with high reflectivity or low contrast surfaces. Another key issue is how to improve detection efficiency while ensuring detection accuracy. The traditional point-by-point scanning method is time-consuming and difficult to meet the high efficiency requirements of modern production lines. How to design an intelligent and adaptive detection strategy that can quickly formulate the optimal scanning path based on the geometric characteristics of the mold and the distribution of potential defects is an urgent problem to be solved. Summary of the invention

[0004] The present application provides a precision plastic mold inspection method, device, equipment and storage medium, thereby improving the inspection coverage and accuracy, balancing the inspection time and resource consumption, and achieving efficient, comprehensive and accurate mold quality control.

[0005] The first aspect of the present application provides a precision plastic mold detection method, the precision plastic mold detection method comprising: Perform multispectral imaging and 3D laser scanning on the surface of precision plastic molds to obtain multispectral image data and 3D point cloud data; Performing multi-scale adaptive threshold segmentation on the multispectral image data to obtain a target segmentation result, and mapping the target segmentation result to the three-dimensional point cloud data to obtain a first three-dimensional model; Performing tensor voting surface reconstruction based on the first three-dimensional model and restoring texture of potential defect areas using a multispectral image restoration network to obtain a second three-dimensional model; Performing global optimization on the second three-dimensional model through a non-local refinement network to obtain a target three-dimensional model; Extracting multi-view features based on the target three-dimensional model, and classifying and locating defects through a multi-scale convolutional neural network and an attention mechanism to obtain defect category and distribution information; According to the defect category and distribution information and the geometric features of the precision plastic mold, a ray tracing-guided sequential convex optimization algorithm is used to calculate the optimal scanning detection path and generate an adaptive mold detection strategy.

[0006] The second aspect of the present application provides a precision plastic mold detection device, the precision plastic mold detection device comprising: Scanning module, used to perform multispectral imaging and three-dimensional laser scanning on the surface of precision plastic molds to obtain multispectral image data and three-dimensional point cloud data; A segmentation module, used for performing multi-scale adaptive threshold segmentation on the multispectral image data to obtain a target segmentation result, and mapping the target segmentation result to the three-dimensional point cloud data to obtain a first three-dimensional model; A reconstruction module, used for performing tensor voting surface reconstruction based on the first three-dimensional model and restoring the texture of the potential defect area using a multispectral image restoration network to obtain a second three-dimensional model; An optimization module, used for globally optimizing the second three-dimensional model through a non-local refinement network to obtain a target three-dimensional model; A classification module is used to extract multi-view features based on the target three-dimensional model, and classify and locate defects through a multi-scale convolutional neural network and an attention mechanism to obtain defect category and distribution information; A generation module is used to calculate the optimal scanning detection path based on the defect category and distribution information and the geometric characteristics of the precision plastic mold using a ray tracing-guided sequential convex optimization algorithm to generate an adaptive mold detection strategy.

[0007] The third aspect of the present application provides an electronic device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the electronic device executes the above-mentioned precision plastic mold detection method.

[0008] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned precision plastic mold detection method.

[0009] Compared with the prior art, the present application has the following beneficial effects: By combining multispectral imaging and three-dimensional laser scanning technology, the present invention can simultaneously obtain the spectral information and geometric information of the mold surface, greatly improving the comprehensiveness and accuracy of defect detection. The multi-scale adaptive threshold segmentation algorithm can effectively handle the illumination changes and material property differences in different regions, and improve the robustness of the segmentation results. The tensor voting surface reconstruction technology combined with the multispectral image restoration network can not only accurately reconstruct the mold surface geometry, but also restore the texture details of the potential defect area. The model is globally optimized by the non-local refinement network, which effectively improves the accuracy and consistency of the three-dimensional model. Based on the target three-dimensional model, multi-view features are extracted, and combined with multi-scale convolutional neural networks and attention mechanisms, the accuracy of defect classification and positioning is significantly improved. The ray tracing-guided sequential convex optimization algorithm can automatically generate the optimal scanning path according to the defect distribution and mold geometric features, greatly improving the detection efficiency. By comprehensively considering the coverage, detection accuracy and scanning time, the generated adaptive mold detection strategy can maximize efficiency while ensuring the detection quality. The detection process of the present invention is automatically performed from data collection to defect identification to path planning, which greatly reduces human intervention and realizes efficient, comprehensive and accurate mold quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] The structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportion or adjustment of size, without affecting the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical contents disclosed by the present invention.

[0012] Figure 1 It is a flow chart of a precision plastic mold detection method provided by an embodiment of the present invention; Figure 2 It is a schematic block diagram of the structure of a precision plastic mold detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0015] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0016] It should be further understood that the term "and / or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. Figure 1 , an embodiment of the precision plastic mold detection method in the embodiment of the present application includes: Step 100, performing multispectral imaging and three-dimensional laser scanning on the surface of the precision plastic mold to obtain multispectral image data and three-dimensional point cloud data; It is understandable that the execution subject of the present application may be a precision plastic mold detection device, or a terminal or a server, which is not specifically limited here. The present application embodiment is described by taking a server as the execution subject as an example.

[0017] Specifically, a multispectral camera is used to image the surface of a precision plastic mold, and image data including visible light, near infrared and ultraviolet spectrum bands are collected to obtain an original multispectral image set. Dark current correction and flat field correction are performed on the original multispectral image set to eliminate the noise generated inside the camera and the influence caused by uneven illumination, and a corrected multispectral image set is obtained. The spectral response curve is calibrated based on the corrected multispectral image set, and the corresponding relationship between the pixel grayscale value and the actual reflectivity is established through precise calculation to obtain a reflectivity correction model. According to the reflectivity correction model, the reflectivity of the corrected multispectral image set is converted to generate multispectral image data with higher accuracy. At the same time, a structured light three-dimensional laser scanner is used to scan the surface of the precision plastic mold to obtain detailed morphological information of the surface and obtain original point cloud data. Noise filtering and outlier deletion are performed on the original point cloud data, and a statistical outlier filtering algorithm is used to remove abnormal points to ensure the accuracy and integrity of the data, and generate filtered point cloud data. Based on the filtered point cloud data, the normal vector is estimated. Using the covariance analysis method, the spatial geometric characteristics of the point cloud data are analyzed by calculating the normal vector of each point to generate accurate three-dimensional point cloud data.

[0018] Step 200, performing multi-scale adaptive threshold segmentation on the multispectral image data to obtain a target segmentation result, and mapping the target segmentation result to three-dimensional point cloud data to obtain a first three-dimensional model; Specifically, the Gaussian-Laplacian pyramid algorithm is used to perform multi-scale pyramid decomposition on the multispectral image data. By constructing image pyramids of 5 different scales, a multi-scale image set is generated to capture the feature information of the image at different scales. Based on the multi-scale image set, the statistical features of each local area are calculated. The sliding window method is used to calculate the mean and variance of the neighborhood of each pixel point to generate a feature map reflecting the local statistical features. An adaptive threshold function is constructed based on the feature map. In order to map the local statistical features to the threshold space, the sigmoid function is used to generate a smooth threshold mapping function. Based on the threshold mapping function, the multi-scale image set is binarized at the pixel level to generate a multi-scale binary image set. A cross-scale consistency check is performed based on the multi-scale binary image set to ensure the coordination between the segmentation results of each scale. The segmentation results at different scales are fused by using the majority voting strategy to obtain a preliminary segmentation result. In order to improve the accuracy of segmentation and remove noise, the initial segmentation result is processed by opening and closing operations. This step optimizes the segmentation result by eliminating isolated noise points and smoothing the boundaries to obtain the target segmentation result. In order to map the segmentation results to the 3D point cloud data, a KD tree index structure is constructed based on the 3D point cloud data, which can accelerate spatial search. The nearest neighbor search algorithm is used to establish the correspondence between 2D pixels and 3D points to generate a mapping matrix. According to the generated mapping matrix, the target segmentation results are projected from 2D to 3D point cloud data, and each 3D point is assigned a corresponding label attribute to construct the first 3D model.

[0019] Step 300: Perform tensor voting surface reconstruction based on the first three-dimensional model, and use a multispectral image restoration network to restore the texture of the potential defect area to obtain a second three-dimensional model; It should be noted that the local covariance matrix is ​​calculated for each point in the first three-dimensional model. By using the K nearest neighbor algorithm, the corresponding neighborhood points are selected from the surrounding neighborhood of each point, and the covariance features of the point cloud are calculated based on the geometric distribution information of the neighborhood points. Based on the covariance features of the point cloud, a tensor field is constructed. In this process, the main direction information of each point is extracted to obtain the initial tensor field. In order to enhance the information accuracy of the tensor field, tensor voting is performed on the initial tensor field. The specific implementation of tensor voting depends on the use of spherical kernel functions. The spherical kernel function calculates the contribution of each point to its neighborhood points, and forms a cumulative tensor field by accumulating these voting results. Based on the cumulative tensor field, feature extraction is performed, including the calculation of the saliency and normal vector of each point. Using the surface reconstruction features, Poisson surface reconstruction is performed to generate a preliminary reconstructed mesh model. The reconstructed mesh model is topologically optimized, and the irregular structure in the mesh is eliminated during the optimization process to generate an optimized mesh model. The input of the multispectral image restoration network is constructed based on the optimized mesh model. The UV unwrapping technique is used to project the 3D model onto a 2D plane to generate a UV map. The UV map can flatten the texture information of the 3D geometric structure to facilitate subsequent texture restoration operations. The UV map is input into the multispectral image restoration network. The network uses a generative adversarial network architecture to generate high-quality texture details through an adversarial learning mechanism, while combining a multi-scale receptive field fusion strategy to ensure that texture information at different scales can be fully restored and potential defective areas on the 3D model can be repaired. Through the operation of the multispectral image restoration network, fine texture details are restored, and combined with the optimized grid structure, a second 3D model is generated.

[0020] Step 400: globally optimize the second three-dimensional model through a non-local refinement network to obtain a target three-dimensional model; Specifically, the second three-dimensional model is mesh downsampled, and the high-resolution three-dimensional model is gradually simplified into a multi-level low-resolution model, a pyramid-like multi-resolution representation is constructed, and a multi-resolution mesh sequence is generated. The multi-resolution mesh sequence is input into the graph convolutional network, and the topological structure of the mesh is encoded using the Laplacian matrix to generate a graph structure representation. The Laplacian matrix can capture the connection relationship between the vertices and their neighbors in the mesh, ensuring that the geometric structure information is accurately represented in the network. The non-local attention mechanism is analyzed for the graph structure representation. The self-attention algorithm is used to calculate the global correlation matrix. The matrix captures the long-distance dependency between different regions in the model to obtain an attention weight map. By analyzing the mutual correlation between different regions, the attention mechanism effectively improves the global perception ability of the model and obtains a more accurate feature expression. Based on the attention weight map, feature aggregation is performed. The local and global features are fused through a weighted sum operation to ensure that the global information is combined with the local details to generate an enhanced feature representation. The feature aggregation process can effectively retain the global geometric structure of the model while strengthening the expression of details. Multi-layer graph convolution operations are performed on the enhanced feature representation. The optimized feature map is extracted by gradually updating the feature representation in multiple convolutional layers. Based on the optimized feature map, upsampling and feature propagation techniques are used, and deconvolution operations are used to restore the original resolution to obtain a more refined refined feature map. Residual connections and jump connections are performed on the refined feature map to obtain a comprehensive feature representation. Residual connections can increase the depth of the network without losing the original features, and jump connections ensure the effective fusion of features at different levels. Based on the comprehensive feature representation, the 3D geometric and texture information are reconstructed. In order to optimize the vertex position and color attributes simultaneously, a multi-task learning framework is adopted to simultaneously handle the optimization of geometry and texture in a unified network, and finally generate the target 3D model.

[0021] Step 500: extract multi-view features based on the target three-dimensional model, and classify and locate defects through a multi-scale convolutional neural network and an attention mechanism to obtain defect category and distribution information; Specifically, the target 3D model is rendered from multiple perspectives. The spherical uniform sampling method is used to sample the 3D model from different directions, generate depth maps and normal maps of 12 perspectives, and construct a multi-perspective feature atlas. The multi-perspective feature atlas is input into the multi-stream convolutional neural network, and the depth map and normal map of each perspective are subjected to residual analysis to obtain the perspective feature tensor. The channel weights of the perspective feature tensor are calculated to evaluate the importance of each perspective in the feature expression and generate weighted perspective features. Through the weighted perspective features, the network can comprehensively process the information of different perspectives, and use the maximum pooling operation to select and fuse the most significant features to generate a global feature representation. The global feature representation uses average pooling of different scales to extract multi-scale features, capture the detailed information of the target at different scales, and ensure that the network has sufficient perception of defects of various sizes. Through top-down paths and lateral connections, the features of different levels are fused to generate a semantically enhanced feature map. By integrating the expressions of features at different levels, the semantic information of the feature expression is made richer and more accurate. The semantically enhanced feature map is processed using a region proposal network. Candidate defect regions are generated using sliding windows and anchor boxes to obtain a set of candidate regions. The region proposal network automatically extracts areas where defects may exist, reduces the computational burden of subsequent classification and positioning, and improves the efficiency and accuracy of detection. Based on the set of candidate regions, feature alignment and classification are performed. Through the RoIAlign layer (region of interest alignment layer), accurate feature extraction is performed on candidate regions of different sizes, and they are converted into feature maps of fixed size to ensure that the features of each candidate region can be consistently represented. The fixed-size feature map is processed using a fully connected layer to output the defect category probability and corresponding bounding box coordinates of each candidate region, and finally accurate defect category and distribution information is obtained.

[0022] Step 600: According to the defect category and distribution information and the geometric features of the precision plastic mold, a ray tracing-guided sequential convex optimization algorithm is used to calculate the optimal scanning detection path and generate an adaptive mold detection strategy.

[0023] Specifically, based on the geometric features of precision plastic molds, the tetrahedron partitioning algorithm is used to discretize the mold surface. The algorithm generates a discretized model by dividing the continuous surface of the mold into a series of tetrahedral grid units. The visibility score of the discretized model is calculated to evaluate the visibility of each grid unit from the perspective of the detection equipment, and a visibility map is obtained to reflect the observability of each part of the mold surface. Based on the visibility map and the acquired defect category and distribution information, the objective function for optimizing the detection path is constructed. The weighted summation method is used to comprehensively consider the three key factors of coverage, detection accuracy and scanning time. Coverage refers to the degree of coverage of the entire mold surface by the scanning path, the detection accuracy reflects the effectiveness of the path in detecting the defect area, and the scanning time is related to the efficiency of the detection process. By comprehensively analyzing these three factors, an optimization objective function is generated to achieve multi-dimensional optimization of path planning. In order to solve the optimization problem, the optimization objective function is sequentially convexly approximated. The first-order Taylor expansion method is used to transform the originally complex non-convex optimization problem into a series of convex subproblems that are easier to handle. Each convex subproblem is solved by a standard convex optimization algorithm to ensure the solvability of the problem and the convergence of the algorithm. After obtaining the set of convex subproblems, the augmented Lagrangian function is constructed, and the dual variables and penalty terms are introduced. The original variables, dual variables and penalty parameters are iteratively solved by the alternating direction multiplier method. In this iterative solution process, the candidate scanning path is gradually optimized to ensure the rationality and effectiveness of the path. Ray tracing simulation is performed based on the candidate scanning path. The imaging process of the multispectral camera and the 3D laser scanner is simulated through the physical rendering engine. This process not only considers the propagation path of light on the mold surface, but also simulates the physical phenomena such as reflection and scattering of light to ensure that the simulation results are consistent with the actual detection environment. The effectiveness of each candidate path is evaluated through the results of ray tracing simulation, and the optimal scanning detection path is finally determined. Combined with the path information, an adaptive mold detection strategy is generated to ensure the accuracy and efficiency of the detection process, so that the detection equipment can efficiently perform comprehensive detection of precision plastic molds and focus on scanning defective areas.

[0024] In the embodiment of the present application, by combining multispectral imaging and three-dimensional laser scanning technology, the present invention can simultaneously obtain the spectral information and geometric information of the mold surface, greatly improving the comprehensiveness and accuracy of defect detection. The multi-scale adaptive threshold segmentation algorithm can effectively handle the illumination changes and material property differences in different regions, and improve the robustness of the segmentation results. The tensor voting surface reconstruction technology combined with the multispectral image restoration network can not only accurately reconstruct the geometric shape of the mold surface, but also restore the texture details of the potential defect area. The model is globally optimized by the non-local refinement network, which effectively improves the accuracy and consistency of the three-dimensional model. Based on the target three-dimensional model, multi-view features are extracted, and combined with multi-scale convolutional neural networks and attention mechanisms, the accuracy of defect classification and positioning is significantly improved. The sequential convex optimization algorithm guided by ray tracing can automatically generate the optimal scanning path according to the defect distribution and mold geometric features, greatly improving the detection efficiency. By comprehensively considering the coverage, detection accuracy and scanning time, the generated adaptive mold detection strategy can maximize efficiency while ensuring the detection quality. The detection process of the present invention is automatic from data acquisition to defect identification to path planning, which greatly reduces manual intervention and realizes efficient, comprehensive and accurate mold quality control.

[0025] In a specific embodiment, the process of executing step 100 may specifically include the following steps: Use a multispectral camera to image the surface of a precision plastic mold, collect image data including visible light, near infrared, and ultraviolet spectrum segments, and obtain an original multispectral image set; Perform dark current correction and flat field correction on the original multispectral image set to eliminate camera noise and illumination inhomogeneity, and obtain a corrected multispectral image set; The spectral response curve is calibrated based on the corrected multispectral image set, the corresponding relationship between the pixel gray value and the actual reflectivity is established, and the reflectivity correction model is obtained; Performing reflectance conversion on the corrected multispectral image set according to the reflectance correction model to obtain multispectral image data; Use structured light 3D laser scanner to scan the surface of precision plastic mold, collect surface morphology information, and obtain original point cloud data; The original point cloud data is subjected to noise filtering and outlier deletion, and the statistical outlier filtering algorithm is used to remove abnormal points to obtain filtered point cloud data; Normal vector estimation is performed based on the filtered point cloud data, and the normal vector of each point is calculated using the covariance analysis method to obtain three-dimensional point cloud data.

[0026] Specifically, a multispectral camera is used to image the mold surface and collect image data including visible light, near infrared and ultraviolet spectral segments. Through multispectral imaging technology, the reflectance characteristics of the mold surface in multiple spectral ranges are obtained simultaneously to generate an original multispectral image set. Dark current correction and flat field correction are performed on the original multispectral image set to remove the signal offset caused by the internal electronic noise of the camera. Flat field correction eliminates the illumination non-uniformity in the image by using a reference image with uniform illumination. Dark current correction can be expressed as: ; in, is the corrected pixel value, is the original pixel value, The noise is generated by the dark current of the camera. Flat field correction is usually performed using a reference image , by dividing each pixel of the original image by the reference pixel value at the corresponding position to ensure uniform illumination. The spectral response curve is calibrated based on the corrected multispectral image set to establish the correspondence between the grayscale value of each pixel and the actual physical reflectance. This response model is established by measuring the images of a series of standard plates with known reflectance. The response curve calibration can be expressed by the following function: ; in, is the wavelength Lower position The actual reflectivity at is the gray value of the corresponding position at this wavelength, and the function Represents the spectral response curve model, which is usually determined by experimental measurement. Based on this model, reflectivity correction is performed to convert the grayscale value of each pixel into actual reflectivity data to obtain a reflectivity correction model. At the same time, a structured light 3D laser scanner is used to scan the surface of the precision plastic mold to collect the surface morphology information of the mold. Structured light scanning calculates 3D point cloud data by projecting a known pattern of light onto the mold surface and capturing the deformation of light at different angles through a camera. The point cloud data consists of a large number of 3D coordinate points, which reflects the geometric shape of the mold surface. The obtained raw point cloud data can describe the overall contour of the mold as well as tiny deformations or defects. Noise filtering and outlier deletion are performed on the raw point cloud data. A statistical outlier filtering algorithm is used, which calculates the distance between the point and the surrounding points based on the local neighborhood information of each point and compares it with the global statistical features. If the distance between a point and its neighborhood exceeds a preset threshold, it is considered to be an outlier and is deleted. The normal vector is estimated for the filtered point cloud data. The normal vector refers to the local geometric direction of each point on the point cloud surface. Using the covariance analysis method, for the point For each point in the cloud, define a neighborhood and calculate the covariance matrix of all points in the neighborhood: ; in, is the covariance matrix, is the number of points in the neighborhood, The first The coordinates of the points, is the centroid of the neighborhood. By performing eigenvalue decomposition on the covariance matrix, the normal vector is obtained. The eigenvector corresponding to the minimum eigenvalue of the covariance matrix is ​​the normal vector of the point. Through the above steps, high-quality three-dimensional point cloud data is finally obtained.

[0027] In a specific embodiment, the process of executing step 200 may specifically include the following steps: Use the Gaussian-Laplacian pyramid algorithm to construct a 5-scale image pyramid, perform multi-scale pyramid decomposition on the multispectral image data, and obtain a multi-scale image set; Calculate the local area statistical features based on the multi-scale image set, and use the sliding window method to calculate the mean and variance of each pixel neighborhood to obtain the feature map; An adaptive threshold function is constructed according to the feature map, and the local statistical features are mapped to the threshold space using the sigmoid function to obtain the threshold mapping function; Based on the threshold mapping function, the multi-scale image set is subjected to a pixel-level binarization operation to obtain a multi-scale binary image set; Based on the multi-scale binary image set, a cross-scale consistency check is performed, and the segmentation results of different scales are fused using the majority voting strategy to obtain the initial segmentation result; Perform opening and closing operations on the initial segmentation results to eliminate noise and smooth the boundaries, and obtain the target segmentation results; Construct a KD tree index structure based on 3D point cloud data, use the nearest neighbor search algorithm to establish the correspondence between 2D pixels and 3D points, and obtain a mapping matrix; The target segmentation result is projected onto the three-dimensional point cloud data according to the mapping matrix, and a label attribute is assigned to each three-dimensional point to obtain the first three-dimensional model.

[0028] Specifically, the Gaussian-Laplacian pyramid algorithm is used to construct image pyramids of 5 different scales. Each layer represents a different spatial resolution, and the resolution of the image decreases layer by layer. The Gaussian pyramid reduces the spatial details of the image by performing Gaussian filtering on the image, while the Laplacian pyramid retains the edge and texture information of the image by calculating the differences between adjacent levels in the Gaussian pyramid. Through pyramid decomposition, the multi-level features of the image can be obtained at multiple scales, capturing key information at different scales. Based on the multi-scale image set obtained by decomposition, the statistical features of the local area of ​​the image at each scale are calculated. The local mean and variance of each pixel are calculated in the neighborhood of the sliding window method. Assume that the size of the sliding window is , the local mean of each pixel in the window and variance Can be expressed as: ; ; in, Indicates the position in the image The pixel value at It's a pixel The local mean of is the corresponding local variance. By moving the sliding window, the local statistical features of the entire image are calculated to generate a feature map. The local statistical features reflect the changing trend and detail information of the image in different areas. Based on the generated feature map, an adaptive threshold function is constructed. In order to achieve adaptive pixel classification, the sigmoid function is used to map the local statistical features to the threshold space to obtain the threshold mapping function. The general form of the sigmoid function is: ; in, is the threshold mapping function, represents local statistical characteristics, such as mean or variance, and are parameters that control the shape of the sigmoid function. By adjusting these two parameters, the sigmoid function can generate smooth thresholds within different feature value ranges, ensuring that the adaptive threshold can flexibly adapt to local feature changes in the image. According to the threshold mapping function, pixel-level binarization operations are performed to convert the multi-scale image set into a binary image set. The binary image can distinguish the foreground and background areas in the image. A cross-scale consistency check is performed based on the multi-scale binary image set. Images of different scales reflect image information at different levels of detail. By performing a consistency check in a cross-scale manner, it can ensure that the segmentation results obtained at multiple scales are consistent. In this process, the majority voting strategy is used to fuse the segmentation results of different scales. The binary results of each pixel at different scales are voted, and the classification result with the highest frequency is selected to obtain the initial segmentation result. The initial segmentation result is subjected to the opening and closing operations in the morphological operation to eliminate the noise in the segmentation result, and the boundary is smoothed to generate a more accurate target segmentation result. The opening operation can remove small foreground noise, while the closing operation can fill small holes in the foreground and improve the overall shape of the segmentation result. Based on the three-dimensional point cloud data, a KD tree index structure is constructed to accelerate the spatial search of point cloud data. KD tree is a structure used to store multi-dimensional spatial data, which can quickly perform nearest neighbor search. Using the nearest neighbor search algorithm, the correspondence between the two-dimensional image pixels and the three-dimensional point cloud data is established, and a mapping matrix is ​​generated to record the three-dimensional point cloud coordinates corresponding to each two-dimensional image pixel, so that the segmentation results in the image can be accurately projected onto the three-dimensional point cloud data. According to the mapping matrix, the target segmentation results are projected from the two-dimensional image space to the three-dimensional point cloud data. Through projection, each three-dimensional point is assigned a corresponding label attribute, indicating the classification result of the point in the image, and the first three-dimensional model is generated.

[0029] In a specific embodiment, the process of executing step 300 may specifically include the following steps: Calculate the local covariance matrix for each point in the first three-dimensional model, and use the K nearest neighbor algorithm to select neighborhood points to obtain the point cloud covariance feature; Construct a tensor field based on the point cloud covariance features, and extract the main direction information of each point to obtain the initial tensor field; Perform tensor voting on the initial tensor field, and use the spherical kernel function to calculate the voting contribution of each point to the neighboring points to obtain the cumulative tensor field; Feature extraction is performed based on the accumulated tensor field, and the saliency and normal vector of each point are calculated to obtain the surface reconstruction features, and Poisson surface reconstruction is performed based on the surface reconstruction features to obtain the reconstructed mesh model; The reconstructed mesh model is topologically optimized to obtain an optimized mesh model, and the input of the multispectral image restoration network is constructed based on the optimized mesh model. The three-dimensional model is projected onto a two-dimensional plane using UV unfolding technology to obtain a UV map. The UV map is input into the multispectral image restoration network, and the texture details are restored using the generative adversarial network architecture and multi-scale receptive field fusion strategy to obtain the second 3D model.

[0030] Specifically, the local covariance matrix is ​​calculated for each point in the first 3D model. The covariance matrix can effectively capture the local geometric features of the point cloud data. The K nearest neighbor algorithm is used to select neighboring points for each point. Given a point , find the neighboring Nearest Neighbor , and construct the covariance matrix based on these points , whose expression is: ; in, It is The coordinates of the neighboring points, is the centroid of the neighborhood of the point. This matrix can describe the geometric structural features of the point in the neighborhood, such as the flatness or curvature of the surface. Based on the covariance matrix, the tensor field of the point cloud is constructed. After the covariance matrix of each point is decomposed by eigenvalue, the eigenvector obtained represents the local main direction information of the point, which is used to construct the initial tensor field. The tensor field is a description of the local structure of the point cloud surface. By extracting the main direction of each point, the local geometric information of the model surface is obtained. Tensor voting operation is performed on the initial tensor field. Each point "votes" for other points in the neighborhood based on the true local structure information to enhance the overall geometric structure representation. The spherical kernel function is used to calculate the voting contribution of each point to its neighborhood points. The spherical kernel function can perform weighted calculations based on the spatial distance and direction between points. The contribution of the vote is usually expressed by the following formula: ; in, Yes For neighboring points The voting weight of represents the Euclidean distance between two points, is the scale parameter that controls the weight decay. By accumulating the weights of all neighborhood points, a cumulative tensor field is obtained, which represents the characteristics of the local structure in the global context. Based on the cumulative tensor field, the saliency and normal vector of each point are extracted. The saliency feature can reflect the importance of the point in the geometric structure, while the normal vector describes the local normal direction of the surface. By extracting these features, feature information for surface reconstruction is obtained. Using the surface reconstruction features, Poisson surface reconstruction is performed. Poisson surface reconstruction is a commonly used 3D reconstruction algorithm that can generate a smooth mesh model through the normal vector information of the point cloud. Poisson reconstruction takes the normal vector information of the point cloud surface as input and solves the Poisson equation to generate a continuous 3D surface. The reconstructed mesh model is topologically optimized to remove unnecessary mesh structures, smooth the surface, and ensure that the geometry and topology of the mesh are reasonable. For example, Laplace smoothing or the centroid method is used to adjust the position of the mesh vertices so that it simplifies the structure while maintaining the geometric shape. The input of the multispectral image restoration network is constructed based on the optimized mesh model. In order to effectively map the texture information on the 3D model to the 2D image, the UV unfolding technology is used to unfold the 3D model to a 2D plane. UV unfolding refers to "flattening" the surface of a three-dimensional mesh into a two-dimensional space so that the texture on the three-dimensional image can be mapped to a two-dimensional image. Through UV unfolding, a UV map is generated, which contains the correspondence between each surface point of the model in the two-dimensional space. The generated UV map is input into the multispectral image restoration network, which uses the generative adversarial network architecture to restore the texture details of the three-dimensional model. In a generative adversarial network, it is usually composed of two networks, a generator and a discriminator. The generator generates high-quality texture details, while the discriminator determines whether the generated texture is realistic by comparing it with the real image. The multispectral image restoration network combines a multi-scale receptive field fusion strategy, so that the details of the model at different scales can be effectively captured and restored, ensuring the accuracy and visual effect of texture restoration. For example, in some high-frequency areas (such as edges or small surface defects), the multi-scale receptive field can restore subtle texture details by capturing local and global information. Generate a second three-dimensional model.

[0031] In a specific embodiment, the process of executing step 400 may specifically include the following steps: Performing grid downsampling on the second three-dimensional model to obtain a multi-resolution grid sequence, and inputting the multi-resolution grid sequence into a graph convolutional network, using a Laplacian matrix to encode the grid topology structure to obtain a graph structure representation; Perform non-local attention mechanism analysis on the graph structure representation, use the self-attention algorithm to calculate the global correlation matrix, and obtain the attention weight map; Feature aggregation is performed based on the attention weight map, and the weighted sum operation is used to fuse local and global features to obtain enhanced feature representation; Perform multi-layer graph convolution operations on the enhanced feature representation to obtain an optimized feature map, and then perform upsampling and feature propagation based on the optimized feature map, and use deconvolution operations to restore the original resolution to obtain a refined feature map; Residual connections and skip connections are performed on the refined feature map to obtain a comprehensive feature representation, and the three-dimensional geometry and texture information are reconstructed based on the comprehensive feature representation. The vertex position and color attributes are simultaneously optimized using a multi-task learning framework to obtain the target three-dimensional model.

[0032] Specifically, the second 3D model is meshed downsampled to reduce the number of vertices in the mesh, thereby reducing the complexity of the model while maintaining the basic features of the overall geometric shape to generate a multi-resolution mesh sequence. The downsampling process selectively removes certain vertices in the mesh and reconnects the remaining vertices to obtain multiple mesh representations with different resolutions. The multi-resolution mesh sequence is input into the graph convolutional network. The Laplacian matrix is ​​used to encode the topological structure of the mesh. Laplacian matrix Used to describe the connection relationship between vertices in a mesh, usually calculated by the following formula: ; in, is the adjacency matrix, which represents the connection relationship between mesh vertices; and is a degree matrix whose diagonal elements represent the degree of each vertex (i.e., the number of neighbors of the vertex). Through the Laplacian matrix, the topological structure of the grid is encoded as a graph structure, so that the graph convolutional network can process the local and global features of the three-dimensional grid. Based on the graph structure representation obtained by the Laplacian matrix, the non-local attention mechanism is analyzed. By calculating the global correlation between any two vertices on the grid, the overall geometric information of the grid is captured. Using the self-attention algorithm, a global correlation matrix is ​​generated, which reflects the similarity between each vertex and other vertices. The calculation of self-attention can be expressed as: ; in, Represents a vertex With Vertex The correlation weight between and is the vertex and The characteristic representation of function is a function used to calculate similarity. In this way, an attention weight map is generated to assign each vertex a weight of relevance to all other vertices. Feature aggregation is performed based on the attention weight map to fuse the local and global features of each vertex and enhance its feature representation. , its enhanced feature representation It can be calculated by a weighted sum operation: ; The weighted sum operation can fuse the features of the local neighborhood and the global correlation features, so that the representation of each vertex can more accurately reflect its position and role in the entire grid structure. Multi-layer graph convolution operations are performed on the enhanced feature representation, and the features are gradually propagated through the adjacency relationship of the vertices, so that the features can be continuously extracted and enhanced at multiple levels. Through the operation of multiple graph convolution layers, an optimized feature map is obtained, which contains the geometric details of the model at different levels. In order to restore the original resolution of the model, upsampling and feature propagation are performed based on the feature map. Upsampling converts the low-resolution feature map back to a high-resolution form so that the geometric details of the model can be reconstructed. In this process, deconvolution operations are used to restore features. Deconvolution is an operation used in the reverse convolution process to generate high-resolution output by expanding low-resolution features. Residual connections and skip connections are performed on the refined feature map. The purpose of residual connections is to prevent the degradation of features during propagation, while skip connections can combine features at different levels to generate a comprehensive feature representation. Based on the comprehensive feature representation, 3D geometry and texture information are reconstructed. In order to optimize the position and color attributes of vertices at the same time, a multi-task learning framework is adopted. In this framework, geometric position and color attributes are considered as two related tasks, which share the same feature representation but optimize different loss functions. The optimization goal of geometric position is to minimize the reconstruction error of vertex position, while the optimization goal of color attribute is to minimize the error of texture reconstruction. Through multi-task learning, the geometry and texture information of the model are optimized simultaneously to generate a high-precision target 3D model.

[0033] In a specific embodiment, the process of executing step 500 may specifically include the following steps: Perform multi-view rendering on the target 3D model, use the spherical uniform sampling method to generate depth maps and normal maps of 12 views, and obtain a multi-view feature atlas; The multi-view feature atlas is input into the multi-stream convolutional neural network, and the depth map and normal map of each view are subjected to residual analysis to obtain the view feature tensor. Calculate the channel weights of the view feature tensor to obtain weighted view features, perform view fusion based on the weighted view features, use the maximum pooling operation to select the most significant features, and obtain the global feature representation; The global feature representation is represented by using average pooling of different scales to extract multi-scale features to obtain a multi-scale feature map. The multi-scale feature map is then fused with features of different levels using a top-down path and lateral connections to obtain a semantically enhanced feature map. The region proposal network uses a sliding window and anchor box to generate candidate defect regions for the semantic enhancement feature map, and obtains a set of candidate regions. Feature alignment and classification are performed based on the candidate region set. The RoIAlign layer is used to extract the fixed-size feature map, and the defect category probability and bounding box coordinates are output through the fully connected layer to obtain the defect category and distribution information.

[0034] Specifically, the multi-view rendering technology is used to perform spherical uniform sampling on the model to generate depth maps and normal maps of multiple views. Spherical uniform sampling ensures uniform coverage of the view. Usually 12 viewpoints are selected for sampling to capture the surface information of the three-dimensional model from different angles. The depth map of each viewpoint can reflect the distance between the model surface and the camera at that viewpoint, while the normal map records the information of the surface normal vector, that is, the direction of each point on the surface. Through these renderings, a multi-view feature atlas is generated. The multi-view feature atlas is input into the multi-stream convolutional neural network. The multi-stream convolutional network can process data streams from different channels at the same time. The depth map and the normal map are used as independent input streams for residual analysis. Residual analysis is a common image processing technology that aims to extract detail changes in the image. By performing residual calculation on the depth map and normal map of each viewpoint, the differences and detail features between the viewpoints are extracted to form a view feature tensor. Channel weight calculation is performed on the view feature tensor, and different weights are assigned to each feature through channel weight calculation to obtain weighted view features. Based on the weighted view features, the view fusion is performed to integrate the information from different view points to form a unified feature representation. The most significant features are selected using the maximum pooling operation. Maximum pooling retains the most prominent information by selecting the maximum value in the local area. Through maximum pooling, the most significant global features are extracted from the features of multiple view points to generate a global feature representation. Multi-scale features are extracted using average pooling of different scales for the global feature representation. Average pooling is a method that takes the average value of multiple pixels as the output. It captures features at different resolutions by pooling at different scales. The purpose of multi-scale feature extraction is to ensure that the network can process large-scale global features and small-scale local details at the same time, so as to comprehensively analyze the model and obtain a multi-scale feature map. In order to improve the expressiveness of the features, the top-down path and lateral connections are used to fuse features at different levels. The top-down path is used to pass high-level semantic information back to the underlying feature map, while the lateral connection can combine features of different resolutions to generate a semantically enhanced feature map. The region proposal network uses sliding windows and anchor boxes to generate candidate defect regions for the semantically enhanced feature map. The sliding window method slides on the feature map with a fixed step size, and the anchor box is a preset rectangular box used to locate potential defects. The region proposal network generates multiple anchor boxes of different scales and aspect ratios at each sliding window position and scores each anchor box to determine the candidate defect area and obtain a set of candidate regions containing potential defects. The candidate region set is feature aligned and classified. A fixed-size feature map is extracted using the RoIAlign layer (Region of Interest Align). RoIAlign is an improved feature pooling method that aligns the features of the candidate region through more precise interpolation operations to ensure that important details are not lost during transformation.Through the RoIAlign layer, each candidate region is resized to a uniform feature map size. The extracted feature map is input into the fully connected layer, and classification and bounding box regression are performed through the fully connected layer. The purpose of classification is to determine the defect category in each candidate region, while bounding box regression is to accurately locate the specific location of the defect. Classification can calculate the probability of each category through the Softmax function, and the coordinates of the bounding box are predicted by the regression model. Output the category probability of each defect and its bounding box coordinates on the 3D model.

[0035] In a specific embodiment, the process of executing step 600 may specifically include the following steps: Based on the geometric features of the precision plastic mold, the mold surface is discretized using the tetrahedron partitioning algorithm to obtain a discretized model; Calculate the visibility score of each grid cell of the discretized model to obtain a visibility map; The objective function is constructed based on the visibility graph and defect category and distribution information. The weighted sum method is used to comprehensively analyze the coverage, detection accuracy and scanning time to obtain the optimized objective function. Perform a sequential convex approximation on the optimization objective function and use the first-order Taylor expansion to transform the non-convex optimization problem into a sequence of convex subproblems to obtain a set of convex subproblems. Based on the set of convex subproblems, an augmented Lagrangian function is constructed and dual variables and penalty terms are introduced. Based on the alternating direction multiplier method, the candidate scanning paths are obtained by iteratively solving the original variables, dual variables and penalty parameters. Ray tracing simulation is performed based on the candidate scanning paths, and the physical rendering engine is used to simulate the imaging process of the multispectral camera and 3D laser scanner to obtain the optimal scanning inspection path and adaptive mold inspection strategy.

[0036] Specifically, based on the geometric features of precision plastic molds, the mold surface is discretized using a tetrahedron partitioning algorithm. Tetrahedron partitioning is to divide a three-dimensional continuous model into multiple non-overlapping tetrahedral units, thereby discretizing a continuous geometric body into a grid model to obtain a discretized model. For the discretized model, the visibility score of each grid unit is calculated. The visibility score reflects whether the unit can be effectively observed from a specified viewing angle. When calculating the visibility score, a ray tracing algorithm is used to simulate the light emitted from a multispectral camera or laser scanner, and each grid unit is judged by the occlusion relationship of the geometric model to determine whether it is blocked by the line of sight. For visible units, a higher visibility score is assigned; for units that are blocked or cannot be directly observed, a lower visibility score is assigned. Through this process, a visibility map of the entire mold surface is generated, providing the visibility of different grid units at various viewing angles. Based on the visibility map and the known defect categories and distribution information, an objective function is constructed to optimize the scanning path. When constructing the objective function, multiple factors are considered comprehensively, including the coverage of the scanning path, the detection accuracy, and the required scanning time. Coverage refers to the degree of coverage of the grid cells on the mold surface by the scanning path, detection accuracy reflects the accuracy of detecting defects at different viewing angles, and scanning time is closely related to the length and complexity of the path. In order to integrate these factors, the objective function can be expressed in a weighted summation manner: ; in, is the value of the objective function, is the coverage of the scan path, is the detection accuracy, is the scan time, , , are weight coefficients, which are used to balance the influence of different factors. By adjusting these weight coefficients, the coverage, accuracy and time are weighed according to actual needs. Using the method of sequential convex approximation, the non-convex optimization problem is transformed into a series of convex sub-problems that are easy to solve. The objective function is linearized into an approximate form of a convex function by performing a first-order Taylor expansion. Assume that a non-convex function , its Taylor expansion is approximated as: ; in, At the initial point The function value at is its gradient. Through this linear approximation, the original non-convex optimization problem is transformed into a series of convex subproblems, and a set of convex subproblems is obtained. Based on the set of convex subproblems, the augmented Lagrangian function is constructed, and dual variables and penalty terms are introduced to constrain the optimization process. The augmented Lagrangian function is used to deal with optimization problems with constraints. By adding dual variables and penalty terms to the objective function, the original problem can be solved in the dual space. Its expression is: ; in, is the augmented Lagrangian function, is the objective function, is a constraint, is the dual variable, is the penalty term coefficient. By alternating direction multiplication method, iteratively solve the original variable , dual variable And the penalty parameter . The alternating direction multiplier method can effectively handle complex nonlinear optimization problems by breaking down the problem into multiple sub-problems and solving them step by step. After multiple iterations, candidate scanning paths are obtained. Ray tracing simulation is performed based on the candidate scanning paths to verify and optimize these paths. The imaging process of the multispectral camera and 3D laser scanner is simulated by the physical rendering engine to evaluate the performance of each scanning path in actual detection. Ray tracing simulation simulates the scanning process in a real environment by tracing the propagation of light on the mold surface, including physical phenomena such as reflection, refraction, and occlusion of light. Through simulation, the feasibility of the candidate path is effectively evaluated to ensure that it can cover the key areas of the mold surface and achieve high-precision detection in the shortest time. Through ray tracing simulation and path optimization, the optimal scanning detection path is determined. This path minimizes the scanning time while ensuring high coverage and high accuracy. Combined with the optimization results, an adaptive mold detection strategy is generated, which can not only be dynamically adjusted according to different mold features, but also achieve efficient defect detection under complex geometric shapes.

[0037] The execution step constructs an augmented Lagrangian function based on a set of convex subproblems and introduces dual variables and penalty terms. Based on the alternating direction multiplier method, the process of iteratively solving the original variables, dual variables and penalty parameters to obtain the candidate scanning path can specifically include the following steps: For each subproblem in the set of convex subproblems, an augmented Lagrangian function is constructed, and Lagrangian multipliers and quadratic penalty terms are introduced to obtain the augmented form of the optimization problem. Based on the augmented form of the optimization problem, the original problem is decomposed into a primal variable subproblem and a dual variable subproblem to obtain two independent optimization subproblems. The proximal gradient descent algorithm is applied to the primal variable subproblem, and the intermediate solution of the primal variable is obtained by iteratively calculating the gradient and updating the variable. Based on the intermediate solution of the primal variable, the dual variable subproblem is updated with a dual ascent, and the Lagrangian multipliers are calculated using the closed-form solution to obtain the dual variable subproblem. The updated value of the quantity; according to the intermediate solution of the original variable and the updated value of the dual variable, the original residual and the dual residual are calculated, and the penalty parameter is dynamically adjusted based on the residual value to obtain an adaptive penalty factor; the adaptive penalty factor is subjected to convergence analysis, and the iteration termination condition is judged using a preset threshold to obtain an iteration termination criterion; based on the iteration termination criterion, a loop iteration is performed to repeatedly execute the original variable update, the dual variable update and the penalty parameter adjustment to obtain the optimal variable value; the scanning path is reconstructed according to the optimal variable value, and the optimization result is mapped back to the original problem space to obtain a candidate scanning path.

[0038] In this embodiment, an augmented Lagrangian function is constructed: for a given convex subproblem minf(x)stAx=b, where x is the scan path parameter, A is the constraint matrix, and b is the constraint vector, an augmented Lagrangian function is constructed: L(x,y,ρ)=f(x)+yT(Ax-b)+(ρ / 2)||Ax-b||2; where y is the Lagrangian multiplier and ρ is the penalty parameter. Problem decomposition: The original problem is decomposed into two subproblems: x-subproblem: minxL(x,yk,ρk); y-subproblem: maxyL(x(k+1),y,ρk); where k represents the number of iterations. Solving the original variable subproblem: Use the proximal gradient descent algorithm to solve the x-subproblem: x(k+1)=prox(f / ρk)(xk-(1 / ρk)∇xL(xk,yk,ρk)); where prox represents the proximal operator and ∇xL represents the gradient of L with respect to x. Dual variable update: Use the closed-form solution to update the Lagrange multiplier: y(k+1)=yk+ρk(Ax(k+1)-b); Residual calculation and penalty parameter adjustment: Calculate the original residual and the dual residual: r(k+1)=Ax(k+1)-b; s(k+1)=ρkAT(x(k+1)-xk); Adjust the penalty parameter according to the residual: if ||r(k+1)||>μ||s(k+1)||, then ρ(k+1)=τρk; if ||s(k+1)||>μ||r(k+1)||, then ρ(k+1)=ρk / τ; otherwise ρ(k+1)=ρk; where μ>1 and τ>1 are preset parameters. Convergence analysis: Define the iteration termination condition: ||r(k+1)||≤εpri and ||s(k+1)||≤εdual; where εpri and εdual are the preset primal and dual tolerances. Loop iteration: until the termination condition is met or the maximum number of iterations is reached. Scan path reconstruction: Map the final optimization variable x* back to the original problem space to obtain the candidate scan path: path = Γ(x*); where Γ is the mapping function from the optimization variable to the actual scan path.

[0039] The above describes the precision plastic mold detection method in the embodiment of the present application. The following describes the precision plastic mold detection device 10 in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of a precision plastic mold detection device 10 includes: Scanning module 11, used to perform multispectral imaging and three-dimensional laser scanning on the surface of the precision plastic mold to obtain multispectral image data and three-dimensional point cloud data; The segmentation module 12 is used to perform multi-scale adaptive threshold segmentation on the multispectral image data to obtain a target segmentation result, and map the target segmentation result to three-dimensional point cloud data to obtain a first three-dimensional model; A reconstruction module 13 is used to perform tensor voting surface reconstruction based on the first three-dimensional model and use a multispectral image restoration network to perform texture restoration on the potential defect area to obtain a second three-dimensional model; An optimization module 14, configured to globally optimize the second three-dimensional model through a non-local refinement network to obtain a target three-dimensional model; The classification module 15 is used to extract multi-view features based on the target three-dimensional model, and classify and locate defects through a multi-scale convolutional neural network and an attention mechanism to obtain defect category and distribution information; The generation module 16 is used to calculate the optimal scanning detection path using a ray tracing-guided sequential convex optimization algorithm according to the defect category and distribution information and the geometric features of the precision plastic mold, and generate an adaptive mold detection strategy.

[0040] Through the synergy of the above components, by combining multispectral imaging and three-dimensional laser scanning technology, the present invention can simultaneously obtain the spectral information and geometric information of the mold surface, greatly improving the comprehensiveness and accuracy of defect detection. The multi-scale adaptive threshold segmentation algorithm can effectively handle the illumination changes and material property differences in different regions, and improve the robustness of the segmentation results. The tensor voting surface reconstruction technology combined with the multispectral image restoration network can not only accurately reconstruct the geometric shape of the mold surface, but also restore the texture details of the potential defect area. The model is globally optimized by the non-local refinement network, which effectively improves the accuracy and consistency of the three-dimensional model. Based on the target three-dimensional model, multi-view features are extracted, and combined with multi-scale convolutional neural networks and attention mechanisms, the accuracy of defect classification and positioning is significantly improved. The sequential convex optimization algorithm guided by ray tracing can automatically generate the optimal scanning path according to the defect distribution and mold geometric features, greatly improving the detection efficiency. By comprehensively considering the coverage, detection accuracy and scanning time, the generated adaptive mold detection strategy can maximize efficiency while ensuring the detection quality. The detection process of the present invention is automatic from data acquisition to defect identification to path planning, which greatly reduces manual intervention and realizes efficient, comprehensive and accurate mold quality control.

[0041] The present application also provides an electronic device, which includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the precision plastic mold detection method in the above-mentioned embodiments.

[0042] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the precision plastic mold detection method.

[0043] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0044] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0045] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A precision plastic mold detection method, characterized in that: The precision plastic mold detection method comprises: Perform multispectral imaging and 3D laser scanning on the surface of precision plastic molds to obtain multispectral image data and 3D point cloud data; Performing multi-scale adaptive threshold segmentation on the multispectral image data to obtain a target segmentation result, and mapping the target segmentation result to the three-dimensional point cloud data to obtain a first three-dimensional model; Performing tensor voting surface reconstruction based on the first three-dimensional model and restoring texture of potential defect areas using a multispectral image restoration network to obtain a second three-dimensional model; Performing global optimization on the second three-dimensional model through a non-local refinement network to obtain a target three-dimensional model; Extracting multi-view features based on the target three-dimensional model, and classifying and locating defects through a multi-scale convolutional neural network and an attention mechanism to obtain defect category and distribution information; According to the defect category and distribution information and the geometric features of the precision plastic mold, a ray tracing-guided sequential convex optimization algorithm is used to calculate the optimal scanning detection path and generate an adaptive mold detection strategy.

2. The precision plastic mold detection method according to claim 1, characterized in that: The multispectral imaging and three-dimensional laser scanning of the surface of the precision plastic mold are performed to obtain multispectral image data and three-dimensional point cloud data, including: Use a multispectral camera to image the surface of a precision plastic mold, collect image data including visible light, near infrared, and ultraviolet spectrum segments, and obtain an original multispectral image set; Performing dark current correction and flat field correction on the original multispectral image set to eliminate camera noise and illumination non-uniformity, thereby obtaining a corrected multispectral image set; Based on the corrected multispectral image set, spectral response curve calibration is performed to establish a corresponding relationship between pixel grayscale value and actual reflectivity, thereby obtaining a reflectivity correction model; Performing reflectance conversion on the corrected multispectral image set according to the reflectance correction model to obtain multispectral image data; Scanning the surface of the precision plastic mold using a structured light three-dimensional laser scanner to collect surface morphology information and obtain original point cloud data; Noise filtering and outlier deletion are performed on the original point cloud data, and outlier points are removed using a statistical outlier filtering algorithm to obtain filtered point cloud data; Normal vector estimation is performed based on the filtered point cloud data, and the normal vector of each point is calculated using a covariance analysis method to obtain three-dimensional point cloud data.

3. The precision plastic mold detection method according to claim 1, characterized in that: The step of performing multi-scale adaptive threshold segmentation on the multispectral image data to obtain a target segmentation result, and mapping the target segmentation result to the three-dimensional point cloud data to obtain a first three-dimensional model includes: Using the Gaussian-Laplacian pyramid algorithm to construct an image pyramid of 5 scales, performing multi-scale pyramid decomposition on the multispectral image data to obtain a multi-scale image set; Calculating local area statistical features based on the multi-scale image set, and using a sliding window method to calculate the mean and variance of each pixel neighborhood to obtain a feature map; Constructing an adaptive threshold function according to the feature map, and using a sigmoid function to map the local statistical features to a threshold space to obtain a threshold mapping function; Based on the threshold mapping function, performing a pixel-level binarization operation on the multi-scale image set to obtain a multi-scale binary image set; Performing a cross-scale consistency check based on the multi-scale binary image set, and fusing segmentation results of different scales using a majority voting strategy to obtain an initial segmentation result; Performing opening and closing operations on the initial segmentation result to eliminate noise and smooth the boundaries, thereby obtaining a target segmentation result; Constructing a KD tree index structure based on the three-dimensional point cloud data, using a nearest neighbor search algorithm to establish a correspondence between two-dimensional pixels and three-dimensional points, and obtaining a mapping matrix; The target segmentation result is projected onto the three-dimensional point cloud data according to the mapping matrix, and a label attribute is assigned to each three-dimensional point to obtain a first three-dimensional model.

4. The precision plastic mold detection method according to claim 1, characterized in that: The tensor voting surface reconstruction is performed based on the first three-dimensional model, and the texture of the potential defect area is restored using a multispectral image restoration network to obtain a second three-dimensional model, including: Calculating a local covariance matrix for each point in the first three-dimensional model, and selecting neighborhood points using a K-nearest neighbor algorithm to obtain a point cloud covariance feature; Constructing a tensor field based on the point cloud covariance feature, and extracting the main direction information of each point to obtain an initial tensor field; Performing tensor voting on the initial tensor field, and using a spherical kernel function to calculate the voting contribution of each point to the neighboring points, to obtain a cumulative tensor field; Performing feature extraction based on the accumulated tensor field, and calculating the saliency and normal vector of each point to obtain surface reconstruction features, and performing Poisson surface reconstruction based on the surface reconstruction features to obtain a reconstructed mesh model; Topologically optimize the reconstructed mesh model to obtain an optimized mesh model, construct an input of a multispectral image restoration network based on the optimized mesh model, and use UV unfolding technology to project the three-dimensional model onto a two-dimensional plane to obtain a UV map; The UV map is input into the multispectral image restoration network, and the texture details are restored using a generative adversarial network architecture and a multi-scale receptive field fusion strategy to obtain a second three-dimensional model.

5. The precision plastic mold detection method according to claim 1, characterized in that: The step of globally optimizing the second three-dimensional model by using a non-local refinement network to obtain a target three-dimensional model includes: Performing grid downsampling on the second three-dimensional model to obtain a multi-resolution grid sequence, and inputting the multi-resolution grid sequence into a graph convolutional network, using a Laplacian matrix to encode the grid topology structure to obtain a graph structure representation; Performing a non-local attention mechanism analysis on the graph structure representation, using a self-attention algorithm to calculate a global correlation matrix, and obtaining an attention weight map; Performing feature aggregation based on the attention weight map, fusing local and global features using a weighted sum operation to obtain an enhanced feature representation; Performing a multi-layer graph convolution operation on the enhanced feature representation to obtain an optimized feature map, performing upsampling and feature propagation based on the optimized feature map, and restoring the original resolution using a deconvolution operation to obtain a refined feature map; Residual connections and jump connections are performed on the refined feature map to obtain a comprehensive feature representation, and three-dimensional geometry and texture information are reconstructed based on the comprehensive feature representation. The vertex positions and color attributes are simultaneously optimized using a multi-task learning framework to obtain a target three-dimensional model.

6. The precision plastic mold detection method according to claim 1, characterized in that: The multi-view features are extracted based on the target three-dimensional model, and defect classification and positioning are performed through a multi-scale convolutional neural network and an attention mechanism to obtain defect category and distribution information, including: Performing multi-view rendering on the target three-dimensional model, generating depth maps and normal maps of 12 viewpoints using a spherical uniform sampling method, and obtaining a multi-view feature atlas; Inputting the multi-view feature atlas into a multi-stream convolutional neural network, performing residual analysis on the depth map and normal map of each view, respectively, to obtain a view feature tensor; Performing channel weight calculation on the perspective feature tensor to obtain weighted perspective features, performing perspective fusion based on the weighted perspective features, using a maximum pooling operation to select the most significant features, and obtaining a global feature representation; The global feature representation is represented by using average pooling of different scales to extract multi-scale features to obtain a multi-scale feature map, and the multi-scale feature map is fused with features of different levels using a top-down path and lateral connections to obtain a semantically enhanced feature map; Generate candidate defect regions for the semantic enhancement feature map using a sliding window and an anchor box through a region proposal network to obtain a candidate region set; Feature alignment and classification are performed based on the candidate region set, a fixed-size feature map is extracted using the RoIAlign layer, and the defect category probability and bounding box coordinates are output through the fully connected layer to obtain the defect category and distribution information.

7. The precision plastic mold detection method according to claim 1, characterized in that: The method of calculating the optimal scanning detection path based on the defect category and distribution information and the geometric features of the precision plastic mold using a ray tracing-guided sequential convex optimization algorithm to generate an adaptive mold detection strategy includes: Based on the geometric features of the precision plastic mold, the mold surface is discretized using a tetrahedron partitioning algorithm to obtain a discretized model; Calculating the visibility score of each grid unit of the discretized model to obtain a visibility map; constructing an objective function based on the visibility graph and the defect category and distribution information, and using a weighted summation method to comprehensively analyze coverage, detection accuracy, and scanning time to obtain an optimized objective function; Performing a sequence convex approximation on the optimization objective function, using a first-order Taylor expansion to transform the non-convex optimization problem into a sequence of convex subproblems, and obtaining a set of convex subproblems; Based on the convex subproblem set, an augmented Lagrangian function is constructed and dual variables and penalty terms are introduced. Based on the alternating direction multiplier method, the candidate scanning path is obtained by iteratively solving the original variable, the dual variable and the penalty parameter; Ray tracing simulation is performed based on the candidate scanning path, and the imaging process of the multispectral camera and the three-dimensional laser scanner is simulated using a physical rendering engine to obtain the optimal scanning detection path and the adaptive mold detection strategy.

8. A precision plastic mold detection device, characterized in that: Used to perform the precision plastic mold detection method according to any one of claims 1 to 7, the precision plastic mold detection device comprises: Scanning module, used to perform multispectral imaging and three-dimensional laser scanning on the surface of precision plastic molds to obtain multispectral image data and three-dimensional point cloud data; A segmentation module, used for performing multi-scale adaptive threshold segmentation on the multispectral image data to obtain a target segmentation result, and mapping the target segmentation result to the three-dimensional point cloud data to obtain a first three-dimensional model; A reconstruction module, used for performing tensor voting surface reconstruction based on the first three-dimensional model and restoring the texture of the potential defect area using a multispectral image restoration network to obtain a second three-dimensional model; An optimization module, used for globally optimizing the second three-dimensional model through a non-local refinement network to obtain a target three-dimensional model; A classification module is used to extract multi-view features based on the target three-dimensional model, and classify and locate defects through a multi-scale convolutional neural network and an attention mechanism to obtain defect category and distribution information; A generation module is used to calculate the optimal scanning detection path based on the defect category and distribution information and the geometric characteristics of the precision plastic mold using a ray tracing-guided sequential convex optimization algorithm to generate an adaptive mold detection strategy.

9. An electronic device, characterized in that: The electronic device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the electronic device executes the precision plastic mold detection method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the precision plastic mold detection method as described in any one of claims 1-7 is implemented.

Images