Three-dimensional size detection control method and system based on structured light camera

Abstract

The invention discloses a three-dimensional size detection control method and system based on a structured light camera, and the method comprises the steps: scanning a to-be-detected product based on the structured light camera, and constructing an actual three-dimensional model of the to-be-detected product; constructing a standard three-dimensional model of the to-be-tested product by utilizing a three-dimensional modeling technology, performing matching analysis in combination with an actual three-dimensional model of the to-be-tested product, judging an area with production defects, and performing preparation quality evaluation; if the to-be-detected product is an unqualified product, analyzing the production qualification rate of other products in the same batch, judging whether production defect optimization needs to be carried out or not, and carrying out production process traceability to obtain production process traceability information; and constructing a defect optimization model, and if production defect optimization needs to be carried out, inputting the production process traceability information into the defect optimization model to generate an optimization scheme to manage and control the corresponding production process. The accuracy and efficiency of size detection are improved, and the production process is optimized to improve the production quality of a production line.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional size detection technology, and in particular to a three-dimensional size detection control method and system based on a structured light camera. Background Art

[0002] In recent years, structured light imaging technology has received widespread attention. Structured light technology uses the deformation of the light spot projected onto the surface of an object, captures the image through a camera, and combines computer vision algorithms to reconstruct the three-dimensional shape of the object. This method can quickly and accurately obtain three-dimensional data of the surface of an object without directly contacting the object, and has high measurement accuracy and strong adaptability. Unlike traditional contact measurement, structured light measurement not only avoids the errors caused by physical contact, but also can efficiently measure hard-to-reach objects such as transparent, reflective or complex surfaces.

[0003] Traditional dimension detection methods, such as tactile measurement or optical measurement, have been widely used in some fields, but they usually have disadvantages such as low measurement efficiency, complex operation, and limited measurement range. Especially in the detection of products with complex shapes or small sizes, these traditional methods often cannot meet the requirements of high precision and high efficiency. Therefore, how to improve the accuracy and efficiency of dimension detection and improve production quality has become an urgent problem to be solved. Summary of the invention

[0004] The present invention overcomes the defects of the prior art and provides a three-dimensional size detection control method and system based on a structured light camera, the important purpose of which is to improve the accuracy and efficiency of size detection and improve the production quality of the production line.

[0005] To achieve the above-mentioned purpose, the first aspect of the present invention provides a three-dimensional size detection and control method based on a structured light camera, comprising:

[0006] The product to be tested is scanned using a structured light camera, and phase analysis is performed based on the scanning results to obtain 3D depth information and convert it into point cloud data to build an actual 3D model of the product to be tested.

[0007] Based on the preparation instructions of the product to be tested, a standard 3D model of the product to be tested is constructed using 3D modeling technology. Matching analysis is performed in combination with the actual 3D model of the product to be tested to determine the areas with production defects and conduct preparation quality assessment;

[0008] If the product to be tested is a substandard product, the production qualification rate of the remaining products in the same batch is analyzed to determine whether production defect optimization is needed, and the production process is traced to obtain production process traceability information;

[0009] Construct a defect optimization model. If production defect optimization is required, input the production process traceability information into the defect optimization model to generate an optimization plan to control the corresponding production process.

[0010] In this solution, the structured light camera is used to scan the product to be tested, and phase calculation is performed according to the scanning results to obtain three-dimensional depth information and convert it into point cloud data to build an actual three-dimensional model of the product to be tested, which specifically includes:

[0011] Scan the product to be tested in the detection area based on the structured light camera according to the preset detection scheme, and obtain the surface reflected light pattern of the product to be tested in each preset scanning area;

[0012] Preprocessing the acquired surface reflected light patterns of each preset scanning area, and extracting the light fringe phase characteristics of the surface reflected light patterns of each preset scanning area by using a Fourier transform method;

[0013] The phase difference of different preset scanning areas is calculated according to the extracted light stripe phase characteristics, and the phase solution is performed to obtain the three-dimensional depth information of the product to be tested, and the three-dimensional depth information is converted and processed to obtain the point cloud data of the product to be tested;

[0014] The point cloud data of the product to be tested is registered, and the registered point cloud data is synthesized to generate a point cloud data model of the product to be tested. The point cloud data model is converted into a continuous surface model based on the triangulation method. The surface model is rendered and visualized to generate an actual three-dimensional model of the product to be tested.

[0015] In this solution, the standard 3D model of the product to be tested is constructed based on the preparation instructions of the product to be tested using 3D modeling technology, and matching analysis is performed in combination with the actual 3D model of the product to be tested to determine the area with production defects and perform preparation quality assessment, specifically including:

[0016] Obtaining a preparation instruction of the product to be tested, constructing a standard three-dimensional model of the product to be tested by using three-dimensional modeling technology based on the preparation instruction of the product to be tested, and performing geometric element decomposition according to the constructed standard three-dimensional model;

[0017] Obtaining the decomposition results of geometric elements of the standard three-dimensional model and extracting key points based on the obtained geometric elements, calculating the feature descriptors of each key point, wherein the feature descriptors are used to describe the local geometric properties of each key point, and generating a feature map of the standard three-dimensional model;

[0018] Acquire an actual three-dimensional model of the product to be tested, decompose geometric elements according to the actual three-dimensional model of the product to be tested and construct a feature map of the actual three-dimensional model, and align the feature map of the actual three-dimensional model with the feature map of the standard three-dimensional model;

[0019] A hash algorithm is introduced to construct a hash table based on the actual three-dimensional model feature map and the standard three-dimensional model feature map to obtain a first hash table and a second hash table, and the first hash table and the second hash table are matched and analyzed;

[0020] Calculate the hash distance between the feature descriptor of each key point in the actual 3D model feature map and the feature descriptor of each key point in the standard 3D model feature map according to the matching analysis result;

[0021] The calculated hash distance is compared with the preset threshold, and the key points less than the preset threshold are defined as matching point pairs according to the judgment result. The feature descriptors corresponding to the matching point pairs obtained based on the analysis are marked in the actual 3D model to generate the marked actual 3D model.

[0022] In this solution, the standard 3D model of the product to be tested is constructed based on the preparation instructions of the product to be tested using 3D modeling technology, and matching analysis is performed in combination with the actual 3D model of the product to be tested to determine the area with production defects and perform preparation quality assessment, which also includes:

[0023] Preset the area division rule and divide the marked actual 3D model into several areas, extract the number of matching point pairs in each sub-area, and determine whether the corresponding sub-area is an abnormal area according to the number of matching point pairs in each sub-area to obtain abnormal area analysis information;

[0024] Based on the abnormal area analysis information extraction, the three-dimensional size features of the abnormal area are extracted using the actual three-dimensional model of the product to be tested to obtain the three-dimensional size feature information of the abnormal area;

[0025] Acquire preparation standard data of the product to be tested, calculate the three-dimensional size feature information of the abnormal area and the preparation standard data of the product to be tested, obtain the deviation value between the three-dimensional size of the abnormal area and the preparation standard, and obtain deviation value information;

[0026] Acquire preparation requirement information of the product to be tested, formulate quality assessment rules according to the preparation requirement information of the product to be tested, judge the deviation value information with the quality assessment rules, analyze the preparation quality of the target product to be tested, and obtain preparation quality analysis information;

[0027] Based on the preparation quality analysis information, it is determined whether the target product to be tested is a qualified product. If it is not a qualified product, the target product to be tested is marked as an unqualified product.

[0028] In this solution, if the product to be tested is an unqualified product, the production qualification rate of the remaining products in the same batch is analyzed to determine whether production defect optimization is needed, and the production process is traced to obtain the production process traceability information, which specifically includes:

[0029] If the target product to be tested is a product that fails to meet the requirements of production, the three-dimensional dimension test results of other products in the same production batch of the target product to be tested are obtained, and a production quality analysis is performed, and the number of failed products in the same production batch is counted according to the analysis results;

[0030] The statistical result is compared with the preset threshold. If the statistical result is greater than the preset threshold, it means that there are defects in the production process of the corresponding production batch, and defect optimization is required, and the corresponding production batch is marked as a defective production batch;

[0031] Obtain the three-dimensional size detection information of the defective products corresponding to the defective production batch, extract defect features, and use the K-means clustering algorithm to classify the defects based on the extracted defect features to obtain defect classification information;

[0032] Extracting the number of defective products corresponding to each defect category according to the defect category classification information, sorting them to generate a sorting table, screening the production defects that need to be optimized based on the sorting table, and obtaining production defect optimization judgment result information;

[0033] The production process is traced based on the production defect optimization judgment result information, and the associated production process and production process parameters during production are analyzed based on the defect area location information of the target product to be tested to obtain the production process traceability information.

[0034] In this solution, the defect optimization model is constructed. If production defect optimization is required, the production process traceability information is input into the defect optimization model to generate an optimization plan to control the corresponding production process, specifically including:

[0035] Extract production records according to each production and preparation process subsequence to obtain production process features, search in a preset search space according to the production process features, and obtain historical production instances and corresponding three-dimensional dimension detection results;

[0036] Based on data retrieval, historical production instances of the target product to be tested and corresponding three-dimensional dimension detection results are obtained, and feature extraction is performed on the historical instances to obtain historical production process parameter features of each historical production instance;

[0037] Obtaining a production and preparation process of the product to be tested, performing redundancy removal and segmentation operations on the production and preparation process of the product to be tested, generating a production and preparation sub-process, and segmenting the acquired historical production process parameter characteristics, associating the historical production process parameter characteristics with the corresponding production sub-processes, and obtaining first analysis information;

[0038] Using the three-dimensional dimension detection results corresponding to each historical production instance, historical production defect information of each historical instance is obtained, and the historical production defect information is associated with the first analysis information to form an associated path of production sub-process-production process parameter characteristics-production defects, and a knowledge graph is constructed;

[0039] Representing the knowledge graph in a graph form, performing deep learning on the graph representation of the knowledge graph using a graph neural network, and building a defect optimization model;

[0040] Acquire production process traceability information, input the production process traceability information into the defect optimization model to generate a target node, and perform first-order neighborhood sampling according to the target node to obtain a neighbor node;

[0041] The attention mechanism is introduced to obtain the attention coefficient, and the attention coefficient corresponding to the neighbor node is aggregated with the feature vector of the target node to update the feature vector of the target node.

[0042] Based on the updated feature vector of the target node, the production process optimization parameters of the production process with production defects are obtained, and a production process optimization plan is generated to control the corresponding production sub-process.

[0043] A second aspect of the present invention provides a three-dimensional size detection and control system based on a structured light camera, the system comprising: a memory, a processor, the memory containing a three-dimensional size detection and control method program based on a structured light camera, the three-dimensional size detection and control method program based on a structured light camera being executed by the processor to implement the following steps:

[0044] The product to be tested is scanned using a structured light camera, and phase analysis is performed based on the scanning results to obtain 3D depth information and convert it into point cloud data to build an actual 3D model of the product to be tested.

[0045] Based on the preparation instructions of the product to be tested, a standard 3D model of the product to be tested is constructed using 3D modeling technology. Matching analysis is performed in combination with the actual 3D model of the product to be tested to determine the areas with production defects and conduct preparation quality assessment;

[0046] If the product to be tested is a substandard product, the production qualification rate of the remaining products in the same batch is analyzed to determine whether production defect optimization is needed, and the production process is traced to obtain production process traceability information;

[0047] Construct a defect optimization model. If production defect optimization is required, input the production process traceability information into the defect optimization model to generate an optimization plan to control the corresponding production process.

[0048] The present invention discloses a three-dimensional size detection control method and system based on a structured light camera, including: scanning a product to be tested based on a structured light camera, and constructing an actual three-dimensional model of the product to be tested; constructing a standard three-dimensional model of the product to be tested using three-dimensional modeling technology, combining the actual three-dimensional model of the product to be tested for matching analysis, judging the area with production defects, and conducting preparation quality assessment; if the product to be tested is a substandard product, analyzing the production qualification rate of the remaining products in the same batch, judging whether production defect optimization is required, and tracing the production process to obtain production process traceability information; constructing a defect optimization model, if production defect optimization is required, inputting the production process traceability information into the defect optimization model to generate an optimization plan to control the corresponding production process. Improve the accuracy and efficiency of size detection, and optimize the production process to improve the production quality of the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the drawings required for use in the embodiments or exemplary descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to the drawings without paying creative work.

[0050] Figure 1 A flow chart of a three-dimensional size detection and control method based on a structured light camera provided by an embodiment of the present invention;

[0051] Figure 2 A flow chart of a production optimization method based on a structured light camera provided by one embodiment of the present invention;

[0052] Figure 3 A block diagram of a three-dimensional size detection and control system based on a structured light camera provided in one embodiment of the present invention;

[0053] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0055] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0056] Figure 1 A flow chart of a three-dimensional size detection and control method based on a structured light camera provided by an embodiment of the present invention;

[0057] like Figure 1 As shown, the present invention provides a flow chart of a three-dimensional size detection and control method based on a structured light camera, including:

[0058] S102, scanning the product to be tested based on the structured light camera, performing phase resolution according to the scanning result to obtain three-dimensional depth information and converting it into point cloud data, and constructing an actual three-dimensional model of the product to be tested;

[0059] S104, constructing a standard three-dimensional model of the product to be tested using three-dimensional modeling technology based on the preparation instructions of the product to be tested, performing matching analysis in combination with the actual three-dimensional model of the product to be tested, determining areas with production defects, and performing preparation quality assessment;

[0060] S106, if the product to be tested is a substandard product, the production qualification rate of the remaining products in the same batch is analyzed to determine whether production defect optimization is required, and the production process is traced to obtain production process traceability information;

[0061] S108, constructing a defect optimization model. If production defect optimization is required, inputting the production process traceability information into the defect optimization model to generate an optimization plan to control the corresponding production process.

[0062] It should be noted that the present invention discloses a three-dimensional size detection control method and system based on a structured light camera, which scans the product to be tested based on a structured light camera, performs phase solution according to the scanning results to obtain three-dimensional depth information and converts it into point cloud data, constructs an actual three-dimensional model of the product to be tested, and understands the three-dimensional size of the target product to be tested after production. Based on the preparation instructions of the product to be tested, a standard three-dimensional model of the product to be tested is constructed using three-dimensional modeling technology, and matching analysis is performed in combination with the actual three-dimensional model of the product to be tested to determine the area where production defects exist, and to perform preparation quality assessment, thereby determining whether the target product to be tested is a qualified product. If the product to be tested is an unqualified product, the production qualification rate of the remaining products in the same batch is analyzed to determine whether production defect optimization is required, and the production process is traced to obtain production process traceability information, and the defective area of ​​the product is analyzed to trace the production process responsible for generating the corresponding area, and understand the defective process. Construct a defect optimization model, and if production defect optimization is required, input the production process traceability information into the defect optimization model to generate an optimization plan to control the corresponding production process. This will improve the accuracy of three-dimensional dimension detection, significantly enhance production efficiency and product quality, and reduce production costs and resource waste.

[0063] Furthermore, in a preferred embodiment of the present invention, the structured light camera is used to scan the product to be tested, and phase resolution is performed according to the scanning result to obtain three-dimensional depth information and convert it into point cloud data to construct an actual three-dimensional model of the product to be tested, which specifically includes:

[0064] Scan the product to be tested in the detection area based on the structured light camera according to the preset detection scheme, and obtain the surface reflected light pattern of the product to be tested in each preset scanning area;

[0065] Preprocessing the acquired surface reflected light patterns of each preset scanning area, and extracting the light fringe phase characteristics of the surface reflected light patterns of each preset scanning area by using a Fourier transform method;

[0066] The phase difference of different preset scanning areas is calculated according to the extracted light stripe phase characteristics, and the phase solution is performed to obtain the three-dimensional depth information of the product to be tested, and the three-dimensional depth information is converted and processed to obtain the point cloud data of the product to be tested;

[0067] The point cloud data of the product to be tested is registered, and the registered point cloud data is synthesized to generate a point cloud data model of the product to be tested. The point cloud data model is converted into a continuous surface model based on the triangulation method. The surface model is rendered and visualized to generate an actual three-dimensional model of the product to be tested.

[0068] It should be noted that the three-dimensional detection method based on the structured light camera first scans the product to be tested in the detection area according to the preset detection scheme. During the scanning process, the structured light camera projects a light spot or stripe pattern onto the surface of the product and receives the reflected light signal through the camera. The surface features of the product to be tested will affect the reflection of light and produce specific reflected light patterns, which contain the geometric information of the product surface. By scanning in multiple preset scanning areas, the reflected light patterns of various parts of the product to be tested can be collected. Next, the surface reflected light patterns of each preset scanning area are preprocessed to ensure the accuracy and integrity of the data. Using Fourier transform technology, it can effectively extract the phase characteristics of the light stripes in the reflected light pattern. Fourier transform can convert complex light signals from the time domain to the frequency domain, making the light stripes that were originally difficult to analyze directly clearer, thereby extracting the phase information therein. After obtaining the phase characteristics of the light stripes in each scanning area, the three-dimensional depth information of the surface of the product to be tested is further derived by calculating the phase difference. The phase difference between different scanning areas represents the change of surface morphology. Through the phase solution method, these differences can be converted into accurate depth data to obtain the three-dimensional shape information of the product to be tested. After the depth information is converted, the three-dimensional point cloud data of the product surface can be generated. The point cloud data contains the spatial position of each sampling point on the surface of the object and is the basic data form for describing the three-dimensional shape. After obtaining the point cloud data, the next step is to align it. Point cloud registration refers to aligning the point cloud data obtained from different scanning areas to ensure that these data can accurately reflect the three-dimensional structure of the entire product to be tested. The registered point cloud data can more comprehensively reflect the true shape of the product to be tested. Finally, by triangulating the registered point cloud data, the discrete point cloud can be converted into a continuous surface model. The triangulation method connects the data points in the point cloud through triangular patches to form a smooth and continuous surface, thereby more accurately representing the appearance of the product to be tested. The triangulated surface model will be rendered and visualized to generate the actual three-dimensional model of the product to be tested.

[0069] Further, in a preferred embodiment of the present invention, the standard three-dimensional model of the product to be tested is constructed by using three-dimensional modeling technology based on the preparation instructions of the product to be tested, and matching analysis is performed in combination with the actual three-dimensional model of the product to be tested to determine the area with production defects and perform preparation quality assessment, which specifically includes:

[0070] Obtaining a preparation instruction of the product to be tested, constructing a standard three-dimensional model of the product to be tested by using three-dimensional modeling technology based on the preparation instruction of the product to be tested, and performing geometric element decomposition according to the constructed standard three-dimensional model;

[0071] Obtaining the decomposition results of geometric elements of the standard three-dimensional model and extracting key points based on the obtained geometric elements, calculating the feature descriptors of each key point, wherein the feature descriptors are used to describe the local geometric properties of each key point, and generating a feature map of the standard three-dimensional model;

[0072] Acquire an actual three-dimensional model of the product to be tested, decompose geometric elements according to the actual three-dimensional model of the product to be tested and construct a feature map of the actual three-dimensional model, and align the feature map of the actual three-dimensional model with the feature map of the standard three-dimensional model;

[0073] A hash algorithm is introduced to construct a hash table based on the actual three-dimensional model feature map and the standard three-dimensional model feature map to obtain a first hash table and a second hash table, and the first hash table and the second hash table are matched and analyzed;

[0074] Calculate the hash distance between the feature descriptor of each key point in the actual 3D model feature map and the feature descriptor of each key point in the standard 3D model feature map according to the matching analysis result;

[0075] The calculated hash distance is compared with the preset threshold, and the key points less than the preset threshold are defined as matching point pairs according to the judgment result. The feature descriptors corresponding to the matching point pairs obtained based on the analysis are marked in the actual 3D model to generate the marked actual 3D model.

[0076] It should be noted that, first, the preparation instructions of the product to be tested are obtained. The preparation instructions usually contain information such as the design requirements, geometric dimensions, and material properties of the product. Based on this information, a standard three-dimensional model of the product to be tested is constructed through three-dimensional modeling technology. The standard three-dimensional model reflects the ideal design form of the product and has all the geometric features of the product. After the standard three-dimensional model is constructed, the model is further decomposed into geometric elements, and its complex shape is decomposed into a series of simple geometric elements (such as faces, edges, angles, etc.). Next, based on the results of the geometric element decomposition of the standard three-dimensional model, the key points on each geometric element are extracted. These key points usually represent significant positions of morphological changes in the three-dimensional model. By calculating the feature descriptors of these key points, the local geometric properties of each key point can be accurately described, such as morphological features such as angles and curvatures. Feature descriptors are tools for distinguishing different geometric elements and their positional relationships. They can help distinguish and match different three-dimensional forms. Subsequently, a similar geometric element decomposition is performed on the actual three-dimensional model to obtain its geometric elements and key points. These key points will also calculate feature descriptors based on local geometric features and generate feature maps of the actual three-dimensional model. Next, the feature map of the standard 3D model and the feature map of the actual 3D model are aligned, and a hash algorithm is introduced to process and optimize feature matching. The hash algorithm can compress the feature point descriptor into a unique identifier by mapping each feature descriptor into a hash table, which is convenient for fast search and matching. In this way, the feature descriptor of the standard 3D model and the feature descriptor of the actual 3D model are respectively constructed into two hash tables, namely the first hash table and the second hash table. By matching and analyzing these two hash tables, key points with similar features in the two can be quickly identified. By calculating the hash distance, similar or corresponding key points in the standard 3D model and the actual 3D model are identified. Finally, based on the matching point pairs and their corresponding feature descriptors, they are marked in the actual 3D model to generate a marked actual 3D model. In this way, it is intuitive to see which parts of the actual 3D model match the standard 3D model, so as to judge the deviation between the geometric shape of the product to be tested and the design specification.

[0077] Furthermore, in a preferred embodiment of the present invention, the standard three-dimensional model of the product to be tested is constructed by using three-dimensional modeling technology based on the preparation instructions of the product to be tested, and matching analysis is performed in combination with the actual three-dimensional model of the product to be tested to determine the area with production defects and perform preparation quality assessment, which also includes:

[0078] Preset the area division rule and divide the marked actual 3D model into several areas, extract the number of matching point pairs in each sub-area, and determine whether the corresponding sub-area is an abnormal area according to the number of matching point pairs in each sub-area to obtain abnormal area analysis information;

[0079] Based on the abnormal area analysis information extraction, the three-dimensional size features of the abnormal area are extracted using the actual three-dimensional model of the product to be tested to obtain the three-dimensional size feature information of the abnormal area;

[0080] Acquire preparation standard data of the product to be tested, calculate the three-dimensional size feature information of the abnormal area and the preparation standard data of the product to be tested, obtain the deviation value between the three-dimensional size of the abnormal area and the preparation standard, and obtain deviation value information;

[0081] Acquire preparation requirement information of the product to be tested, formulate quality assessment rules according to the preparation requirement information of the product to be tested, judge the deviation value information with the quality assessment rules, analyze the preparation quality of the target product to be tested, and obtain preparation quality analysis information;

[0082] Based on the preparation quality analysis information, it is determined whether the target product to be tested is a qualified product. If it is not a qualified product, the target product to be tested is marked as an unqualified product.

[0083] It should be noted that, first, the preset area division rule divides the actual three-dimensional model of the product to be tested into several sub-areas, so that each area can be analyzed more accurately. The number of matching point pairs in each sub-area is counted in the marked actual three-dimensional model. If the number of matching point pairs in a sub-area is far below the preset threshold, it may mean that there is a large difference between the morphology of the area and the standard model, so it is marked as an abnormal area to accurately identify the potential defect area and provide a basis for subsequent quality assessment. Based on the analysis information of the abnormal area, the three-dimensional size features of the abnormal area are further extracted to obtain detailed three-dimensional size feature information. Next, the preparation standard data of the product to be tested is compared and calculated with the three-dimensional size feature information of the abnormal area. The preparation standard data usually includes detailed information such as the design size, shape tolerance, and material properties of the product. By comparing with the actual measured three-dimensional size features of the abnormal area, the size deviation value of each abnormal area is calculated, reflecting the gap between the actual production and the standard design. The deviation value information can be used to quantitatively evaluate the degree of deviation in the production process and help discover problems in the manufacturing process or equipment. Then, the preparation requirement information of the product to be tested is obtained, which is usually related to the function, quality standard, and usage scenario of the product. Based on these requirements, the system can formulate quality assessment rules to determine whether the quality of the product to be tested meets the current production requirements. Quality assessment rules usually combine indicators such as the product's dimensional tolerance and performance requirements to obtain the preparation quality analysis information of the product to be tested. Finally, based on the preparation quality analysis information, it is determined whether the target product to be tested is a qualified product. If the analysis finds that the deviation value exceeds the preset tolerance range, the product will be judged as a non-conforming product. At this time, the target product to be tested is marked as a non-conforming product.

[0084] Furthermore, in a preferred embodiment of the present invention, if the product to be tested is an unqualified product, the production qualification rate of the remaining products in the same batch is analyzed to determine whether production defect optimization is required, and the production process is traced to obtain the production process traceability information, which specifically includes:

[0085] If the target product to be tested is a product that fails to meet the requirements of production, the three-dimensional dimension test results of other products in the same production batch of the target product to be tested are obtained, and a production quality analysis is performed, and the number of failed products in the same production batch is counted according to the analysis results;

[0086] The statistical result is compared with the preset threshold. If the statistical result is greater than the preset threshold, it means that there are defects in the production process of the corresponding production batch, and defect optimization is required, and the corresponding production batch is marked as a defective production batch;

[0087] Obtain the three-dimensional size detection information of the defective products corresponding to the defective production batch, extract defect features, and use the K-means clustering algorithm to classify the defects based on the extracted defect features to obtain defect classification information;

[0088] Extracting the number of defective products corresponding to each defect category according to the defect category classification information, sorting them to generate a sorting table, screening the production defects that need to be optimized based on the sorting table, and obtaining production defect optimization judgment result information;

[0089] The production process is traced based on the production defect optimization judgment result information, and the associated production process and production process parameters during production are analyzed based on the defect area location information of the target product to be tested to obtain the production process traceability information.

[0090] It should be noted that when the target product to be tested is judged as a non-conforming product, the three-dimensional dimension detection results of the remaining products in the same production batch are first obtained. By analyzing the detection data of these products, the quality status of other products in the batch is understood, and the preparation quality analysis is performed. By counting the number of non-conforming products in the same production batch, the overall quality level of the production batch is evaluated, and possible problems in the production process are revealed. The statistical results are compared and analyzed with the preset threshold. If the statistical results show that the number of non-conforming products is greater than the preset threshold, it means that there may be defects in the production process of the batch. Therefore, the production batch is marked as a defective production batch. Next, the three-dimensional dimension detection information of the defective product corresponding to the defective production batch is obtained, and the features of the defective product are extracted. These features usually include dimensional deviation, geometric abnormality, position error, etc., which help to reveal the specific type and cause of the defect. Based on the extracted defect features, the K-means clustering algorithm is used to divide the defect categories. The K-means algorithm groups the products with similar defect features into the same category by grouping the similarity of the defect features, thereby providing a basis for subsequent defect analysis and optimization. After the defect categories are divided, they will be sorted according to the number of defective products corresponding to each defect category to generate a sorting table. The sorting table can help analyze which defect types appear more frequently in production, screen out production defects that need to be optimized first based on this sorting table, and generate production defect optimization judgment result information. Finally, the production process is traced based on the production defect optimization judgment result information. According to the defect area location information of the target product to be tested, the production process and process parameters that may be related in the production process are analyzed to provide a basis for adjusting and optimizing the production process.

[0091] Further, in a preferred embodiment of the present invention, the defect optimization model is constructed. If production defect optimization is required, the production process traceability information is input into the defect optimization model to generate an optimization plan to control the corresponding production process, specifically including:

[0092] Extract production records according to each production and preparation process subsequence to obtain production process features, search in a preset search space according to the production process features, and obtain historical production instances and corresponding three-dimensional dimension detection results;

[0093] Based on data retrieval, historical production instances of the target product to be tested and corresponding three-dimensional dimension detection results are obtained, and feature extraction is performed on the historical instances to obtain historical production process parameter features of each historical production instance;

[0094] Obtaining a production and preparation process of the product to be tested, performing redundancy removal and segmentation operations on the production and preparation process of the product to be tested, generating a production and preparation sub-process, and segmenting the acquired historical production process parameter characteristics, associating the historical production process parameter characteristics with the corresponding production sub-processes, and obtaining first analysis information;

[0095] Using the three-dimensional dimension detection results corresponding to each historical production instance, historical production defect information of each historical instance is obtained, and the historical production defect information is associated with the first analysis information to form an associated path of production sub-process-production process parameter characteristics-production defects, and a knowledge graph is constructed;

[0096] Representing the knowledge graph in a graph form, performing deep learning on the graph representation of the knowledge graph using a graph neural network, and building a defect optimization model;

[0097] Acquire production process traceability information, input the production process traceability information into the defect optimization model to generate a target node, and perform first-order neighborhood sampling according to the target node to obtain a neighbor node;

[0098] The attention mechanism is introduced to obtain the attention coefficient, and the attention coefficient corresponding to the neighbor node is aggregated with the feature vector of the target node to update the feature vector of the target node.

[0099] Based on the updated feature vector of the target node, the production process optimization parameters of the production process with production defects are obtained, and a production process optimization plan is generated to control the corresponding production sub-process.

[0100] It should be noted that, first, production records are extracted and production process features are obtained according to each production preparation process subsequence. These features represent key information such as process parameters, steps, time, etc. used in the production process. By searching in the preset search space, the system can find historical production instances similar to the target product to be tested and their corresponding three-dimensional dimension detection results. Next, based on the retrieved historical production instances and their three-dimensional dimension detection results, feature extraction is performed on these instances to obtain the historical production process parameter features of each historical production instance. These features contain information such as temperature, humidity, pressure, and production equipment used in the production process of each historical instance. Then, the production preparation process of the product to be tested is obtained, and redundancy and segmentation operations are performed on it. The production preparation process will be disassembled into multiple sub-processes, each of which represents an independent step in the production process. Next, the historical production process parameter features are segmented and associated with the corresponding production sub-processes to reveal the process parameter relationship between different production steps. The three-dimensional dimension detection results corresponding to each historical production instance are further used to obtain the production defect information of each historical instance. These defect information are associated with the first analysis information to form the association path between production sub-processes, production process parameter characteristics, and production defects, thereby constructing a knowledge graph. Then, the constructed knowledge graph is deeply learned through the graph neural network to discover the potential rules therein. The graph neural network can learn the relationship between each node by performing feature aggregation on the nodes and edges of the graph structure, and build a defect optimization model based on this information. By inputting the production process traceability information, the production process with defects in the production process is identified, and the production process optimization parameters are provided to generate the production process optimization plan, and the production sub-processes are managed and controlled, thereby achieving continuous optimization of the production process and quality improvement.

[0101] Figure 2 A flow chart of a production optimization method based on a structured light camera provided by one embodiment of the present invention;

[0102] like Figure 2 As shown, the present invention provides a flow chart of a production optimization method based on a structured light camera, comprising:

[0103] S202, using a structured light camera to perform three-dimensional size detection on the product to be tested, and analyzing whether there are defects based on the three-dimensional size detection result;

[0104] S204, counting the types and degrees of defects in the current production batch, and determining whether production process optimization is required;

[0105] S206, if optimization is required, the defect type that needs to be optimized is determined according to the type and degree of defects in the current production batch, and the production process is traced to obtain production process traceability information;

[0106] S208, based on the production process traceability information, use the defect optimization model to optimize and analyze the production sub-processes with production defects, and generate a production process optimization plan for management and control.

[0107] It should be noted that during the production process, the three-dimensional size of the products is detected by the structured light camera installed in the production line or quality inspection area, so that it can be understood whether there are three-dimensional production defects in the current production batch. At the same time, due to the fast detection rate and high accuracy of the structured light camera, the type and degree of defects in the current production batch can be understood in real time. When a certain defect type reaches the threshold, it means that production optimization is needed. At this time, through the location information of the area where the defect occurs, the production sub-process responsible for the production of the corresponding area can be further traced, so as to optimize the production process parameters of this production sub-process, and realize adjustments and improvements in the production process to improve production quality and efficiency.

[0108] Figure 3 A three-dimensional size detection and control system 3 based on a structured light camera is provided in one embodiment of the present invention. The system includes: a memory 31 and a processor 32. The memory 31 contains a three-dimensional size detection and control method program based on a structured light camera. When the three-dimensional size detection and control method program based on a structured light camera is executed by the processor 32, the following steps are implemented:

[0109] The product to be tested is scanned using a structured light camera, and phase analysis is performed based on the scanning results to obtain 3D depth information and convert it into point cloud data to build an actual 3D model of the product to be tested.

[0110] Based on the preparation instructions of the product to be tested, a standard 3D model of the product to be tested is constructed using 3D modeling technology. Matching analysis is performed in combination with the actual 3D model of the product to be tested to determine the areas with production defects and conduct preparation quality assessment;

[0111] If the product to be tested is a substandard product, the production qualification rate of the remaining products in the same batch is analyzed to determine whether production defect optimization is needed, and the production process is traced to obtain production process traceability information;

[0112] Construct a defect optimization model. If production defect optimization is required, input the production process traceability information into the defect optimization model to generate an optimization plan to control the corresponding production process.

[0113] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0114] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0115] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0116] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0117] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0118] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A three-dimensional size detection and control method based on a structured light camera, characterized in that: include: The product to be tested is scanned using a structured light camera, and phase analysis is performed based on the scanning results to obtain 3D depth information and convert it into point cloud data to build an actual 3D model of the product to be tested. Based on the preparation instructions of the product to be tested, a standard 3D model of the product to be tested is constructed using 3D modeling technology. Matching analysis is performed in combination with the actual 3D model of the product to be tested to determine the areas with production defects and conduct preparation quality assessment; If the product to be tested is a substandard product, the production qualification rate of the remaining products in the same batch is analyzed to determine whether production defect optimization is needed, and the production process is traced to obtain production process traceability information; Construct a defect optimization model. If production defect optimization is required, input the production process traceability information into the defect optimization model to generate an optimization plan to control the corresponding production process.

2. A three-dimensional size detection and control method based on a structured light camera according to claim 1, characterized in that: The structured light camera is used to scan the product to be tested, and phase resolution is performed according to the scanning result to obtain three-dimensional depth information and convert it into point cloud data to construct an actual three-dimensional model of the product to be tested, which specifically includes: Scan the product to be tested in the detection area based on the structured light camera according to the preset detection scheme, and obtain the surface reflected light pattern of the product to be tested in each preset scanning area; Preprocessing the acquired surface reflected light patterns of each preset scanning area, and extracting the light fringe phase characteristics of the surface reflected light patterns of each preset scanning area by using a Fourier transform method; The phase difference of different preset scanning areas is calculated according to the extracted light stripe phase characteristics, and the phase solution is performed to obtain the three-dimensional depth information of the product to be tested, and the three-dimensional depth information is converted and processed to obtain the point cloud data of the product to be tested; The point cloud data of the product to be tested is registered, and the registered point cloud data is synthesized to generate a point cloud data model of the product to be tested. The point cloud data model is converted into a continuous surface model based on the triangulation method. The surface model is rendered and visualized to generate an actual three-dimensional model of the product to be tested.

3. The three-dimensional size detection and control method based on a structured light camera according to claim 1, characterized in that: The method of constructing a standard three-dimensional model of the product to be tested based on the preparation instructions of the product to be tested by using three-dimensional modeling technology, performing matching analysis in combination with the actual three-dimensional model of the product to be tested, determining areas with production defects, and performing preparation quality assessment specifically includes: Obtaining a preparation instruction of the product to be tested, constructing a standard three-dimensional model of the product to be tested by using three-dimensional modeling technology based on the preparation instruction of the product to be tested, and performing geometric element decomposition according to the constructed standard three-dimensional model; Obtaining the decomposition results of geometric elements of the standard three-dimensional model and extracting key points based on the obtained geometric elements, calculating the feature descriptors of each key point, wherein the feature descriptors are used to describe the local geometric properties of each key point, and generating a feature map of the standard three-dimensional model; Acquire an actual three-dimensional model of the product to be tested, decompose geometric elements according to the actual three-dimensional model of the product to be tested and construct a feature map of the actual three-dimensional model, and align the feature map of the actual three-dimensional model with the feature map of the standard three-dimensional model; A hash algorithm is introduced to construct a hash table based on the actual three-dimensional model feature map and the standard three-dimensional model feature map to obtain a first hash table and a second hash table, and the first hash table and the second hash table are matched and analyzed; Calculate the hash distance between the feature descriptor of each key point in the actual 3D model feature map and the feature descriptor of each key point in the standard 3D model feature map according to the matching analysis result; The calculated hash distance is compared with the preset threshold, and the key points less than the preset threshold are defined as matching point pairs according to the judgment result. The feature descriptors corresponding to the matching point pairs obtained based on the analysis are marked in the actual 3D model to generate the marked actual 3D model.

4. The three-dimensional size detection and control method based on a structured light camera according to claim 1, characterized in that: The method of constructing a standard three-dimensional model of the product to be tested by using three-dimensional modeling technology based on the preparation instructions of the product to be tested, performing matching analysis in combination with the actual three-dimensional model of the product to be tested, determining areas with production defects, and performing preparation quality assessment also includes: Preset the area division rule and divide the marked actual 3D model into several areas, extract the number of matching point pairs in each sub-area, and determine whether the corresponding sub-area is an abnormal area according to the number of matching point pairs in each sub-area to obtain abnormal area analysis information; Based on the abnormal area analysis information extraction, the three-dimensional size features of the abnormal area are extracted using the actual three-dimensional model of the product to be tested to obtain the three-dimensional size feature information of the abnormal area; Acquire preparation standard data of the product to be tested, calculate the three-dimensional size feature information of the abnormal area and the preparation standard data of the product to be tested, obtain the deviation value between the three-dimensional size of the abnormal area and the preparation standard, and obtain deviation value information; Acquire preparation requirement information of the product to be tested, formulate quality assessment rules according to the preparation requirement information of the product to be tested, judge the deviation value information with the quality assessment rules, analyze the preparation quality of the target product to be tested, and obtain preparation quality analysis information; Based on the preparation quality analysis information, it is determined whether the target product to be tested is a qualified product. If it is not a qualified product, the target product to be tested is marked as an unqualified product.

5. The three-dimensional size detection and control method based on a structured light camera according to claim 1, characterized in that: If the product to be tested is a substandard product, the production qualification rate of the remaining products in the same batch is analyzed to determine whether production defect optimization is required, and the production process is traced to obtain the production process traceability information, which specifically includes: If the target product to be tested is a product that fails to meet the requirements of production, the three-dimensional dimension test results of other products in the same production batch of the target product to be tested are obtained, and a production quality analysis is performed, and the number of failed products in the same production batch is counted according to the analysis results; The statistical result is compared with the preset threshold. If the statistical result is greater than the preset threshold, it means that there are defects in the production process of the corresponding production batch, and defect optimization is required, and the corresponding production batch is marked as a defective production batch; Obtain the three-dimensional size detection information of the defective products corresponding to the defective production batch, extract defect features, and use the K-means clustering algorithm to classify the defects based on the extracted defect features to obtain defect classification information; Extracting the number of defective products corresponding to each defect category according to the defect category classification information, sorting them to generate a sorting table, screening the production defects that need to be optimized based on the sorting table, and obtaining production defect optimization judgment result information; The production process is traced based on the production defect optimization judgment result information, and the associated production process and production process parameters during production are analyzed based on the defect area location information of the target product to be tested to obtain the production process traceability information.

6. The three-dimensional size detection and control method based on a structured light camera according to claim 1, characterized in that: The defect optimization model is constructed. If production defect optimization is required, the production process traceability information is input into the defect optimization model to generate an optimization plan to control the corresponding production process, specifically including: Extract production records according to each production and preparation process subsequence to obtain production process features, search in a preset search space according to the production process features, and obtain historical production instances and corresponding three-dimensional dimension detection results; Based on data retrieval, historical production instances of the target product to be tested and corresponding three-dimensional dimension detection results are obtained, and feature extraction is performed on the historical instances to obtain historical production process parameter features of each historical production instance; Obtaining a production and preparation process of the product to be tested, performing redundancy removal and segmentation operations on the production and preparation process of the product to be tested, generating a production and preparation sub-process, and segmenting the acquired historical production process parameter characteristics, associating the historical production process parameter characteristics with the corresponding production sub-processes, and obtaining first analysis information; Using the three-dimensional dimension detection results corresponding to each historical production instance, historical production defect information of each historical instance is obtained, and the historical production defect information is associated with the first analysis information to form an associated path of production sub-process-production process parameter characteristics-production defects, and a knowledge graph is constructed; Representing the knowledge graph in a graph form, performing deep learning on the graph representation of the knowledge graph using a graph neural network, and building a defect optimization model; Acquire production process traceability information, input the production process traceability information into the defect optimization model to generate a target node, and perform first-order neighborhood sampling according to the target node to obtain a neighbor node; The attention mechanism is introduced to obtain the attention coefficient, and the attention coefficient corresponding to the neighbor node is aggregated with the feature vector of the target node to update the feature vector of the target node. Based on the updated feature vector of the target node, the production process optimization parameters of the production process with production defects are obtained, and a production process optimization plan is generated to control the corresponding production sub-process.

7. A three-dimensional size detection and control system based on a structured light camera, characterized in that: The system includes: a memory and a processor, wherein the memory contains a three-dimensional size detection and control method program based on a structured light camera, and when the three-dimensional size detection and control method program based on a structured light camera is executed by the processor, the following steps are implemented: The product to be tested is scanned using a structured light camera, and phase analysis is performed based on the scanning results to obtain 3D depth information and convert it into point cloud data to build an actual 3D model of the product to be tested. Based on the preparation instructions of the product to be tested, a standard 3D model of the product to be tested is constructed using 3D modeling technology. Matching analysis is performed in combination with the actual 3D model of the product to be tested to determine the areas with production defects and conduct preparation quality assessment; If the product to be tested is a substandard product, the production qualification rate of the remaining products in the same batch is analyzed to determine whether production defect optimization is needed, and the production process is traced to obtain production process traceability information; Construct a defect optimization model. If production defect optimization is required, input the production process traceability information into the defect optimization model to generate an optimization plan to control the corresponding production process.

Images