An optical scene reconstruction system combining structured light and 3D vision

Abstract

The present invention discloses an optical scene reconstruction system combining structured light and three-dimensional vision, which relates to the technical field of optical scene reconstruction, including an initial spot projection module, a region division module, a spot data acquisition module, a spot morphological feature extraction module, a feature vector conversion and evaluation module, a spot density adjustment module, and a spot re-projection module. The present invention adopts a strategy of dynamically optimizing the spot projection density, and adaptively adjusts the spot distribution of different sub-regions through deep learning model evaluation and cluster analysis, so that the spot density is increased on curved surfaces, concave-convex areas or places with complex geometric features, thereby capturing more details; while in flat or regular areas, the spot density is appropriately reduced to reduce calculation and resource waste. This optimization makes the spot distribution of the entire surface more refined and balanced, can cover all key feature areas, avoid data loss and error amplification, and ensure that the reconstruction result is more complete and accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of optical scene reconstruction, and in particular to an optical scene reconstruction system combining structured light and three-dimensional vision. Background Art

[0002] The optical scene reconstruction system combining structured light and 3D vision is a system that uses structured light technology and 3D vision technology to achieve high-precision, real-time 3D scene reconstruction. Structured light technology projects a known light pattern onto the surface of an object and uses a camera to capture the deformation of the light spot to obtain the geometric shape of the object surface. 3D vision technology further reconstructs the 3D model of the object or scene through image processing and depth perception from multiple perspectives. The combination of the two can provide more accurate depth information and stronger anti-interference capabilities, especially in complex lighting conditions or dynamic environments, which can effectively improve the accuracy and reliability of scene reconstruction. This system is widely used in robot navigation, autonomous driving, virtual reality, medical imaging and other fields.

[0003] Structured light technology is a 3D shape reconstruction method based on light pattern projection. It projects known light patterns (such as stripes, dots, grids, etc.) onto the surface of an object, and uses a camera to capture the deformation of the pattern to infer the 3D geometric information of the object. The spot array light pattern is a special form of structured light, in which the light pattern consists of a series of evenly distributed small light spots (dots). Unlike stripe light or checkerboard patterns, the spot array generates a pattern through a point light source to describe the geometric shape of the object surface. When the spot array is projected onto the surface of an object by a projector or laser, each surface point receives a light spot. Different surface shapes of objects affect the shape and position of the light spot: on a flat surface, the light spot remains regularly distributed, while on a curved or irregular surface, the light spot will shift or deform, and the degree of deformation is closely related to the geometric shape and characteristics of the object surface.

[0004] The prior art has the following deficiencies:

[0005] Existing technologies usually use a uniform spot projection density to simplify system design and reduce computational complexity. However, when the spot deformation in certain areas is very large, it is often because the surface details or geometric features of these areas are relatively complex, resulting in the spot being too sparse. In this case, continuing to use a uniform spot projection density may result in the inability to effectively capture subtle changes in the surface, especially in areas with curved surfaces or complex structures. Due to the lack or inaccuracy of depth information, the three-dimensional reconstruction results may have obvious errors or even completely lose the accurate description of these areas. In addition, the sparse spot cannot cover all the details and key areas on the surface of the object, resulting in the loss of depth data in these areas and the inability to reconstruct the complete shape of the object. This data loss will not only reduce the quality of reconstruction, but may also affect subsequent measurements and analysis, and thus affect the overall reliability and application effect of the system.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0007] The purpose of the present invention is to provide an optical scene reconstruction system that combines structured light with three-dimensional vision. The system adopts a strategy of dynamically optimizing the light spot projection density, and adaptively adjusts the light spot distribution in different sub-areas through deep learning model evaluation and cluster analysis, so that the light spot density is increased on curved surfaces, concave-convex areas or places with complex geometric features, thereby capturing more details; while in flat or regular areas, the light spot density is appropriately reduced to reduce calculation and resource waste. This optimization makes the light spot distribution of the entire surface more refined and balanced, able to cover all key feature areas, avoid data loss and error amplification, and ensure that the reconstruction results are more complete and accurate, so as to solve the problems in the above-mentioned background technology.

[0008] In order to achieve the above-mentioned object, the present invention provides the following technical solutions: an optical scene reconstruction system combining structured light and three-dimensional vision, comprising an initial spot projection module, a region division module, a spot data acquisition module, a spot morphology feature extraction module, a feature vector conversion and evaluation module, a spot density adjustment module and a spot re-projection module;

[0009] The initial spot projection module adopts the initial spot projection density and uses the laser to project the spot pattern onto the surface of the object to form a preliminary spot projection area;

[0010] The area division module divides the preliminary light spot projection area into several sub-areas of equal area;

[0011] A light spot data acquisition module, in each divided sub-area, uses a sensor to collect light spot data in the sub-area;

[0012] The light spot morphology feature extraction module extracts the features related to the light spot morphology change from the collected light spot data for each sub-area, analyzes the extracted light spot morphology change features, and quantifies the light spot morphology change;

[0013] The feature vector conversion and evaluation module converts the quantified spot morphology change features after analysis into feature vectors, and inputs the converted feature vectors into the pre-trained deep learning model to conduct a comprehensive evaluation of the spot morphology changes in each sub-area;

[0014] The spot density adjustment module uses a cluster analysis algorithm to adjust the spot density of each sub-area based on the evaluation results of the deep learning model, and adaptively allocates different spot projection densities to different surface features;

[0015] The light spot re-projection module re-projects the light spot onto the surface of the object according to the adjusted light spot projection density, so that the light spot distribution in each sub-area is more refined, and finally the detailed geometric information of the object surface is obtained through the projected light spot.

[0016] Preferably, the initial spot projection density setting is preset according to the overall size, surface features and task requirements of the object, and the specific steps are as follows:

[0017] First, the surface area of ​​the object is estimated according to the size of the object, and the preliminary spot projection density is calculated according to the surface area and the required scanning accuracy;

[0018] Next, analyze the geometric features of the object surface, including flat areas, curved areas, and areas with complex structures, and consider the spot density requirements of different areas. For flat areas, set a low spot density; for curved areas or areas with complex structures, increase the spot density to ensure the capture of details;

[0019] Finally, according to the actual requirements of the task, a balanced initial light spot projection density is determined to ensure that enough light spots can be covered on the surface of the object without causing excessive computational burden or data redundancy.

[0020] Preferably, the size and shape of each sub-region are adjusted according to the surface characteristics of the object to ensure that the area of ​​each sub-region is relatively uniform, which facilitates subsequent analysis and calculation.

[0021] Preferably, for each sub-area, features related to the spot morphology change are extracted from the collected spot data, wherein the extracted spot morphology change features include the geometric complexity of the spot morphology and the degree of change of the spot edge angle. After analyzing the extracted geometric complexity features of the spot morphology and the degree of change of the spot edge angle, a spot geometric complexity reference value and a spot angle distortion reference value are generated respectively, and the spot morphology change is quantified by the spot geometric complexity reference value and the spot angle distortion reference value.

[0022] Preferably, the specific steps of generating a light spot geometric complexity reference value after analyzing the extracted geometric complexity characteristics of the light spot morphology are as follows:

[0023] The edge contour of the light spot is extracted by image processing method to obtain the projection shape of each light spot on the surface of the object. Then, the shape characteristics of each light spot are analyzed, the edge curvature and shape distortion of the light spot are calculated, and the curvature of the light spot is defined as , the calculation expression of the spot curvature is: ,in, and are the first and second order derivatives of the spot profile function on the x and y coordinates respectively;

[0024] The geometric distortion of the light spot is further quantified based on the edge profile and curvature of the light spot. In order to fully capture the changes in morphology, the geometric distortion index is introduced to evaluate the deviation between the light spot shape and the ideal circular or ideal plane light spot. The geometric distortion index is defined as: ,in, is the geometric distortion index, and are the areas of the actual light spot and the ideal light spot at positions x and y, respectively. and are the local curvatures of the actual light spot and the ideal light spot respectively, and is an exponential constant that controls distortion sensitivity;

[0025] The geometric distortion indexes obtained from multiple spots are combined to form a global spot geometric complexity reference value. The generation expression of the spot geometric complexity reference value is: ,in, is the reference value of the spot geometry complexity, is the geometric distortion index of the i-th light spot, N is the total number of light spots, is the area of ​​the ith light spot, is the total area of ​​the subregion, is an adjustment index used to control the weight of the geometric distortion index in the total complexity.

[0026] Preferably, for each sub-region, the specific steps of analyzing the extracted characteristics of the variation degree of the light spot edge angle and generating the light spot angle distortion reference value are as follows:

[0027] For the sub-region, first extract the edge point set of all light spots and calculate the local angle change of each edge point. The edge point set of , the local tangent direction of each edge point is , the local angle change of each edge point is calculated by the angle between adjacent tangent directions, and the calculation expression is: ,in: Represents edge points The tangent direction of Represents edge points The tangent direction of is calculated by the coordinate difference of adjacent edge points, such as , Represents edge points The local angle change between its neighboring points;

[0028] Light spot The angle changes of all edge points are weighted and aggregated to generate the overall edge angle change to reflect the overall morphological change of the light spot. The overall edge angle change is defined as: ,in: is the overall edge angle change of the jth light spot, n is the light spot The number of edge points, For edge points The curvature of Represents edge points and The distance between them is used to evaluate the contribution of edge points to the overall morphological change. It is a light spot The total edge length is used to normalize the influence of each edge point, It is a nonlinear adjustment parameter used to enhance the influence of edge points with large morphological changes;

[0029] All the spots in the sub-area The overall edge angle change is aggregated to generate the spot angle distortion reference value of the sub-region. In order to enhance the sensitivity to complex geometric areas, the relative geometric relationship between the spots is added as a weight factor. The calculation expression of the spot angle distortion reference value is: ,in: is the reference value of the spot angle distortion, m is the number of spots in the sub-area, It is a light spot The distance between the center point and the center of the sub-area reflects the distribution position of the light spot in the area. is the distance attenuation coefficient, which is used to reduce the impact of the light spot in the edge area on the overall result.

[0030] Preferably, the spot geometry complexity reference value and the spot angle distortion reference value quantified after analysis are converted into feature vectors, and the converted feature vectors are input into a pre-trained deep learning model, and a morphological variation coefficient is generated by the deep learning model, and the spot morphological changes in each sub-area are comprehensively evaluated by the morphological variation coefficient.

[0031] Preferably, based on the evaluation results of the deep learning model, a cluster analysis algorithm is used to adjust the spot density of each sub-area. The specific steps are as follows:

[0032] After the initial spot projection density is projected, the spot data features of each sub-area are extracted to form a feature set. The expression of the feature set is: ,in: represents the eigenvector of the qth light spot in the pth sub-region, is the initial light spot projection density, indicating the initial distribution of the light spot in the sub-area, is the morphological variation coefficient of the pth sub-region, reflecting the complex changes in the spot morphology due to surface characteristics. It is an auxiliary feature;

[0033] Using clustering algorithm, the sub-region feature vector Clustering is performed to classify spots with similar characteristics into the same category. The goal is to minimize the difference within the cluster. The clustering formula is as follows: ,in: The kth cluster contains sub-regions with similar characteristics. is the center point of the kth cluster, representing the average characteristics of the cluster, is the Euclidean distance between the feature vector and the cluster center;

[0034] The spot density of each sub-area is adjusted according to the morphological variation coefficient, and the optimized spot density is calculated. The calculation expression of the adjusted spot projection density is: ,in: is the adjusted light spot projection density, is the initial spot projection density, is the density adjustment factor, which controls the density adjustment range. and are the minimum and maximum values ​​of the morphological variation coefficient, respectively, which are used for normalization processing;

[0035] The spot projection system is reset according to the optimized spot density, and the adjusted spot projection density is projected onto the surface of the object. The distribution expression of the adjusted spot projection density is: ,in: is the optimized overall spot density distribution set, K is the number of clustering categories, represents the joint result of the optimized density of all sub-regions.

[0036] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0037] The present invention adopts a strategy of dynamically optimizing the light spot projection density, and adaptively adjusts the light spot distribution of different sub-areas through deep learning model evaluation and cluster analysis, so that the light spot density is increased on curved surfaces, concave-convex areas or places with complex geometric features, thereby capturing more details; while in flat or regular areas, the light spot density is appropriately reduced to reduce calculation and resource waste. This optimization makes the light spot distribution of the entire surface more refined and balanced, and can cover all key feature areas, avoid data loss and error amplification, and ensure that the reconstruction results are more complete and accurate. In addition, the design of the system takes into account both efficiency and flexibility. Through intelligent density allocation, it not only meets the needs of high-precision reconstruction, but also reduces the overall computational complexity of the system, improves the reconstruction efficiency and the reliability of subsequent analysis, and is suitable for high-precision scene applications such as industrial inspection, reverse engineering and complex geometric modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0039] Figure 1 The module diagram of an optical scene reconstruction system combining structured light and three-dimensional vision according to the present invention. DETAILED DESCRIPTION

[0040] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art.

[0041] The present invention provides Figure 1 An optical scene reconstruction system combining structured light and three-dimensional vision is shown, comprising an initial spot projection module, a region division module, a spot data acquisition module, a spot morphology feature extraction module, a feature vector conversion and evaluation module, a spot density adjustment module, and a spot re-projection module;

[0042] The initial spot projection module adopts the initial spot projection density and uses the laser to project the spot pattern onto the surface of the object to form a preliminary spot projection area;

[0043] The initial spot projection density setting is usually preset according to the overall size of the object, surface characteristics and task requirements. The specific steps are as follows:

[0044] First, estimate the surface area of ​​the object based on its size (such as length, width, height, etc.), and calculate the preliminary spot projection density based on the surface area and the required scanning accuracy. For example, larger objects or tasks that require higher precision will require a higher spot density. Next, analyze the geometric features of the object surface, including flat areas, curved areas, and areas with complex structures, and consider the spot density requirements in different areas. For flatter areas, a lower spot density can be set; for curved surfaces or areas with complex structures, the spot density needs to be increased to ensure the capture of details. Finally, based on the actual requirements of the task (such as measurement accuracy, processing speed, etc.), determine a balanced initial spot projection density to ensure that enough spots can be covered on the surface of the object without causing excessive computational burden or data redundancy. This initial setting provides the basis for subsequent dynamic adjustments.

[0045] The main purpose of this step is to create an initial spot pattern for the entire object surface. By setting the initial spot projection density, the entire object surface can be quickly covered and preliminary spot data can be obtained, providing a basis for subsequent detail processing and density adjustment. This stage mainly ensures the rapid coverage of the system without the need to refine the density distribution at the beginning.

[0046] The area division module divides the preliminary light spot projection area into several sub-areas of equal area;

[0047] The size and shape of each sub-region can be appropriately adjusted according to the surface characteristics of the object (such as flat or curved surface), but the basic principle is to ensure that the area of ​​each sub-region is relatively uniform to facilitate subsequent analysis and calculation.

[0048] The main purpose of this stage is to divide the entire object surface into multiple small blocks, thereby decomposing the complex surface into local areas that are easy to process. Equal area division helps to balance computing resources and processing accuracy, ensuring that each area can be processed independently while avoiding overly dense spots or overly sparse spots in certain areas.

[0049] The light spot data acquisition module uses a sensor (such as a camera, a laser scanner, etc.) to collect light spot data in each divided sub-area;

[0050] This includes information such as the shape, position, and degree of deformation of the light spot. These data are acquired in real time through image acquisition technology or laser scanning technology. The purpose is to obtain data on the change of the light spot in each sub-area. These data are the basis for subsequent analysis and adjustment, especially the light spot deformation information, which can reveal the geometric shape and complexity of the object surface. This stage ensures the accurate acquisition of the light spot data and provides support for the next step of feature extraction and analysis.

[0051] The light spot morphology feature extraction module extracts the features related to the light spot morphology change from the collected light spot data for each sub-area, analyzes the extracted light spot morphology change features, and quantifies the light spot morphology change;

[0052] The core purpose of this stage is to conduct a deeper analysis of the characteristics of the spot morphology changes and convert them into numerical feature quantities. These quantitative features can provide input for subsequent deep learning models, thereby helping the model to evaluate the spot density requirements of each sub-area.

[0053] For each sub-area, features related to the spot morphology change are extracted from the collected spot data, wherein the extracted spot morphology change features include the geometric complexity of the spot morphology and the degree of change of the spot edge angle. After analyzing the extracted geometric complexity features of the spot morphology and the degree of change of the spot edge angle, a spot geometric complexity reference value and a spot angle distortion reference value are generated respectively. The spot morphology change is quantified by the spot geometric complexity reference value and the spot angle distortion reference value.

[0054] A high geometric complexity of the spot morphology usually means that there are significant deformations or geometric features on the surface, which will cause a large change in the spot morphology. When the shape of the object surface is more complex, such as sharp changes in curvature, sharp angles, protrusions or depressions, the light spot projected onto the surface will be significantly deformed due to the geometric characteristics of the surface. In these areas, the shape of the spot may not only be offset, but may even be stretched, compressed or twisted. This complex geometric change directly leads to an increase in the complexity of the spot morphology. Therefore, the higher the geometric complexity of the spot morphology, it usually means that there are more significant morphological changes on the object surface, reflecting that the geometric characteristics of these areas are more complex or sudden, resulting in a greater degree of change in the spot morphology.

[0055] The specific steps of generating a spot geometric complexity reference value after analyzing the extracted geometric complexity characteristics of the spot morphology are as follows:

[0056] The edge contour of the light spot is extracted by image processing method to obtain the projection shape of each light spot on the surface of the object. Common edge detection techniques such as Canny operator or edge tracking algorithm can be applied to extract the shape of the light spot. Then, the shape characteristics of each light spot are analyzed, the edge curvature and shape distortion of the light spot are calculated, and the curvature of the light spot is defined as , the calculation expression of the spot curvature is: ,in, and are the first and second order derivatives of the spot profile function on the x and y coordinates respectively;

[0057] The purpose of this step is to capture the local changes in the spot shape, especially the response to complex surfaces or geometric feature areas, providing a basis for subsequent analysis.

[0058] The geometric distortion of the light spot is further quantified based on the edge profile and curvature of the light spot. In order to fully capture the changes in morphology, the geometric distortion index is introduced to evaluate the deviation between the light spot shape and the ideal circular or ideal plane light spot. The geometric distortion index is defined as: ,in, is the geometric distortion index, and are the areas of the actual light spot and the ideal light spot at positions x and y, respectively. and are the local curvatures of the actual light spot and the ideal light spot respectively, and is an exponential constant that controls distortion sensitivity;

[0059] The purpose of this step is to accurately evaluate the degree of geometric distortion of each light spot in the sub-region, especially in the complex surface or curved area, reflecting the intensity of the change in the light spot morphology.

[0060] The geometric distortion indexes obtained from multiple spots are combined to form a global spot geometric complexity reference value. The generation expression of the spot geometric complexity reference value is: ,in, is the reference value of the spot geometry complexity, is the geometric distortion index of the i-th light spot, N is the total number of light spots, is the area of ​​the ith light spot, is the total area of ​​the subregion, is an adjustment index used to control the weight of the geometric distortion index in the total complexity;

[0061] The function of this step is to weight the geometric distortion of each light spot in the sub-area to obtain an overall reference value of the light spot geometric complexity.

[0062] It can be seen from the spot geometric complexity reference value that for each sub-region, the larger the performance value of the spot geometric complexity reference value generated after analyzing the geometric complexity characteristics of the extracted spot morphology, the greater the change in the spot morphology in the sub-region. This is because the spot geometric complexity reference value is obtained by considering multiple factors such as the distortion of the spot, the change in local curvature, and the deviation from the ideal morphology when quantifying the change in the spot morphology. A larger spot geometric complexity reference value indicates that the morphology of the spot has undergone a large deformation or change, especially in complex surfaces or curved areas, where the spot may be significantly stretched, twisted, or offset, resulting in an increase in its geometric complexity. Conversely, a smaller spot geometric complexity reference value indicates that the spot morphology has changed less, which usually means that the spot is relatively regular and stable in the sub-region and does not change much. Therefore, the spot geometric complexity reference value can effectively reflect the intensity of change in the spot morphology in different sub-regions, and serves as an important indicator for measuring morphological changes.

[0063] When the degree of change in the edge angle of the light spot is high, it usually means that the morphology of the light spot has changed significantly. This is because the edge angle of the light spot is directly affected by the geometric morphology of the object surface, especially in areas with sharp curvature changes or irregular surface structures. When there are features such as protrusions, depressions, sharp angles or complex curved surfaces on the surface of the object, the projection path of the light spot will bend or deform, resulting in a significant change in the edge angle of the light spot. For example, in areas with curved surfaces or sharp edges, the edge of the light spot will produce a sharp angle change due to the influence of the surface geometry. This change in angle reflects the complexity of the geometric features of the object surface, especially when the angle change reaches a high degree, indicating that the surface morphology has changed significantly, which may be due to the height difference, curvature change or irregular structure of the surface. Therefore, a large change in the edge angle of the light spot usually indicates that the morphology of the light spot in this area has changed significantly, and may involve complex surface geometry.

[0064] For each sub-region, the specific steps of analyzing the variation characteristics of the extracted spot edge angle and generating the spot angle distortion reference value are as follows:

[0065] For the sub-region, first extract the edge point set of all light spots and calculate the local angle change of each edge point. The edge point set of , the local tangent direction of each edge point is , the local angle change of each edge point is calculated by the angle between adjacent tangent directions, and the calculation expression is: ,in: Represents edge points The tangent direction of Represents edge points The tangent direction of is calculated by the coordinate difference of adjacent edge points, such as , Represents edge points The local angle change between its neighboring points;

[0066] By calculating the local angle change of the spot edge, the degree of deformation of each edge point can be quantified. This step captures the edge characteristics of the spot caused by surface geometry changes in the sub-region and is the basis for generating a reference value for the spot angle distortion.

[0067] Light spot The angle changes of all edge points are weighted and aggregated to generate the overall edge angle change to reflect the overall morphological change of the light spot. The overall edge angle change is defined as: ,in: is the overall edge angle change of the jth light spot, n is the light spot The number of edge points, For edge points The curvature is usually calculated by the rate of change of the normal vector, reflecting the geometric complexity of the edge. Represents edge points and The distance between them is used to evaluate the contribution of edge points to the overall morphological change. It is a light spot The total edge length is used to normalize the influence of each edge point, It is a nonlinear adjustment parameter used to enhance the influence of edge points with large morphological changes;

[0068] By combining the weights of curvature and edge length distribution to calculate the overall angle change, this step can more accurately quantify the morphological changes of the light spot, especially in complex edge areas, and highlight significant distortion features.

[0069] All the spots in the sub-area The overall edge angle change is aggregated to generate the spot angle distortion reference value of the sub-region. In order to enhance the sensitivity to complex geometric areas, the relative geometric relationship between the spots (such as the center point spacing) is added as an additional weight factor. The calculation expression of the spot angle distortion reference value is: ,in: is the reference value of the spot angle distortion, m is the number of spots in the sub-area, It is a light spot The distance between the center point and the center of the sub-area reflects the distribution position of the light spot in the area. is the distance attenuation coefficient, which is used to reduce the impact of the light spot in the edge area on the overall result.

[0070] It can be seen from the spot angle distortion reference value that for each sub-region, the larger the performance value of the spot angle distortion reference value generated after analyzing the characteristics of the degree of change of the extracted spot edge angle, the greater the change in the morphology of the spot in the sub-region, and vice versa. This is because the spot angle distortion reference value quantifies the complexity of the spot morphology by analyzing the characteristics of the change in the spot edge angle. The larger the value, the more drastic the change in the spot edge angle in the sub-region, which may be due to the significant geometric complexity of the object surface, such as sharp curvature changes, unevenness or sharp angle features. On the contrary, if the spot angle distortion reference value is small, it means that the angle change of the spot edge is relatively gentle, reflecting that the surface morphology in the sub-region is relatively flat or regular. Therefore, the spot angle distortion reference value can directly characterize the degree of change of the spot morphology in the sub-region, providing a key reference for subsequent spot density optimization and three-dimensional reconstruction.

[0071] The feature vector conversion and evaluation module converts the quantified spot morphology change features after analysis into feature vectors, and inputs the converted feature vectors into the pre-trained deep learning model to conduct a comprehensive evaluation of the spot morphology changes in each sub-area;

[0072] The quantified spot geometry complexity reference value and spot angle distortion reference value after analysis are converted into feature vectors, and the converted feature vectors are input into a pre-trained deep learning model. The morphological variation coefficient is generated by the deep learning model, and the spot morphological changes in each sub-area are comprehensively evaluated by the morphological variation coefficient.

[0073] A pre-trained deep learning model refers to a deep learning framework that has completed preliminary training on a large amount of data and has high predictive ability. This type of model can handle the relationship between high-dimensional, complex, and nonlinear features, and is very suitable for analyzing and quantifying the comprehensive characteristics of the spot geometry complexity reference value and the spot angle distortion reference value. In the current task, the choice of deep learning model should fit the evaluation requirements of the spot morphological changes and be able to extract deep patterns from the input feature vector to generate morphological variation coefficients. For example, using a fully connected neural network is an ideal choice because its structure is simple but can effectively capture the nonlinear relationship between input features. Through the processing of multi-layer neural networks, the model can automatically learn the complex interactions of spot features and generate morphological variation coefficients to comprehensively evaluate the spot changes in sub-areas.

[0074] In this scenario, the pre-trained deep learning model has usually been preliminarily trained on a large-scale dataset similar to the current task, such as a geometric feature analysis task related to the characteristics of the spot morphological change. This process can be completed through transfer learning, that is, loading the weights of the original model into the new task, retaining the original low-level feature extraction capabilities, and fine-tuning and optimizing according to the current spot data. This approach has two significant advantages: first, it reduces the dependence on large-scale annotated data because the model has learned basic features in similar tasks; second, transfer learning significantly speeds up the training time of the model and improves the model's generalization ability to small-scale datasets. Through the application of this deep learning model, the system can generate the morphological variation coefficient with higher accuracy and robustness, thereby achieving a comprehensive evaluation of the sub-region spot morphological changes.

[0075] The deep learning model is not specifically limited here. Any model that can achieve the reference value of the spot geometry complexity and spot angle distortion reference value In order to realize the technical solution of the present invention, the present invention provides a specific implementation method, by taking the spot geometric complexity reference value as and spot angle distortion reference value It is converted into a standardized feature vector and input into the deep learning model, which generates a morphological variation coefficient after comprehensive analysis by the deep learning model, further realizing the accurate quantification of the morphological changes of the sub-area light spot, providing a reliable basis for subsequent dynamic optimization and adjustment.

[0076] In order to realize the technical solution of the present invention, the present invention provides a specific implementation method; morphological variation coefficient The generated expression is: , where , are the reference values ​​of the light spot geometry complexity and spot angle distortion reference value The preset scaling factor of , Both are greater than 0.

[0077] It can be seen from the morphological variation coefficient that, for each sub-region, the greater the performance value of the spot geometric complexity reference value generated after analyzing the extracted geometric complexity characteristics of the spot morphology, the greater the performance value of the spot angle distortion reference value generated after analyzing the extracted characteristics of the change degree of the spot edge angle. That is, the greater the performance value of the morphological variation coefficient generated by comprehensively evaluating the spot morphology changes in each sub-region through the pre-trained deep learning model, the greater the change in the spot morphology in the sub-region, and vice versa.

[0078] The spot density adjustment module uses a cluster analysis algorithm to adjust the spot density of each sub-area based on the evaluation results of the deep learning model, and adaptively allocates different spot projection densities to different surface features;

[0079] The deep learning model analyzes the changes in the spot morphology of each sub-area and intelligently determines which areas require a higher density of spots and which areas can maintain a lower density. This adjustment process is dynamic, and corresponding density allocation is adopted for different surface features. The main function of this step is to dynamically adjust the spot density according to the actual situation of each sub-area and the changes in the spot morphology. Through this adaptive density adjustment, accurate spot distribution can be achieved between various areas on the surface of the object, ensuring that complex areas have sufficient spot density to capture subtle changes and avoid data loss or reconstruction errors caused by too low density.

[0080] Based on the evaluation results of the deep learning model, a cluster analysis algorithm is used to adjust the spot density of each sub-area. The specific steps are as follows:

[0081] After the initial spot projection density is projected, the spot data features of each sub-area are extracted to form a feature set. The expression of the feature set is: ,in: represents the eigenvector of the qth light spot in the pth sub-region, is the initial light spot projection density, indicating the initial distribution of the light spot in the sub-area, is the morphological variation coefficient of the pth sub-region, reflecting the complex changes in the spot morphology due to surface characteristics. Other auxiliary features (such as spot edge complexity, position offset, etc.);

[0082] The spot features are organized into a unified high-dimensional vector form to facilitate subsequent clustering analysis. At the same time, data preprocessing helps to eliminate noise and improve data consistency.

[0083] Use clustering algorithms (such as k-means or DBSCAN) to cluster the sub-region feature vectors Clustering is performed to classify spots with similar characteristics into the same category. The goal is to minimize the difference within the cluster. The clustering formula is as follows: ,in: The kth cluster contains sub-regions with similar characteristics. is the center point of the kth cluster, representing the average characteristics of the cluster, is the Euclidean distance between the feature vector and the cluster center;

[0084] Clustering sub-regions with similar spot change characteristics into groups helps to identify regular changes on the object surface and provide a regional basis for subsequent spot density adjustment.

[0085] The spot density of each sub-area is adjusted according to the morphological variation coefficient, and the optimized spot density is calculated. The calculation expression of the adjusted spot projection density is: ,in: is the adjusted light spot projection density, is the initial spot projection density, is the density adjustment factor, which controls the density adjustment range (usually 0.5-1.5), and are the minimum and maximum values ​​of the morphological variation coefficient, respectively, which are used for normalization processing;

[0086] Dynamically optimize the spot density of each sub-area to ensure that areas with complex surface features are covered with higher density, while simple areas maintain lower density, thereby achieving efficient allocation of spot resources.

[0087] The spot projection system is reset according to the optimized spot density, and the adjusted spot projection density is projected onto the surface of the object. The distribution expression of the adjusted spot projection density is: ,in: is the optimized overall spot density distribution set, K is the number of clustering categories, represents the joint result of the optimized density of all sub-regions;

[0088] Through re-projection, the optimized spot density can accurately cover complex areas and capture more details, while reducing redundant sampling in flat areas, improving the accuracy of 3D reconstruction and resource utilization efficiency.

[0089] The light spot re-projection module re-projects the light spot onto the surface of the object according to the adjusted light spot projection density, so that the light spot distribution in each sub-area is more refined, and finally the detailed geometric information of the object surface is obtained through the projected light spot;

[0090] The goal of this step is to accurately capture subtle changes on the surface of the object through the redistributed spot density, especially in complex or curved areas. With a higher density spot, surface details can be acquired more accurately, reducing data loss caused by sparse spots, thereby improving the accuracy and quality of 3D reconstruction.

[0091] Through the above-mentioned optical scene reconstruction system combining structured light and three-dimensional vision, the three-dimensional reconstruction accuracy and system adaptability of complex surface areas are significantly improved, and the shortcomings of uniform spot projection density when processing complex surfaces are effectively solved. Specifically, the solution adopts a strategy of dynamically optimizing the spot projection density, and adaptively adjusts the spot distribution in different areas through deep learning model evaluation and cluster analysis, so that the spot density is increased on curved surfaces, concave and convex areas or places with complex geometric features, thereby capturing more details; while in flat or regular areas, the spot density is appropriately reduced to reduce calculation and resource waste. This optimization makes the spot distribution of the entire surface more refined and balanced, covering all key feature areas, avoiding data loss and error amplification, and ensuring that the reconstruction results are more complete and accurate. In addition, the design of the system takes into account both efficiency and flexibility. Through intelligent density allocation, it not only meets the needs of high-precision reconstruction, but also reduces the overall computational complexity of the system, improves the reconstruction efficiency and the reliability of subsequent analysis, and is suitable for high-precision scene applications such as industrial inspection, reverse engineering and complex geometric modeling.

[0092] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0093] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0094] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. An optical scene reconstruction system combining structured light and three-dimensional vision, characterized in that: It includes an initial spot projection module, a region division module, a spot data acquisition module, a spot morphology feature extraction module, a feature vector conversion and evaluation module, a spot density adjustment module and a spot re-projection module; The initial spot projection module adopts the initial spot projection density and uses the laser to project the spot pattern onto the surface of the object to form a preliminary spot projection area; The area division module divides the preliminary light spot projection area into several sub-areas of equal area; A light spot data acquisition module, in each divided sub-area, uses a sensor to collect light spot data in the sub-area; The light spot morphology feature extraction module extracts the features related to the light spot morphology change from the collected light spot data for each sub-area, analyzes the extracted light spot morphology change features, and quantifies the light spot morphology change; The feature vector conversion and evaluation module converts the quantified spot morphology change features after analysis into feature vectors, and inputs the converted feature vectors into the pre-trained deep learning model to conduct a comprehensive evaluation of the spot morphology changes in each sub-area; The spot density adjustment module uses a cluster analysis algorithm to adjust the spot density of each sub-area based on the evaluation results of the deep learning model, and adaptively allocates different spot projection densities to different surface features; The light spot re-projection module re-projects the light spot onto the surface of the object according to the adjusted light spot projection density, so that the light spot distribution in each sub-area is more refined, and finally the detailed geometric information of the object surface is obtained through the projected light spot.

2. The optical scene reconstruction system combining structured light and three-dimensional vision according to claim 1, characterized in that: The initial spot projection density setting is preset according to the overall size of the object, surface characteristics and task requirements. The specific steps are as follows: First, the surface area of ​​the object is estimated according to the size of the object, and the preliminary spot projection density is calculated according to the surface area and the required scanning accuracy; Next, analyze the geometric features of the object surface, including flat areas, curved areas, and areas with complex structures, and consider the spot density requirements of different areas. For flat areas, set a low spot density; for curved areas or areas with complex structures, increase the spot density to ensure the capture of details; Finally, according to the actual requirements of the task, a balanced initial light spot projection density is determined to ensure that enough light spots can be covered on the surface of the object without causing excessive computational burden or data redundancy.

3. The optical scene reconstruction system combining structured light and three-dimensional vision according to claim 1, characterized in that: The size and shape of each sub-region are adjusted according to the surface characteristics of the object to ensure that the area of ​​each sub-region is relatively uniform, which is convenient for subsequent analysis and calculation.

4. The optical scene reconstruction system combining structured light and three-dimensional vision according to claim 1, characterized in that: For each sub-area, features related to the spot morphology change are extracted from the collected spot data, wherein the extracted spot morphology change features include the geometric complexity of the spot morphology and the degree of change of the spot edge angle. After analyzing the extracted geometric complexity features of the spot morphology and the degree of change of the spot edge angle, a spot geometric complexity reference value and a spot angle distortion reference value are generated respectively. The spot morphology change is quantified by the spot geometric complexity reference value and the spot angle distortion reference value.

5. The optical scene reconstruction system combining structured light and three-dimensional vision according to claim 3, characterized in that: The specific steps of generating a spot geometric complexity reference value after analyzing the extracted geometric complexity characteristics of the spot morphology are as follows: The edge contour of the light spot is extracted by image processing method to obtain the projection shape of each light spot on the surface of the object. Then, the shape characteristics of each light spot are analyzed, the edge curvature and shape distortion of the light spot are calculated, and the curvature of the light spot is defined as , the calculation expression of the spot curvature is: ,in, and are the first and second order derivatives of the spot profile function on the x and y coordinates respectively; The geometric distortion of the light spot is further quantified based on the edge profile and curvature of the light spot. In order to fully capture the changes in morphology, the geometric distortion index is introduced to evaluate the deviation between the light spot shape and the ideal circular or ideal plane light spot. The geometric distortion index is defined as: ,in, is the geometric distortion index, and are the areas of the actual light spot and the ideal light spot at positions x and y, respectively. and are the local curvatures of the actual light spot and the ideal light spot, respectively. and is an exponential constant that controls distortion sensitivity; The geometric distortion indexes obtained from multiple spots are combined to form a global spot geometric complexity reference value. The generation expression of the spot geometric complexity reference value is: ,in, is the reference value of the spot geometry complexity, is the geometric distortion index of the ith light spot, N is the total number of light spots, is the area of ​​the ith light spot, is the total area of ​​the subregion, is a tuning index used to control the weight of the geometric distortion index in the total complexity.

6. The optical scene reconstruction system combining structured light and three-dimensional vision according to claim 5, characterized in that: For each sub-region, the specific steps of analyzing the variation characteristics of the extracted spot edge angle and generating the spot angle distortion reference value are as follows: For the sub-region, first extract the edge point set of all light spots and calculate the local angle change of each edge point. The edge point set is , the local tangent direction of each edge point is , the local angle change of each edge point is calculated by the angle between adjacent tangent directions, and the calculation expression is: ,in: Represents edge points The tangent direction of Represents edge points The tangent direction is calculated by the coordinate difference of adjacent edge points, specifically: , Represents edge points The local angle change between its neighboring points; Light spot The angle changes of all edge points are weighted and aggregated to generate the overall edge angle change to reflect the overall morphological change of the light spot. The overall edge angle change is defined as: ,in: is the overall edge angle change of the jth light spot, n is the light spot The number of edge points, For edge points The curvature of Represents edge points and The distance between them is used to evaluate the contribution of edge points to the overall morphological change. It is a light spot The total edge length is used to normalize the influence of each edge point, It is a nonlinear adjustment parameter used to enhance the influence of edge points with large morphological changes; All the spots in the sub-area The overall edge angle change is aggregated to generate the spot angle distortion reference value of the sub-region. In order to enhance the sensitivity to complex geometric areas, the relative geometric relationship between the spots is added as a weight factor. The calculation expression of the spot angle distortion reference value is: ,in: is the reference value of the spot angle distortion, m is the number of spots in the sub-area, It is a light spot The distance between the center point and the center of the sub-area reflects the distribution position of the light spot in the area. is the distance attenuation coefficient, which is used to reduce the impact of the light spot in the edge area on the overall result.

7. The optical scene reconstruction system combining structured light and three-dimensional vision according to claim 6, characterized in that: The quantified spot geometry complexity reference value and spot angle distortion reference value after analysis are converted into feature vectors, and the converted feature vectors are input into a pre-trained deep learning model. The morphological variation coefficient is generated by the deep learning model, and the spot morphological changes in each sub-area are comprehensively evaluated by the morphological variation coefficient.

8. The optical scene reconstruction system combining structured light and three-dimensional vision according to claim 7, characterized in that: Based on the evaluation results of the deep learning model, a cluster analysis algorithm is used to adjust the spot density of each sub-area. The specific steps are as follows: After the initial spot projection density is projected, the spot data features of each sub-area are extracted to form a feature set. The expression of the feature set is: ,in: represents the eigenvector of the qth light spot in the pth sub-region, is the initial light spot projection density, indicating the initial distribution of the light spot in the sub-area, is the morphological variation coefficient of the pth sub-region, reflecting the complex changes in the spot morphology due to surface characteristics. It is an auxiliary feature; Using clustering algorithm, the sub-region feature vector Clustering is performed to classify spots with similar characteristics into the same category. The goal is to minimize the difference within the cluster. The clustering formula is as follows: ,in: The kth cluster contains sub-regions with similar characteristics. is the center point of the kth cluster, representing the average characteristics of the cluster, is the Euclidean distance between the feature vector and the cluster center; The spot density of each sub-area is adjusted according to the morphological variation coefficient, and the optimized spot density is calculated. The calculation expression of the adjusted spot projection density is: ,in: is the adjusted light spot projection density, is the initial spot projection density, is the density adjustment factor, which controls the density adjustment range. and are the minimum and maximum values ​​of the morphological variation coefficient, respectively, which are used for normalization processing; The spot projection system is reset according to the optimized spot density, and the adjusted spot projection density is projected onto the surface of the object. The distribution expression of the adjusted spot projection density is: ,in: is the optimized overall spot density distribution set, K is the number of clustering categories, represents the joint result of the optimized density of all sub-regions.

Images