Dynamic cooperative rendering industrial surface defect detection method, system, device and medium

Abstract

The present application relates to the field of computer vision and industrial detection technology, and particularly relates to a dynamic collaborative rendering industrial surface defect detection method, system, device and medium, the method comprising the following steps: acquiring multi-view point cloud data and high-resolution color image sequences of a defect-free standard part; constructing a high-precision global dense point cloud under an absolute world coordinate system; generating a Mesh model and a 3DGS model under the world coordinate system; online collecting a two-dimensional image of a workpiece to be detected, performing two-dimensional anomaly preliminary screening based on the two-dimensional image, locking a suspected defect area, and extracting a two-dimensional bounding box of the suspected defect area; calculating a global defect probability, dynamically scheduling the Mesh model and / or the 3DGS model based on the global defect probability to render the image to be detected collected in the current frame, and outputting a hybrid rendering reference image; performing consistency checking on the hybrid rendering reference image, and obtaining a final detection result. The method reduces the calculation cost and improves the defect detection accuracy.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and industrial inspection technology, and in particular to a dynamic collaborative rendering method, system, equipment and medium for detecting industrial surface defects. Background Technology

[0002] In the field of surface defect detection for industrial products, traditional two-dimensional vision inspection methods mainly rely on image processing algorithms, such as edge detection and frequency domain analysis. These methods are effective in handling planar texture defects, but their accuracy is limited for detecting minute three-dimensional geometric defects (such as pits, bumps, and scratches), and they are easily affected by changes in lighting and texture interference. With the development of three-dimensional reconstruction technology, mesh-based three-dimensional inspection methods are gradually being applied in industrial inspection. By acquiring three-dimensional point cloud data of objects through structured light, laser scanning, or multi-view stereo vision, and then reconstructing it into a triangular mesh model (Mesh), the geometric structure of the object can be accurately represented. However, high-precision mesh rendering has high computational complexity, making it difficult to meet the real-time inspection needs of industrial production lines. The main problems with existing technologies are: The contradiction between accuracy and efficiency: Although high-precision mesh rendering can accurately detect tiny geometric defects, its computational complexity is high, making it difficult to meet the frame rate requirements of industrial real-time detection. Insufficient ability to express details: Although the emerging 3D Gaussian Splatting (3DGS) technology has a fast rendering speed and can achieve real-time interactive level, it often smooths high-frequency geometric details in pursuit of efficiency, resulting in the loss of minor defect features. Lack of intelligent scheduling mechanism: Existing 3D detection methods usually adopt a uniform rendering strategy, ignoring the prior fact that only a very small number of objects usually have defects, resulting in a waste of computing resources.

[0003] Therefore, it is necessary to provide a dynamic collaborative rendering method for detecting industrial surface defects to solve the above-mentioned technical problems. Summary of the Invention

[0004] The main objective of this invention is to provide a dynamic collaborative rendering method, system, device, and medium for detecting industrial surface defects, aiming to solve the problem of the trade-off between accuracy and efficiency in existing technologies, and to achieve high-precision, real-time detection of industrial surface defects. The specific technical solution is as follows: A dynamic collaborative rendering method for detecting industrial surface defects includes the following steps: Step S1: Obtain multi-view point cloud data and high-resolution color image sequences of defect-free standard parts; construct a high-precision global dense point cloud in the absolute world coordinate system; Step S2: Based on the global dense point cloud, generate Mesh model and 3DGS model from the same source in the world coordinate system; Step S3: Acquire two-dimensional images of the workpiece to be inspected online, perform preliminary screening of two-dimensional anomalies based on the two-dimensional images, locate suspected defect areas, and extract the two-dimensional bounding boxes of the suspected defect areas. ; Step S4: Calculate the global defect probability Based on global defect probability Dynamically schedule the Mesh model and / or 3DGS model to render the image to be detected acquired online in the current frame, and output a hybrid rendering baseline map; Step S5: Perform consistency verification on the hybrid rendering baseline map and obtain the final detection result.

[0005] Furthermore, in step S2, based on the global dense point cloud, the Poisson reconstruction algorithm is used to extract isosurfaces and generate a Mesh model that retains high-frequency geometric details; the three-dimensional spatial coordinates of the global dense point cloud are extracted as the initial center position of the 3DGS Gaussian sphere, and the 3DGS model is generated by combining it with a high-resolution color image sequence.

[0006] Furthermore, step S3 includes: Step S3.1: Acquire a two-dimensional image of the workpiece to be inspected online, input the two-dimensional image of the workpiece to be inspected into a pre-trained multi-scale feature distribution modeling network, and evaluate the anomaly score of each spatial location in the feature map that deviates from the normal distribution. Obtain a global anomaly score heatmap; Step S3.2: Perform spatial smoothing on the global anomaly score heatmap. Set an empirical threshold to perform binary truncation on the smoothed global anomaly score heatmap. Use morphological filtering and connected component labeling algorithms to find abnormal connected regions exceeding the empirical threshold as suspected defect regions. Calculate and extract the minimum bounding rectangle of the suspected defect region and output the two-dimensional bounding box. .

[0007] Furthermore, step S3.1 includes: ① Establish normal feature distribution parameters during the offline phase: Collect a sufficient number of defect-free standard part images containing normal operating condition fluctuations to construct a normal sample set; input all normal samples into a pre-trained multi-scale feature distribution modeling network identical to that used in the online stage to obtain feature maps; for each spatial pixel coordinate on the feature map... Independently count the feature vectors of all normal samples at this position, and calculate the mean vector of their multivariate Gaussian distribution. and sample covariance matrix And calculate the inverse of the covariance matrix. ; ② Input the online-acquired image to be detected into a pre-trained multi-scale feature distribution modeling network to extract the spatial pixel coordinates. high-dimensional feature vectors at the location ; ③ Call the normal feature distribution parameters established in the offline phase, and calculate the Mahalanobis distance deviating from the normal distribution as the anomaly score using the following formula. : ; in: This represents the transpose of a matrix.

[0008] Furthermore, in step S4, the global defect probability is calculated. Specifically: After spatial smoothing the global anomaly score heatmap, the global maximum anomaly score is extracted. ; The global defect probability is calculated using the following formula. : ; in: This represents the preset sensitivity coefficient, which is derived by working backward from the production line's quality control requirements. This represents the extracted global maximum anomaly score; Represented by natural constant An exponential function with base 0.

[0009] Furthermore, in step S4, based on the global defect probability... With preset trigger threshold Dynamically schedule the Mesh model and / or 3DGS model to render the image to be detected acquired online in the current frame, and output a hybrid rendering baseline map; specifically: when When the image to be detected acquired online in the current frame is determined to be normal, only the 3DGS model is called to quickly render and generate a complete defect-free two-dimensional reference image, i.e., a global baseline view, under the current virtual camera view, to complete the macroscopic verification. when At that time, the Mesh model and the 3DGS model are used for heterogeneous hybrid rendering. Specifically: The two-dimensional bounding box extracted in step S3 Back-projected to the 3D world coordinate system, a 3D view frustum is generated; the Mesh model is spatially clipped, and the local Mesh mesh patches that intersect with the 3D view frustum are rendered using high-precision rasterization; the background area outside the 3D view frustum continues to be rendered using the 3DGS model; Z-Buffer Feathering: In heterogeneous hybrid rendering, local meshes and background 3DGS are forced to share the Z-Buffer depth buffer under a unified virtual camera view. Through depth testing, the depth values ​​of the rendered fragments from the camera optical center of the two models are compared pixel by pixel, retaining fragments closer to the virtual camera to achieve correct geometric occlusion at the physical level. At the same time, a non-linear blending weight factor is introduced at the 3D boundary of the view frustum. Through formula Perform alpha blending to output a seamlessly stitched, high-precision blending rendering baseline image; where: This represents the final pixel color value after blending and rendering. This represents the color contribution value of local mesh rasterization rendering. This represents the color contribution value of the background 3DGS rendering.

[0010] Furthermore, generating a 3D view frustum specifically involves: Spatial mapping preparation: Pre-calibrating the intrinsic parameter matrix of industrial field cameras. and extrinsic parameter matrix ; View frustum boundary line calculation: obtaining pixel coordinates of the four corner points , Using the camera's intrinsic parameter matrix and extrinsic parameter matrix Calculate the 3D optical center coordinates of the virtual camera. ,in For the camera's rotation matrix, The translation vector of the camera; the direction vectors of the four boundary rays of the view frustum in the world coordinate system. The solution is as follows: ; 3D view frustum construction: with optical center With the vertex as the top and the four boundary rays as the side edges, a closed detection cone shaped like a truncated cone is constructed in three-dimensional physical space. .

[0011] Furthermore, spatial clipping of the mesh model specifically involves: organizing the offline mesh model using hierarchical bounding boxes (BVH); calculating the relationship between each bounding box and the view frustum. Intersection relationships, eliminating those located at the view cone External nodes only load local mesh faces that pass the intersection test into the GPU memory rendering queue for high-precision rasterization rendering.

[0012] Furthermore, preset trigger thresholds Generated based on offline calibration of normal production line samples; specifically: Based on the normal sample set, a sufficient number of defect-free normal samples are extracted, their global anomaly probabilities are calculated, and they are sorted in descending order. According to the maximum tolerable false trigger rate set by the production line, the probability value of the corresponding quantile is directly extracted as... .

[0013] Furthermore, depth testing specifically involves: Construct a two-dimensional global Z-Buffer that corresponds one-to-one with each screen pixel to record the current screen pixel. Minimum physical depth value from the optical center of the virtual camera When a local mesh rasterized fragment or a 3DGS integral fragment attempts to write screen coordinates... At that time, it will attempt to write screen coordinates. The fragment is used as a new fragment, and the depth value of the new fragment is extracted. ; calculate The Boolean value indicates that the new fragment is closer to the virtual camera in physical 3D space and passes the depth test. The color buffer is overwritten with the color of the new fragment. If the value is false, the new fragment is physically occluded and fails the depth test. The new fragment will be discarded directly.

[0014] Furthermore, in the visual cone The distance extended inward from the four inner boundaries Construct a 3D attenuation transition zone; calculate the orthogonal projection distance from the point to be rendered to the side boundary of the view frustum. Define nonlinear mixed weighting factors :when , ;when hour, The rest are 0.

[0015] Furthermore, step S5 specifically involves: Anti-vibration subpixel registration was performed between the actual acquired image and the hybrid rendering baseline image: geometric edge gradient structure features of the two images were extracted respectively. Luminous and color distribution photometric characteristics Through weighted formula Generate a comprehensive residual heatmap, where: For geometric structure feature weights, For photometric feature weights, The calculated composite residual value; Residual back projection reconstruction: Obtain the pixel coordinates greater than the judgment threshold in the comprehensive residual heatmap. Extract its physical depth value from Z-Buffer ; Using a camera matrix, pixel coordinates are inversely projected to absolute three-dimensional world coordinates. : ; Cross-plane ray intersection verification: Requesting the viewpoint Calibration pose of adjacent view cameras ;Will Reprojection to viewpoint If residuals exceeding the judgment threshold also exist, then construct the optical center from the main viewpoint to the... rays and perspective Light Heart rays ; Calculate the length of the shortest common perpendicular segment between two rays. ; Artifact removal: If , As a tolerance threshold, it proves that multi-view anomalous residuals truly intersect at the same three-dimensional surface point, confirming it as a real physical geometric defect; if If the score of the reprojected position in the comprehensive residual heat map does not exceed the maximum tolerable false trigger rate set by the production line, it is judged as anisotropic reflection and forcibly removed. The defect is classified by combining the area of ​​the connected domain of the actual physical geometric defect after verification with the peak value of the residual, and an inspection report with absolute three-dimensional coordinates is output.

[0016] This invention also provides an industrial surface defect detection system based on dynamic collaborative rendering, used to implement the dynamic collaborative rendering industrial surface defect detection method described above. It includes an offline homogeneous reconstruction module and an online dynamic collaborative detection module. The offline homogeneous reconstruction module is used to collect multi-view point cloud data and high-resolution color image sequences of defect-free standard parts, construct a high-precision global dense point cloud in the absolute world coordinate system, and generate Mesh models and 3DGS models in the world coordinate system based on the global dense point cloud. The online dynamic collaborative detection module is used to perform dynamic collaborative detection on the online acquired images to be detected based on the Mesh model and the 3DGS model, and output the detection results.

[0017] Furthermore, the offline homogeneous reconstruction module includes a multimodal data acquisition unit, an intensity-geometric fusion registration unit, and a model homogeneous generation unit; the online dynamic collaborative detection module includes an online image acquisition and pose calibration unit, a global anomaly probability assessment and ROI extraction unit, a pure 3DGS global fast rendering unit, a view frustum backprojection and local mesh on-demand loading unit, a Z-Buffer feathering hybrid rendering unit, a multidimensional feature residual analysis unit, a multi-view ray intersection verification unit, and a unit for outputting three-dimensional defect coordinates and levels.

[0018] The present invention also provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program; the processor executes the computer program to realize the dynamic collaborative rendering industrial surface defect detection method as described above.

[0019] The present invention also provides a readable storage medium storing a computer program, wherein a processor executes the computer program to implement the dynamic collaborative rendering industrial surface defect detection method as described above.

[0020] The beneficial effects achieved by this solution are: (1) This invention provides a dynamic collaborative rendering method for detecting industrial surface defects, comprising the following steps: Step S1: Acquire multi-view point cloud data and high-resolution color image sequences of defect-free standard parts; construct a high-precision global dense point cloud in an absolute world coordinate system; Step S2: Based on the global dense point cloud, generate a Mesh model and a 3DGS model in the world coordinate system; Step S3: Online acquisition of two-dimensional images of the workpiece to be inspected, perform initial screening of two-dimensional anomalies based on the two-dimensional images, lock the suspected defect area, and extract the two-dimensional bounding box of the suspected defect area. Step S4: Calculate the global defect probability. Based on global defect probability The method dynamically schedules the Mesh model and / or 3DGS model to render the image to be detected acquired online in the current frame, and outputs a hybrid rendering benchmark map; step S5: the hybrid rendering benchmark map is checked for consistency to obtain the final detection result. This invention achieves an optimal configuration of rendering computing power and detection accuracy: through two-dimensional visual attention (ROI) extraction and three-dimensional view frustum back-projection technology, this invention completely breaks through the computing power bottleneck of global high-precision rendering; in actual industrial production lines, since most areas are normal surfaces, the system will most likely only call the efficient 3DGS for rendering, significantly reducing computational costs; and when a suspected defect is triggered, only the local Mesh patches where view frustums intersect are redrawn with high-precision rasterization. This dynamic collaborative mechanism ensures micron-level geometric defect detection accuracy while avoiding the smoothing of high-frequency details by the 3DGS method, exponentially reducing the overall inference time and rendering resource consumption of the system.

[0021] (2) This invention constructs a zero-bias cross-dimensional spatial mapping chain: This invention proposes a "homogeneous dual-model reconstruction" mechanism for defect-free standard parts, using the same set of high-precision dense point clouds simultaneously for Mesh isosurface extraction and 3DGS sphere center initialization, eliminating the risk of spatial misalignment between the two data structures from a physical source. Combined with Z-Buffer depth sharing and boundary feathering fusion, perfect alignment of 2D images and 3D models and seamless stitching of hybrid rendering reference maps are achieved.

[0022] (3) This invention greatly improves the robustness against interference in complex industrial environments: In response to the technical bias that traditional pixel-level residuals are prone to false alarms, this invention introduces a multi-dimensional feature fusion residual of "geometric structure + photometric color". More importantly, through three-dimensional spatial view cone back projection and multi-view ray intersection verification, the system can effectively distinguish between real geometric defects and artifacts caused by industrial reflections / mechanical micro-vibrations from the physical space dimension, thereby achieving low false alarm and high reliability detection on industrial production lines.

[0023] (4) This invention proposes to use the bounding box output by a lightweight two-dimensional anomaly detection network as a spatial anchor point, and back-project it into a three-dimensional physical view frustum through the camera intrinsic and extrinsic parameter matrix. It abandons the "either / or" global replacement strategy in traditional hybrid rendering and realizes a computational drip scheduling of "only loading local meshes inside the view frustum".

[0024] (5) This invention proposes a physical-level feathering fusion strategy for heterogeneous 3D models based on shared depth buffer (Z-Buffer). In the underlying graphics rendering pipeline, it forces the local high-precision rasterization mesh and the global volume sputtering 3DGS to share a unified depth buffer. This not only achieves realistic occlusion relationships from a physical dimension, but also introduces a 3D attenuation weight factor based on spatial distance for alpha interpolation, eliminating the sense of disjointness at the splicing of different 3D data structures, and providing a solid foundation for subsequent defect detection.

[0025] (6) The method of the present invention breaks through the technical bias of existing methods that rely too much on “pixel-level residuals of two-dimensional images”. The present invention creatively reconstructs two-dimensional residuals in reverse to three-dimensional world coordinates and uses the shortest common perpendicular distance of spatial skew rays from multiple perspectives for verification, thus completely isolating real structural defects from wandering specular / reflective artifacts from the perspective of physical geometry. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating the dynamic collaborative rendering method for detecting industrial surface defects in an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the principle of the offline homogeneous dual-model (Mesh and 3DGS) joint generation method in an embodiment of the present invention; Figure 3This is a flowchart of the dynamic collaborative rendering decision-making process based on two-dimensional ROI view frustum back projection and Z-Buffer depth fusion in an embodiment of the present invention. Figure 4 This is a schematic diagram illustrating the principle of multi-view three-dimensional consistency ray intersection verification.

[0028] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0030] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0031] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0032] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0033] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0034] Example 1: See Figures 1-4 This embodiment proposes a dynamic collaborative rendering method for detecting industrial surface defects, including the following steps: Step S1: Obtain multi-view point cloud data and high-resolution color image sequences of defect-free standard parts; use the conventional ICP (Iterative Closest Point) algorithm that combines reflection intensity and geometric curvature, and the nonlinear optimization algorithm that fuses intensity and geometry to register the multi-view point clouds and construct a high-precision global dense point cloud in the absolute world coordinate system.

[0035] Step S2: Based on the global dense point cloud, generate a Mesh model and a 3DGS model in the world coordinate system; specifically: Based on global dense point cloud, the Poisson reconstruction algorithm is used to extract isosurfaces and generate a Mesh model that retains high-frequency geometric details; The three-dimensional spatial coordinates of the global dense point cloud are extracted as the initial center position of the 3DGS (3D Gaussian Splatting) Gaussian sphere. Then, the opacity, covariance matrix and spherical harmonic function of the Gaussian sphere are iteratively optimized by combining high-resolution color image sequences with existing differentiable rasterization technology. This generates a dual model of the same origin, Mesh and 3DGS, with absolute spatial consistency and zero pose deviation in the world coordinate system.

[0036] Step S3: Acquire two-dimensional images of the workpiece to be inspected online, perform preliminary screening of two-dimensional anomalies based on the two-dimensional images, locate suspected defect areas, and extract the two-dimensional bounding boxes of the suspected defect areas. Specifically, this includes: Step S3.1: Acquire a two-dimensional image of the workpiece to be inspected online, and input the two-dimensional image of the workpiece to be inspected into a pre-trained multi-scale feature distribution modeling network (this multi-scale feature distribution modeling network uses ResNet-18 or WideResNet-50 pre-trained on the ImageNet dataset as the backbone network), and evaluate the anomaly score of each spatial location in the feature map that deviates from the normal distribution. Obtain a global anomaly score heatmap.

[0037] Step S3.1 includes: ① Establish normal feature distribution parameters during the offline phase: Collect a sufficient number of defect-free standard part images containing normal operating condition fluctuations to construct a normal sample set; input all normal samples into a pre-trained multi-scale feature distribution modeling network identical to that used in the online stage to obtain feature maps; for each spatial pixel coordinate on the feature map... Independently count the feature vectors of all normal samples at this position, and calculate the mean vector of their multivariate Gaussian distribution. and sample covariance matrix And calculate the inverse of the covariance matrix. .

[0038] ② Input the online-acquired image to be detected into a pre-trained multi-scale feature distribution modeling network to extract the spatial pixel coordinates. high-dimensional feature vectors at the location The feature maps of the image to be detected at each network layer are extracted. The extracted feature maps at different scales are then upsampled using bilinear interpolation to align their spatial dimensions. Finally, the feature maps are analyzed at each pixel coordinate. The feature vector at that spatial location is obtained by concatenating the features along the channel dimension. .

[0039] ③ Call the normal feature distribution parameters (mean vector) established in the offline stage. With the inverse of the covariance matrix The Mahalanobis distance deviating from the normal distribution is calculated as the anomaly score using the following formula. : ; in: This represents the transpose of a matrix.

[0040] Step S3.2: Apply a conventional Gaussian filter to the global anomaly score heatmap for spatial smoothing to eliminate isolated noise interference; set an empirical threshold to perform binary truncation on the smoothed global anomaly score heatmap; use morphological filtering and connected component labeling algorithms to find abnormal connected regions exceeding the empirical threshold as suspected defect regions; calculate and extract the minimum bounding rectangle of the suspected defect region; extract the bounding box of the suspected defect region on the two-dimensional image as the region of interest (ROI); and output the two-dimensional bounding box. .

[0041] Step S4: Calculate the global defect probability Based on global defect probability The Mesh model and / or 3DGS model are dynamically scheduled to render the image to be detected acquired online in the current frame, outputting a hybrid rendering baseline map. Specifically, the steps include: ① Calculate the global defect probability Specifically: After spatial smoothing the global anomaly score heatmap, the global maximum anomaly score is extracted. ; The global defect probability is calculated using the following formula. : ; in: This represents the preset sensitivity coefficient, which is derived by working backward from the production line's quality control requirements. This represents the extracted global maximum anomaly score. Represented by natural constant An exponential function with base 0.

[0042] ②Based on global defect probability With preset trigger threshold Dynamically schedule the Mesh model and / or 3DGS model to render the image to be detected acquired online in the current frame, and output a hybrid rendering baseline map; specifically: when When the image to be detected acquired online in the current frame is determined to be normal, only the 3DGS model is called to quickly render and generate a complete defect-free two-dimensional reference image, i.e., a global baseline view, under the current virtual camera view, to complete the macroscopic verification. when At that time, the Mesh model and the 3DGS model are used for heterogeneous hybrid rendering. Specifically: The two-dimensional bounding box extracted in step S3 Back-projected to the 3D world coordinate system, a 3D view frustum is generated; the mesh model is spatially clipped, and the local mesh patches that intersect with the 3D view frustum are rendered with high precision rasterization; the background area outside the 3D view frustum continues to be rendered using the 3DGS model; Z-Buffer Feathering: In heterogeneous hybrid rendering, since Mesh uses a rasterization pipeline while 3DGS uses a volumetric sputtering pipeline, simply overlaying two-dimensional images will inevitably lead to severe 3D spatial distortion. This step aims to solve the physical occlusion and boundary fragmentation problems when the two heterogeneous 3D data models are rendered together on the same screen. This step forces the local Mesh and the background 3DGS to share the depth buffer Z-Buffer under a unified virtual camera view. Through depth testing, the depth values ​​of the rendered fragments of the two models from the camera's optical center are compared pixel by pixel, retaining the fragments closer to the virtual camera to achieve correct geometric occlusion at the physical level. At the same time, a non-linear blending weight factor is introduced at the 3D boundary of the view frustum. Through formula Perform alpha blending to output a seamlessly stitched, high-precision blending rendering baseline image; where: This represents the final pixel color value after blending and rendering. This represents the color contribution value of local mesh rasterization rendering. This represents the color contribution value of the background 3DGS rendering.

[0043] In this embodiment, in the view cone The distance extended inward from the four inner boundaries Construct a 3D attenuation transition zone; calculate the orthogonal projection distance from the point to be rendered to the side boundary of the view frustum. Define nonlinear mixed weighting factors :when , ;when hour, The rest are 0.

[0044] In this embodiment, a preset trigger threshold is used. Generated based on offline calibration of normal production line samples; specifically: Based on the normal sample set, a sufficient number of defect-free normal samples are extracted, their global anomaly probabilities are calculated, and they are sorted in descending order. According to the maximum tolerable false trigger rate set by the production line, the probability value of the corresponding quantile is directly extracted as... .

[0045] In this embodiment, generating the three-dimensional view frustum specifically involves: Spatial mapping preparation: Pre-calibrating the intrinsic parameter matrix of industrial field cameras. and extrinsic parameter matrix Existing conventional calibration techniques are employed, such as Zhang's calibration method based on a checkerboard calibration board. By acquiring calibration board images from different perspectives, the distortion coefficients and intrinsic and extrinsic parameter matrices of the camera are accurately calculated. The obtained matrix parameters provide a physical space coordinate mapping reference for subsequent two-dimensional view frustum three-dimensional back projection.

[0046] View frustum boundary line calculation: obtaining pixel coordinates of the four corner points , Using the camera's intrinsic parameter matrix and extrinsic parameter matrix Calculate the 3D optical center coordinates of the virtual camera. ,in For the camera's rotation matrix, The translation vector of the camera; the direction vectors of the four boundary rays of the view frustum in the world coordinate system. The solution is as follows: ; 3D view frustum construction: with optical center With the vertex as the top and the four boundary rays as the side edges, a closed detection cone shaped like a truncated cone is constructed in three-dimensional physical space. .

[0047] In this embodiment, spatial clipping of the Mesh model specifically involves: organizing the offline Mesh model using hierarchical bounding boxes (BVH); calculating the relationship between each bounding box and the view frustum. Intersection relationships, eliminating those located at the view cone External nodes only load local mesh faces that pass the intersection test into the GPU memory queue for high-precision rasterization rendering, achieving spatial-level dimensionality reduction in memory allocation.

[0048] In this embodiment, during the fragment coloring stage, a rigorous depth test (Z-Testing) is performed, specifically: Construct a two-dimensional global Z-Buffer that corresponds one-to-one with each screen pixel to record the current screen pixel. Minimum physical depth value from the optical center of the virtual camera When a local mesh rasterized fragment or a 3DGS integral fragment attempts to write screen coordinates... At that time, it will attempt to write screen coordinates. The fragment is used as a new fragment, and the depth value of the new fragment is extracted. ; calculate The Boolean value indicates that the new fragment is closer to the virtual camera in physical 3D space and passes the depth test. The color buffer is overwritten with the color of the new fragment. If the value is false, the new fragment is physically occluded and fails the depth test. The new fragment will be discarded directly.

[0049] Step S5: Perform consistency verification on the hybrid rendering baseline image to filter out false positive interference, thereby obtaining accurate detection results that include real physical geometric defects; specifically: Phase correlation or feature matching is introduced to perform vibration-resistant subpixel registration between the actual acquired image and the hybrid rendering reference image: geometric edge gradient structure features of the two images are extracted respectively. Luminous and color distribution photometric characteristics Through weighted formula Generate a comprehensive residual heatmap, where: For geometric structure feature weights, For photometric feature weights, The calculated composite residual value; Residual back projection reconstruction: Obtain the pixel coordinates greater than the judgment threshold in the comprehensive residual heatmap. Extract its physical depth value from Z-Buffer ; Using a camera matrix, pixel coordinates are inversely projected to absolute three-dimensional world coordinates. : ; Cross-plane ray intersection verification: Requesting the viewpoint Calibration pose of adjacent view cameras ;Will Reprojection to viewpoint If residuals exceeding the judgment threshold also exist, then construct the optical center from the main viewpoint to the... rays and perspective Light Heart rays ; Calculate the length of the shortest common perpendicular segment between two rays. ; Artifact removal: If , As a tolerance threshold, it proves that multi-view anomalous residuals truly intersect at the same three-dimensional surface point, confirming it as a real physical geometric defect; if If the score of the reprojected position in the comprehensive residual heat map does not exceed the maximum tolerable false trigger rate set by the production line, it is judged as anisotropic reflection and forcibly removed. The defect is classified by combining the area of ​​the connected domain of the actual physical geometric defect after verification with the peak value of the residual, and an inspection report with absolute three-dimensional coordinates is output.

[0050] The method provided in this embodiment solves the problem of the trade-off between accuracy and efficiency in the prior art, achieving high-precision, real-time industrial surface defect detection. This is specifically reflected in the following three dimensions: (1) Achieving a balance between high-precision 3D rendering and high real-time detection: In view of the contradiction that high-precision Mesh rendering consumes a lot of computing power while 3DGS is prone to losing small geometric defects, this invention aims to establish a dynamic rendering scheduling mechanism guided by lightweight 2D visual attention (ROI), which drives efficient allocation of computing power resources based on lightweight pre-inspection results, and ensures lossless super-resolution rendering of micron-level defects while maintaining a very high frame rate of global rendering.

[0051] (2) Establish a high-precision cross-dimensional spatial mapping and data collaboration mechanism: In view of the problems of lack of unified underlying physical coordinates and blind scheduling in the existing hybrid detection scheme, this invention aims to propose the "offline homogeneous dual model reconstruction" and "online view cone back projection" mechanism to construct a rigorous coordinate mapping chain from 2D pixels to 3D space and back projection to 2D reference map, so as to achieve seamless deep integration of different data structures.

[0052] (3) Improve the robustness of three-dimensional defect detection in complex industrial environments: In view of the weakness of traditional “pixel-level residual” being easily affected by mechanical micro-oscillation and metal specular / reflection interference, resulting in false positives, this invention aims to introduce the “fusion of structural and photometric multi-dimensional features” and the “three-dimensional spatial consistency verification of multi-view ray intersection” mechanism to greatly improve the robustness of the detection system in real complex industrial environments.

[0053] The method provided by this invention can be widely applied to the following industrial testing scenarios: (1) Automobile manufacturing industry: Surface defect detection of precision parts such as engine blocks and gearbox housings; (2) Consumer electronics: CNC machining defect detection for products such as mobile phone frames and laptop casings; (3) Aerospace: Quality inspection of key components such as aircraft skin and turbine blades; (4) Precision casting: Automatic detection of surface quality of die castings, forgings and other products.

[0054] To further illustrate the feasibility and superiority of this solution in a real industrial environment, the following analysis compares and analyzes the mechanism and theoretical effectiveness of this invention with existing mainstream solutions by setting typical engineering parameters, using specific industrial application cases.

[0055] Application scenario: 3D defect detection of precision aluminum alloy die-cast parts for new energy vehicles; Core aluminum alloy die-cast parts for new energy vehicles, such as motor housings and steering gear housings, not only have complex curved surfaces, reinforcing ribs, and deep holes, but their CNC-machined mounting surfaces also have extremely strong metallic mirror-like high-gloss reflections. Traditional 2D vision is easily misled by cutting fluid residue or specular drift, resulting in a large number of false alarms, while conventional 3D inspection faces the problem of excessive computing power for single-piece scanning and rendering, which cannot meet the needs of high-speed automated production lines. Implementation parameter settings: Set the total number of faces in the global high-precision mesh model of the die-casting part. (15 million faces); Total number of Gaussian spheres in the global 3DGS model Assuming the resolution of the online inspection camera is 4K, based on the prior knowledge of the actual automated pipeline, the area ratio of the two-dimensional bounding boxes (ROIs) of suspected defects in a single image is... (i.e., not exceeding 5% of the total area of ​​the image), and the triggering probability of real physical defects in continuous detection. .

[0056] The expected computational cost per frame of the system is: ; Simplifying, we get: ; in: This is the defect trigger rate, which is extremely low in actual production. This represents the area percentage of a local ROI. This indicates the computational cost per frame using the hybrid method of this invention. The computational complexity of rendering using the 3DGS method. This represents the computational complexity of rendering using the Mesh method. This mechanism is for It has achieved significant simplification.

[0057] Referring to NVIDIA's official 3DGS renderer benchmark data and Puget Systems' Metashape hardware recommendations, approximately 8GB of VRAM is required when testing a complex indoor scene (15.3M Particles Green House) using the 3DGS pipeline. When building a professional mesh rendering workstation, at least 24GB of VRAM is typically required as a safe baseline for handling high-resolution textures and dense meshes.

[0058] Taking the defect detection of precision aluminum alloy die-cast parts, one of the application scenarios of this invention, as an example, the global mesh's memory configuration consists of: tens of millions of vertex / normal data + 4K-level PBR multi-channel material maps + multiple layers of G-Buffer required for the deferred rendering pipeline. According to industry benchmarks, the peak memory usage for this global high-fidelity rasterization scene... A conservative estimate is around 24GB. The memory structure of 3DGS primarily consists of Gaussian spheres. In this invention's architecture, since large scene structures are not required, its global base rendering memory can be estimated. Around 4GB.

[0059] When using the dynamic collaborative scheme of this invention, the maximum peak memory performance is... = Base 3DGS memory + partial Mesh memory. Since the ROI area of ​​tiny defects in actual industrial applications (such as pinholes and scratches) is extremely small, a typical ROI area percentage is set. ,but Conservatively estimated peak memory reduction rate = .

[0060] Referring to the rendering efficiency analysis given in the paper "KERBL B, KOPANAS G, LEIMKÜHLER T, et al. 3D Gaussiansplatting for real-time radiance field rendering[J]. ACM Transactions on Graphics, 2023, 42(4): 1-14.", the rendering rate of 3DGS can reach 30 frames per second ( ). The rendering time complexity of Mesh is linearly positively correlated with the number of polygons; the rendering rate for tens of millions of polygons is approximately 2-3 frames per second. As the previous complexity analysis shows, when using the dynamic collaborative scheme of this invention, the defect triggering rate and the proportion of local ROI area are extremely low in actual production. Therefore, the expected time consumption per frame is very high. Conservative calculation speedup ratio = times.

[0061] The above analysis shows that, compared to traditional global mesh rendering, this invention reduces GPU memory peak speed by more than 80% while ensuring high precision, and achieves an inference speedup of 5. More than 10 times faster, effectively meeting the high real-time and high-precision requirements of real-world industrial environments.

[0062] The theoretical performance and technical mechanism of the method of this invention are quantitatively compared with those of traditional single-element detection methods. The comparison results are as follows: ①Method A (Traditional High-Precision Mesh Global Detection Method): Operating mechanism: For each frame of 4K image, a Mesh model with tens of millions of polygons is globally loaded for rasterization rendering, and pixel-level residual judgment is adopted.

[0063] Performance analysis: Single-frame inference time ≈ 350~450 ms, which cannot meet high-frequency requirements and encounters the sub-pixel rasterization bottleneck; peak memory usage ≈ 24.0 GB, which is extremely high, and polygon and multi-channel PBR materials are fully loaded; traditional high-precision Mesh global detection results have high physical fidelity, and the architecture is naturally sensitive to small defects; however, it is severely affected by metallic highlights and wandering reflections.

[0064] ②Method B (Traditional Pure 3DGS Real-Time Detection Method): Operating mechanism: 3DGS is called globally for rendering of each frame, and defects are determined by pixel-level residuals.

[0065] Performance Analysis: Single-frame inference time ≈30 ms, with extremely high frame rate and pure computational shader parallel rendering; peak memory usage ≈4.0 GB, low power consumption, no complex topology mesh, easily meeting real-time requirements. However, due to the volume integral and low-frequency approximation characteristics of Gaussian point clouds, high-frequency geometric features of micron-level sand holes, burrs, or CNC machining marks on the surface of die-cast parts are smoothed and filtered out, leading to a surge in the missed detection rate of minor structural damage, and it is also susceptible to environmental noise interference, with poor edge perception capabilities.

[0066] ③Method C (Dynamic Collaboration and Three-Dimensional Verification Detection Method of the Invention): Computing power scheduling optimization analysis: This system spends 98% of its time (i.e. Normal frames only run pure 3DGS rendering; on-demand loading accounts for only 5% of the area with a 2% probability (when the defect is triggered). The local mesh of ) . Its expected number of rasterized patches per frame drops sharply from 15,000,000 to The single-frame inference time is approximately 30 ms, meeting the requirements of high-frequency operation and utilizing industrial priors to reduce time complexity; the peak memory usage is approximately 4.7 GB, a reduction of over 70% in peak memory usage, loading only ROI physical slices; it can wake up local meshes on demand, comparable to the accuracy of global rasterization; 3D ray intersection completely filters out 2D view artifacts, effectively reducing false alarms.

[0067] The method of this invention, through view frustum clipping and dynamic scheduling, can theoretically reduce the computational load of the graphics pipeline's mesh rasterization and the dynamic loading of video memory by at least 80% compared to global mesh rendering, achieving the ultimate "drip irrigation" of computing power. Thus, while maintaining micron-level detection accuracy, it achieves a real-time inference frame rate comparable to pure 3DGS solutions.

[0068] Anti-reflective robustness analysis: Traditional methods would directly identify gloss interference on machined aluminum alloy surfaces as defects due to two-dimensional pixel differences. This invention's unique three-dimensional ray intersection verification mechanism back-projects suspected two-dimensional residuals back to three-dimensional world coordinates. Because the metallic speckle moves with the camera's viewpoint, its multi-view rays in physical space (… Therefore, these elements will inevitably fail to converge. Thus, this invention, based on the principles of physical multi-view geometry, theoretically eliminates false positives caused by the wandering of light spots from a single viewpoint, completely overcoming the engineering bottleneck in the 3D inspection of high-gloss metal parts.

[0069] This method maintains near-full-mesh rendering accuracy while significantly reducing computational power consumption and inference time. Simultaneously, the introduced multi-view 3D consistency verification mechanism completely filters out false alarms caused by complex reflections in industrial automation inspection, achieving a dual breakthrough in efficiency and accuracy. This provides a new 3D vision technology solution for industrial automation inspection that offers high real-time performance, high precision, and high robustness.

[0070] Example 2: This embodiment provides an industrial surface defect detection system based on dynamic collaborative rendering, which is used to implement the dynamic collaborative rendering industrial surface defect detection method as described in Embodiment 1. It includes an offline homogeneous reconstruction module and an online dynamic collaborative detection module. The offline homogeneous reconstruction module is used to collect multi-view point cloud data and high-resolution color image sequences of defect-free standard parts, construct a high-precision global dense point cloud in the absolute world coordinate system, and generate Mesh model and 3DGS model homogeneously in the world coordinate system based on the global dense point cloud. The online dynamic collaborative detection module is used to perform dynamic collaborative detection on the online acquired images to be detected based on the Mesh model and the 3DGS model, and output the detection results.

[0071] In this embodiment, the offline homogeneous reconstruction module includes a multimodal data acquisition unit, an intensity-geometric fusion registration unit, and a model homogeneous generation unit. The multimodal data acquisition unit is used to acquire multi-view point cloud data and high-resolution color image sequences of defect-free standard parts. The intensity-geometric fusion registration unit is used to register multi-view point clouds and construct a high-precision global dense point cloud in the absolute world coordinate system. The model homogeneous generation unit is used to generate Mesh models and 3DGS models homogeneously in the world coordinate system based on the global dense point cloud.

[0072] The online dynamic collaborative detection module includes an online image acquisition and pose calibration unit, a global anomaly probability assessment and ROI extraction unit, a pure 3DGS global fast rendering unit, a view frustum back projection and local mesh on-demand loading unit, a Z-Buffer feathering hybrid rendering unit, a multi-dimensional feature residual analysis unit, a multi-view ray intersection verification unit, and an output three-dimensional defect coordinates and level unit. The online image acquisition and pose calibration unit is used to realize online image acquisition and camera pose calibration. The global anomaly probability assessment and ROI extraction unit is used to perform ROI extraction and global anomaly probability assessment. The pure 3DGS global fast rendering unit is used to quickly render a complete, defect-free 2D reference image using a 3DGS model to complete macroscopic verification. The view frustum backprojection and local mesh on-demand loading unit is used to generate a 3D view frustum and load the local mesh patches intersecting with the 3D view frustum into video memory for high-precision rasterization rendering. The Z-Buffer feathering hybrid rendering unit is used to force the local mesh and the background 3DGS to share the depth buffer Z-Buffer under a unified virtual camera view in heterogeneous hybrid rendering. Through depth testing, the depth values ​​of the two models' rendered fragments to the camera optical center are compared pixel by pixel, and fragments closer to the virtual camera are retained to achieve correct geometric occlusion at the physical level; at the same time, a nonlinear hybrid weighting factor is introduced at the 3D boundary of the view frustum. Through formula Alpha blending is performed to output a seamlessly stitched, high-precision hybrid rendering baseline image. The multi-dimensional feature residual analysis unit is used to perform vibration-resistant sub-pixel registration between the actual acquired image and the hybrid rendering baseline image, and generate a comprehensive residual heatmap. The multi-view ray intersection verification unit is used to distinguish between real geometric defects and artifacts caused by industrial reflections / mechanical micro-vibrations. The output three-dimensional defect coordinates and level unit is used to output the final detection results.

[0073] Example 3: This embodiment provides an electronic device corresponding to the above embodiments. The electronic device can be a processing device for a client, such as a mobile phone, laptop, tablet computer, desktop computer, etc., to execute the method of embodiment 1 above.

[0074] The electronic device of this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method of Embodiment 1 described above.

[0075] In some implementations, the memory may be high-speed random access memory (RAM), and may also include nonvolatile memory, such as at least one disk storage.

[0076] In other implementations, the processor can be any type of general-purpose processor, such as a central processing unit (CPU) or a digital signal processor (DSP), and there is no limitation here.

[0077] This embodiment also provides a readable storage medium corresponding to the above embodiments, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, they implement the steps of the method in Embodiment 1 above.

[0078] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0079] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0080] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] The above description is only a preferred embodiment of the present invention and does not limit the scope of protection of the present invention. All equivalent structural transformations made under the inventive concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the scope of protection of the present invention.

Claims

1. A dynamic collaborative rendering method for detecting industrial surface defects, characterized in that, Includes the following steps: Step S1: Obtain multi-view point cloud data and high-resolution color image sequences of defect-free standard parts; construct a high-precision global dense point cloud in the absolute world coordinate system; Step S2: Based on the global dense point cloud, generate Mesh model and 3DGS model from the same source in the world coordinate system; Step S3: Acquire two-dimensional images of the workpiece to be inspected online, perform preliminary screening of two-dimensional anomalies based on the two-dimensional images, locate suspected defect areas, and extract the two-dimensional bounding boxes of the suspected defect areas. ; Step S4: Calculate the global defect probability Based on global defect probability Dynamically schedule the Mesh model and / or 3DGS model to render the image to be detected acquired online in the current frame, and output a hybrid rendering baseline map; Step S5: Perform consistency verification on the hybrid rendering baseline map and obtain the final detection result.

2. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 1, characterized in that, In step S2, based on the global dense point cloud, the Poisson reconstruction algorithm is used to extract isosurfaces and generate a Mesh model that retains high-frequency geometric details; the three-dimensional spatial coordinates of the global dense point cloud are extracted as the initial center position of the 3DGS Gaussian sphere, and the 3DGS model is generated by combining it with a high-resolution color image sequence.

3. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 1, characterized in that, Step S3 includes: Step S3.1: Acquire a two-dimensional image of the workpiece to be inspected online, input the two-dimensional image of the workpiece to be inspected into a pre-trained multi-scale feature distribution modeling network, and evaluate the anomaly score of each spatial location in the feature map that deviates from the normal distribution. Obtain a global anomaly score heatmap; Step S3.2: Perform spatial smoothing on the global anomaly score heatmap. Set an empirical threshold to perform binary truncation on the smoothed global anomaly score heatmap. Use morphological filtering and connected component labeling algorithms to find abnormal connected regions exceeding the empirical threshold as suspected defect regions. Calculate and extract the minimum bounding rectangle of the suspected defect region and output the two-dimensional bounding box. .

4. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 3, characterized in that, Step S3.1 includes: ① Establish normal feature distribution parameters during the offline phase: Collect a sufficient number of defect-free standard part images containing normal operating condition fluctuations to construct a normal sample set; input all normal samples into a pre-trained multi-scale feature distribution modeling network identical to that used in the online stage to obtain feature maps; for each spatial pixel coordinate on the feature map... Independently count the feature vectors of all normal samples at this position, and calculate the mean vector of their multivariate Gaussian distribution. and sample covariance matrix And calculate the inverse of the covariance matrix. ; ② Input the online-acquired image to be detected into a pre-trained multi-scale feature distribution modeling network to extract the spatial pixel coordinates. high-dimensional feature vectors at the location ; ③ Call the normal feature distribution parameters established in the offline phase, and calculate the Mahalanobis distance deviating from the normal distribution as the anomaly score using the following formula. : ； in: This represents the transpose of a matrix.

5. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 4, characterized in that, In step S4, the global defect probability is calculated. Specifically: After spatial smoothing the global anomaly score heatmap, the global maximum anomaly score is extracted. ; The global defect probability is calculated using the following formula. : ； in: This represents the preset sensitivity coefficient, which is derived by working backward from the production line's quality control requirements. This represents the extracted global maximum anomaly score; Represented by natural constant An exponential function with base 0.

6. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 5, characterized in that, In step S4, based on the global defect probability With preset trigger threshold Dynamically schedule the Mesh model and / or 3DGS model to render the image to be detected acquired online in the current frame, and output a hybrid rendering baseline map; specifically: when When the image to be detected acquired online in the current frame is determined to be normal, only the 3DGS model is called to quickly render and generate a complete defect-free two-dimensional reference image, i.e., a global baseline view, under the current virtual camera view, to complete the macroscopic verification. when At that time, the Mesh model and the 3DGS model are used for heterogeneous hybrid rendering. Specifically: The two-dimensional bounding box extracted in step S3 Back-projected onto the 3D world coordinate system, generating a 3D view frustum; Spatial clipping is performed on the Mesh model, and the local Mesh mesh areas intersecting with the 3D view frustum are rendered using high-precision rasterization; the background area outside the 3D view frustum continues to be rendered using the 3DGS model; Z-Buffer Feathering: In heterogeneous hybrid rendering, local meshes and background 3DGS are forced to share the Z-Buffer depth buffer under a unified virtual camera view. Through depth testing, the depth values ​​of the rendered fragments from the camera optical center of the two models are compared pixel by pixel, retaining fragments closer to the virtual camera to achieve correct geometric occlusion at the physical level. At the same time, a non-linear blending weight factor is introduced at the 3D boundary of the view frustum. Through formula Perform alpha blending to output a seamlessly stitched, high-precision blending rendering baseline image; where: This represents the final pixel color value after blending and rendering. This represents the color contribution value of local mesh rasterization rendering. This represents the color contribution value of the background 3DGS rendering.

7. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 6, characterized in that, The specific process of generating a 3D view frustum is as follows: Spatial mapping preparation: Pre-calibrating the intrinsic parameter matrix of industrial field cameras. and extrinsic parameter matrix ; View frustum boundary line calculation: obtaining pixel coordinates of the four corner points , Using the camera's intrinsic parameter matrix and extrinsic parameter matrix Calculate the 3D optical center coordinates of the virtual camera. ,in For the camera's rotation matrix, The translation vector of the camera; the direction vectors of the four boundary rays of the view frustum in the world coordinate system. The solution is as follows: ； 3D view frustum construction: with optical center With the vertex as the top and the four boundary rays as the side edges, a closed detection cone shaped like a truncated cone is constructed in three-dimensional physical space. .

8. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 7, characterized in that, Spatial clipping of the mesh model specifically involves: organizing the offline mesh model using hierarchical bounding boxes (BVH); calculating the relationship between each bounding box and the view frustum. Intersection relationships, eliminating those located at the view cone External nodes only load local mesh faces that pass the intersection test into the GPU memory rendering queue for high-precision rasterization rendering.

9. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 8, characterized in that, Preset trigger threshold Generated based on offline calibration of normal production line samples; specifically: Based on the normal sample set, a sufficient number of defect-free normal samples are extracted, their global anomaly probabilities are calculated, and they are sorted in descending order. According to the maximum tolerable false trigger rate set by the production line, the probability value of the corresponding quantile is directly extracted as... .

10. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 9, characterized in that, Depth testing specifically involves: Construct a two-dimensional global Z-Buffer that corresponds one-to-one with each screen pixel to record the current screen pixel. Minimum physical depth value from the optical center of the virtual camera When a local mesh rasterized fragment or a 3DGS integral fragment attempts to write screen coordinates... At that time, it will attempt to write screen coordinates. The fragment is used as a new fragment, and the depth value of the new fragment is extracted. ; calculate The Boolean value indicates that the new fragment is closer to the virtual camera in physical 3D space and passes the depth test. The color buffer is overwritten with the color of the new fragment. If the value is false, the new fragment is physically occluded and fails the depth test. The new fragment will be discarded directly.

11. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 10, characterized in that, In the cone The distance extended inward from the four inner boundaries Construct a three-dimensional attenuation transition band; Calculate the orthogonal projection distance from the point to be rendered to the side boundary of the view frustum. Define nonlinear mixed weighting factors :when , ;when hour, The rest are 0.

12. The dynamic collaborative rendering method for detecting industrial surface defects according to claim 6, characterized in that, Step S5 is as follows: Anti-vibration subpixel registration was performed between the actual acquired image and the hybrid rendering baseline image: geometric edge gradient structure features of the two images were extracted respectively. Luminous and color distribution photometric characteristics Through weighted formula Generate a comprehensive residual heatmap, where: For geometric structure feature weights, For photometric feature weights, The calculated composite residual value; Residual back projection reconstruction: Obtain the pixel coordinates greater than the judgment threshold in the comprehensive residual heatmap. Extract its physical depth value from Z-Buffer ; Using a camera matrix, pixel coordinates are inversely projected to absolute three-dimensional world coordinates. : ； Cross-plane ray intersection verification: Requesting the viewpoint Calibration pose of adjacent view cameras ;Will Reprojection to viewpoint If residuals exceeding the judgment threshold also exist, then construct the optical center from the main viewpoint to the... rays and perspective Light Heart rays ; Calculate the length of the shortest common perpendicular segment between two rays. ; Artifact removal: If , As a tolerance threshold, it proves that multi-view anomalous residuals truly intersect at the same three-dimensional surface point, confirming it as a real physical geometric defect; if If the score of the reprojected position in the comprehensive residual heat map does not exceed the maximum tolerable false trigger rate set by the production line, it is judged as anisotropic reflection and forcibly removed. The defect is classified by combining the area of ​​the connected domain of the actual physical geometric defect after verification with the peak value of the residual, and an inspection report with absolute three-dimensional coordinates is output.

13. An industrial surface defect detection system based on dynamic collaborative rendering, used to implement the dynamic collaborative rendering industrial surface defect detection method as described in any one of claims 1-12, characterized in that, It includes an offline homogeneous reconstruction module and an online dynamic collaborative detection module. The offline homogeneous reconstruction module is used to collect multi-view point cloud data and high-resolution color image sequences of defect-free standard parts, construct a high-precision global dense point cloud in the absolute world coordinate system, and generate Mesh model and 3DGS model in the world coordinate system based on the global dense point cloud. The online dynamic collaborative detection module is used to perform dynamic collaborative detection on the online acquired images to be detected based on the Mesh model and the 3DGS model, and output the detection results.

14. The industrial surface defect detection system based on dynamic collaborative rendering according to claim 13, characterized in that, The offline homogeneous reconstruction module includes a multimodal data acquisition unit, an intensity-geometric fusion registration unit, and a model homogeneous generation unit; the online dynamic collaborative detection module includes an online image acquisition and pose calibration unit, a global anomaly probability assessment and ROI extraction unit, a pure 3DGS global fast rendering unit, a view frustum backprojection and local mesh on-demand loading unit, a Z-Buffer feathering hybrid rendering unit, a multidimensional feature residual analysis unit, a multi-view ray intersection verification unit, and a unit for outputting three-dimensional defect coordinates and levels.

15. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program; the processor executes the computer program to implement the dynamic collaborative rendering industrial surface defect detection method as described in any one of claims 1-12.

16. A readable storage medium, characterized in that, The readable storage medium stores a computer program, and the processor executes the computer program to implement the dynamic collaborative rendering industrial surface defect detection method as described in any one of claims 1-12.

Images