Lightweight network-based industrial image dual-view reconstruction method and system

Abstract

This invention discloses a method and system for dual-view reconstruction of industrial images based on a lightweight network. The method includes: simultaneously acquiring dual-view transmission images using dual X-ray sources, constructing a dataset, and performing preprocessing optimization; constructing an adaptive heterogeneous reconstruction network with a dual-branch architecture to achieve global structure perception, local detail compensation, and cross-view feature fusion; building an epipolar cross-attention framework based on epipolar geometry and multi-level attention to complete dual-view feature registration; and relying on a fast Gaussian prediction network to achieve 3D Gaussian primitive prediction and frequency-adaptive rendering. This invention employs a three-level processing mechanism of the epipolar cross-attention framework, combined with a dual-frequency activation function, fast Gaussian prediction, and an adaptive heterogeneous reconstruction network to achieve end-to-end 3D Gaussian primitive prediction. By constructing a lightweight end-to-end reconstruction architecture, it solves the problems of high reconstruction latency and inability to deploy on mobile devices in industrial scenarios, significantly improving overall real-time performance and applicability.

Description

Technical Field

[0001] This invention relates to a method for dual-view reconstruction of industrial images based on lightweight networks, and also to a corresponding dual-view reconstruction system for industrial images, belonging to the field of image reconstruction technology. Background Technology

[0002] Image reconstruction is a core technology for accurate analysis and defect detection in industrial image processing. Dual-view reconstruction recovers three-dimensional information from two images with different perspectives, providing richer geometric features than single-view reconstruction. It has been widely used in scenarios such as surface defect detection, texture analysis, and reconstruction of complex industrial parts. However, existing dual-view reconstruction technologies generally suffer from high computational complexity and insufficient real-time performance. The limitations of traditional methods are particularly prominent under the high-speed processing requirements of industrial sites. Traditional reconstruction relies on geometric algorithms such as stereo matching and image registration. Although it can achieve basic alignment and reconstruction, it has high requirements for hardware calibration, consumes a lot of resources, and has a slow processing speed. It is also susceptible to dynamic environmental factors such as uneven lighting, perspective deviation, and object deformation, resulting in high errors and excessive processing time, making it difficult to meet the needs of real-time industrial inspection.

[0003] End-to-end dual-view reconstruction technology based on deep learning can automatically complete image alignment and 3D reconstruction through a unified network. It can learn robust features from massive amounts of data, effectively improving accuracy and efficiency, and has become the mainstream of current research. However, this technology still faces several key challenges: First, due to the large differences in shooting angle, field of view, and lighting, coupled with factors such as deformation, occlusion, and noise, image registration and alignment of dual-view images are extremely difficult. Second, deep learning models are large in size and computationally intensive, resulting in significant latency bottlenecks in real-time industrial scenarios. Third, efficient fusion of complementary information from both perspectives is difficult, easily leading to information loss or redundancy, affecting reconstruction reliability. Fourth, the large scale of high-resolution industrial image data makes existing models prone to computational bottlenecks, resulting in low training and inference efficiency. Therefore, achieving lightweight network design, efficient registration, and rapid information fusion while ensuring high accuracy has become crucial for promoting the large-scale implementation of end-to-end dual-view reconstruction technology in the industrial field.

[0004] Chinese invention patent CN113538243B discloses a super-resolution image reconstruction method based on a combination of multi-parallax attention modules. This technical solution improves existing super-resolution image network models in a more reasonable and flexible way by constructing a stereo image super-resolution network model based on a multi-parallax module combination structure and an image smoothing loss function. It not only effectively improves the quality of super-resolution imaging but also has better anti-interference capabilities and higher super-resolution performance compared to existing super-resolution image reconstruction techniques, providing richer detail information for further processing of the super-resolution reconstructed image. However, the above technical solution only uses multi-parallax attention modules to achieve super-resolution reconstruction and does not involve 3D reconstruction and real-time deployment on mobile devices, failing to solve the problems of high reconstruction latency and inability to deploy on mobile devices in industrial scenarios. Summary of the Invention

[0005] The primary technical problem to be solved by this invention is to provide a method for dual-view reconstruction of industrial images based on lightweight networks.

[0006] Another technical problem to be solved by the present invention is to provide an industrial image dual-view reconstruction system based on a lightweight network.

[0007] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution: According to a first aspect of the present invention, a method for dual-view reconstruction of industrial images based on a lightweight network is provided, comprising the following steps: S1: Simultaneously acquire dual-view transmission images of the target object using dual X-ray sources, obtain the projection matrix corresponding to each image, and construct a dataset for sparse view 3D reconstruction based on the dual-view transmission images and projection matrices, and perform preprocessing on the dataset. S2: Based on the dual-frequency feature distribution law of industrial transmission images, an adaptive heterogeneous reconstruction network is constructed. Through a dual-branch serial architecture, global structure perception and local detail feature extraction of dual-view images are realized, and cross-view feature fusion and scale-invariant feature encoding are completed. S3: Based on epipolar geometric constraints and multi-level attention mechanisms, an epipolar cross-attention framework is constructed to perform global overview perception, pixel-level center alignment and epipolar constraint matching on the dual-view features output by the adaptive heterogeneous reconstruction network, and feature registration is achieved through multi-path attention coupling. S4: Based on the epipolar cross-attention framework for registering key points and the dual-frequency features of the adaptive heterogeneous reconstruction network, a fast Gaussian prediction network is constructed to achieve end-to-end 3D Gaussian meta-parameter prediction. The Gaussian meta-space layout is optimized by combining the dual-frequency feature distribution strategy, while a lightweight rendering mechanism and inference efficiency optimization are introduced.

[0008] Preferably, in step S1, two X-ray sources are simultaneously exposed at a preset angle to generate a first industrial transmission image acquired from a dual-view perspective. and the second industrial transmission image acquired from dual perspectives And record the projection matrix of the ray source. The corresponding formula is:

[0009] in, For the first A standardized representation of a 3D reconstructed scene; For the first Image tensors from a single perspective; It is a scaling factor, and , It is a positive real number; For the first Projection matrix of each ray source; This is an index variable for the scene and viewpoint, with values ​​1, 2, ... ; This represents the total number of viewpoints in the scene.

[0010] Preferably, in step S2, the first branch of the serial dual-branch structure is a coarse structure perception branch, which is used to extract low-frequency macroscopic features of the target and achieve feature dimensionality reduction and global perception through continuous lightweight downsampling layers, deep convolutional layers and bottleneck structures; the second branch is a local detail compensation branch, which is used to enhance the ability to express local details and make up for the high-frequency information lost by the coarse branch in the dimensionality reduction process.

[0011] In a preferred manner, to automatically separate and enhance high-frequency information and low-frequency macroscopic features, a dual-frequency activation function is introduced, with the corresponding formula as follows: ,

[0012] in, The interaction weights of high-frequency and low-frequency features between dual-view images; These are the learnable initial weight coefficients; For depthwise separable convolution; For Fast Fourier Transform; This is the first industrial transmission image acquired from a dual-view perspective. This is the second industrial transmission image acquired from a dual-view perspective. This is the output value of the dual-frequency activation function; ω represents the input feature values ​​of the activation function; ω represents the basic weight coefficients. The smoothing coefficient is a learnable factor. It is a natural constant.

[0013] Preferably, in step S3, the epipolar cross attention framework is executed according to a three-level mechanism of "overview-alignment-compensation", which sequentially generates global perception markers, pixel center markers and epipolar constraint markers, and registers key point pairs with the R-SoftMax output through depth convolution.

[0014] Preferably, in step S4, the initial dimension is determined through a star-shaped operation mechanism. Adjusted to This is used to improve feature interaction efficiency and reduce rendering time; among which Amplification factor.

[0015] Preferably, in step S4, the three-dimensional Gaussian element is represented as:

[0016] in, For the first A three-dimensional Gaussian element; For the first The three-dimensional center coordinates of each Gaussian element; For the first The transparency parameter of each Gaussian element has a value range of [0,1]. For the first The shape covariance matrix of Gaussian elements; For the first Appearance feature parameters of each high-resolution element; This is an index variable for Gaussian elements, taking values ​​1, 2, ... ; This represents the total number of Gaussian elements in the 3D scene.

[0017] Preferably, the three-dimensional Gaussian elements are densely distributed in the high-frequency region and sparsely distributed in the low-frequency region, and the granularity is adaptively adjusted according to the frequency during rendering.

[0018] According to a second aspect of the present invention, an industrial image dual-view reconstruction system based on a lightweight network is provided, comprising a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, which, when executed by the processor, enables the processor to implement the above-described industrial image dual-view reconstruction method.

[0019] Compared with existing technologies, this invention achieves the following beneficial technical effects: First, by introducing a three-level epipolar cross-attention framework of "overview-alignment-compensation" that simulates biological vision, it effectively solves the registration difficulties caused by factors such as viewpoint differences, target aliasing, and deformation occlusion in dual-view images, significantly improving the accuracy and robustness of 3D reconstruction. Second, by designing an adaptive point extraction strategy based on frequency features, combined with a dual-frequency activation function and a fast Gaussian prediction network, it achieves effective separation and differentiated processing of high-frequency details and low-frequency contours, significantly improving rendering efficiency while suppressing redundant computation, enabling the model to run in real time on mobile terminals and embedded devices. Finally, by adopting an adaptive heterogeneous reconstruction network based on depthwise separable convolution and a dual-branch serial architecture, it significantly reduces the number of model parameters and computational complexity while ensuring reconstruction quality. Compared with traditional reconstruction methods, it significantly reduces hardware deployment costs and system power consumption, providing a practical and feasible technical path for the large-scale implementation of dual-view transmission image 3D reconstruction technology in resource-constrained scenarios such as security inspection, industrial inspection, and medical imaging. Attached Figure Description

[0020] Figure 1 This is a flowchart of a method for dual-view reconstruction of industrial images based on lightweight networks, as shown in the first embodiment of the present invention. Figure 2 This is a schematic diagram of the adaptive heterogeneous reconstruction network in the first embodiment of the present invention; Figure 3 This is a schematic diagram of the epipolar cross-attention framework in the first embodiment of the present invention; Figure 4 This is a schematic diagram of the fast Gaussian prediction framework in the first embodiment of the present invention; Figure 5 This is a schematic diagram of the lightweight unit Gaoski element point in the first embodiment of the present invention; Figure 6 This is a schematic diagram of Gaussian rendering of a lightweight unit in the first embodiment of the present invention; Figure 7 This is a schematic diagram of an industrial image dual-view reconstruction system based on a lightweight network, as shown in the second embodiment of the present invention. Detailed Implementation

[0021] The technical content of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0022] First Embodiment like Figure 1 As shown, the first embodiment of the present invention provides a method for dual-view reconstruction of industrial images based on lightweight networks, which includes at least the following steps: S1: Simultaneously acquire dual-view transmission images of the target object using dual X-ray sources, obtain the projection matrix corresponding to each image, and construct a dataset for sparse view 3D reconstruction based on the dual-view transmission images and projection matrices, and perform preprocessing on the dataset. S2: Based on the dual-frequency feature distribution law of industrial transmission images, an adaptive heterogeneous reconstruction network is constructed. Through a dual-branch serial architecture, global structure perception and local detail feature extraction of dual-view images are realized, and cross-view feature fusion and scale-invariant feature encoding are completed. S3: Based on epipolar geometric constraints and multi-level attention mechanisms, an epipolar cross-attention framework is constructed to perform global overview perception, pixel-level center alignment and epipolar constraint matching on the dual-view features output by the adaptive heterogeneous reconstruction network, and feature registration is achieved through multi-path attention coupling. S4: Based on the epipolar cross-attention framework for registering key points and the dual-frequency features of the adaptive heterogeneous reconstruction network, a fast Gaussian prediction network is constructed to achieve end-to-end 3D Gaussian meta-parameter prediction. The Gaussian meta-space layout is optimized by combining the dual-frequency feature distribution strategy, while a lightweight rendering mechanism and inference efficiency optimization are introduced.

[0023] The following is a detailed explanation of the specific steps of the above-mentioned dual-view reconstruction method for industrial images: In step S1, a dual-X-ray source synchronous imaging system deployed in an industrial security inspection line is used to acquire dual-view transmission images of targets with different shapes, materials, and placement states. The system is then calibrated and data is regularized by combining X-ray imaging geometric parameters, projection matrix, and scene scale information. Finally, a high-quality, standardized dataset that meets the requirements of sparse viewpoint 3D reconstruction and can be directly used for network training is constructed. This provides a stable, accurate, and high-coverage data foundation for subsequent lightweight network model training and 3D reconstruction. Throughout the process, the impact of imaging bias, data inconsistency, and insufficient sample diversity on reconstruction accuracy and model generalization ability is eliminated from the source.

[0024] In the specific implementation process, firstly, the hardware system of the dual X-ray source is installed and calibrated according to the actual needs of industrial security inspection and transmission imaging. Two sets of X-ray sources are fixed above the detection channel at preset spatial angles and distances to ensure that both sources can simultaneously irradiate the same target and form a dual-view image with a stable parallax relationship. During the calibration phase, the intrinsic and extrinsic parameters, projection matrix, and distortion coefficients of the X-ray sources are accurately calculated to ensure that the imaging geometry is quantifiable and reproducible. Subsequently, the target to be inspected is placed in various forms, such as random placement, stacking, or individual placement, along with the conveyor belt into the imaging area. The dual X-ray sources synchronously trigger exposure, simultaneously acquiring two transmission images with complementary structural information, denoted as... and Simultaneously, it automatically records the ray source pose and projection matrix corresponding to each group of images. Geometric information such as imaging scale provides necessary prior constraints for subsequent 3D reconstruction. To ensure the dataset covers all real security inspection scenarios, the collected targets include various types such as regular-shaped objects, irregular-shaped objects, metal materials, organic materials, liquid items, stacked packages, and deformed objects, comprehensively simulating target shapes and interference factors that may occur in actual applications. After the original images are acquired, they enter a standardized preprocessing process: First, the images are screened for validity, removing overexposed, underexposed, blurry, incomplete, or excessively angled images, retaining valid images with clear imaging, complete viewpoints, and rich structural information. Then, all images undergo uniform resolution scaling, grayscale normalization, grayscale stretching, and noise reduction. A combination of Gaussian filtering and median filtering is used to remove electronic noise and transmission scattering noise from the images, while mitigating the grayscale differences caused by uneven dose distribution, ensuring consistency in grayscale range and contrast between the two-view images, and improving the stability of subsequent feature extraction and registration. In the data structuring stage, according to the data format requirements for sparse viewpoint 3D reconstruction, each reconstructed scene is represented as a sequence of tuples composed of image tensors and projection matrices, i.e. ,in For the first Image tensors from a single perspective For the corresponding ray source projection matrix, at least two images are required to meet the minimum requirements for sparse reconstruction. Simultaneously, the scale factor for each scene is calculated using Structure-from-Motion (SfM) software. The scene is ultimately represented as:

[0025] in, For the first A standardized representation of a 3D reconstructed scene; For the first Image tensors from a single perspective; It is a scale factor, and , The value is a positive real number to ensure that the 3D reconstruction is carried out in a unified metric space and to avoid distortion of the reconstruction results due to scale inconsistency. For the first Projection matrix of each ray source; This is an index variable for the scene and viewpoint, with values ​​1, 2, ... ; This represents the total number of viewpoints in the scene.

[0026] In the data partitioning and augmentation stage, the completed dataset is divided into training, validation, and test sets in a fixed ratio. The training set is used for learning network parameters, the validation set is used for monitoring model performance and tuning hyperparameters during training, and the test set is used for objective evaluation of the final reconstruction effect.

[0027] Simultaneously, data augmentation operations such as random rotation, translation, scaling, cropping, brightness adjustment, and noise perturbation are performed on the training set to expand the data distribution, improve the model's generalization ability and robustness in complex environments, and prevent the model from overfitting.

[0028] The entire S1 process, from hardware calibration, image acquisition, preprocessing, structured construction to data augmentation, forms a fully automated, standardized, and highly adaptable dataset construction workflow. It can efficiently generate large-scale, high-quality, and highly generalizable dual-view transmission image datasets, providing solid and reliable data support for subsequent lightweight network construction, epipolar cross-attention registration, and end-to-end 3D Gaussian primitive prediction. It is the prerequisite and foundation for achieving high-precision, high-efficiency, and high-stability dual-view 3D reconstruction.

[0029] In step S2, with the goal of achieving real-time deployment on industrial and mobile devices, a lightweight, efficient, and highly expressive adaptive heterogeneous reconstruction network is designed and constructed. It adopts a dual-branch serial architecture of coarse structure perception and local detail compensation, combined with key technologies such as depthwise separable convolution, dual-frequency feature decoupling, and lightweight residual modules. While significantly reducing the number of model parameters and computational complexity, it completes multi-scale and dual-frequency domain feature extraction and fusion of dual-view transmission images. This provides compact, accurate, and high signal-to-noise ratio feature representations for subsequent epipolar cross-attention registration and 3D Gaussian primitive prediction. It solves the industry pain points of traditional deep learning models, such as large size, high computing power requirements, and difficulty in real-time operation, from the network structure level.

[0030] In the specific implementation process, such as Figure 2 As shown, based on the feature distribution patterns of dual-view transmission images and the requirements of the reconstruction task, the overall architecture of the adaptive heterogeneous reconstruction network is determined to be a serial dual-branch structure: The first branch is the coarse structure perception branch, which is mainly responsible for quickly extracting low-frequency macro features such as the global contour, overall shape, and spatial layout of the target. It achieves feature dimensionality reduction and global perception through continuous lightweight downsampling layers, deep convolutional layers and bottleneck structures. It preserves the overall consistency of the three-dimensional structure while minimizing the amount of computation and parameters, and avoids the loss of spatial structure due to excessive compression. The second branch is the local detail compensation branch. It takes the feature map output by the coarse structure perception branch as input and focuses on high-frequency detail information such as target edges, corners, textures, and small structures. It enhances the ability to express local details through dilated convolution, lightweight residual connections, and feature weighted fusion, making up for the high-frequency information lost by the coarse branch during dimensionality reduction. This ensures that the final output features have both global structural integrity and local detail accuracy.

[0031] To further achieve network lightweighting, the adaptive heterogeneous reconstruction network fully adopts depthwise separable convolution to replace the traditional standard convolution. The convolution operation is decomposed into depthwise convolution that processes each channel independently and pointwise convolution that integrates channel information, which greatly reduces the number of parameters and computational cost, while maintaining the feature extraction effect at a basically unchanged level.

[0032] The proposed Bifrequency Activation Function (BiAF) is introduced in the network activation stage to effectively decouple and enhance high-frequency detail features and low-frequency global features, avoiding the suppression or loss of high-information key points during lightweight encoding. The calculation of the Bifrequency Activation Function follows the formula below: ,

[0033] in, The interaction weights of high-frequency and low-frequency features between dual-view images; These are the learnable initial weight coefficients; It is a depthwise separable convolution used for dimensionality reduction and cross-view fusion of frequency domain features; The Fast Fourier Transform is used to transform an image from the spatial domain to the frequency domain, making it easier to separate high-frequency and low-frequency components. This is the first industrial transmission image acquired from a dual-view perspective. This is the second industrial transmission image acquired from a dual-view perspective. This is the output value of the dual-frequency activation function; ω represents the input feature values ​​of the activation function; ω represents the basic weight coefficients. It is a learnable smoothing coefficient used to control the nonlinear response strength of the activation function and adjust the "steepness" of the function curve; It is a natural constant.

[0034] Through this dual-frequency activation function, the network can automatically enhance the response to target frequency features, suppress irrelevant frequency domain noise and redundant information, and improve the purity and usability of features. In the dual-branch feature fusion stage, the adaptive heterogeneous reconstruction network performs element-wise weighted fusion of the global features output by the coarse structure perception branch and the high-frequency features output by the local detail compensation branch. The weights are automatically assigned according to the importance of the features, so that the fused features can be adapted to the subsequent registration and reconstruction tasks at the same time, thereby enhancing the representation ability and robustness of the features.

[0035] During network training, a scale-invariant loss function is used for supervised learning, which effectively reduces the scale bias caused by differences in viewpoint and pose, ensuring that the reconstruction results have stable scale consistency under different viewpoints. The adaptive heterogeneous reconstruction network is designed with "lightweight, high efficiency, and high expressiveness" as its core principles. It does not use any modules with large number of parameters or high computational complexity. At the same time, through innovative designs such as dual-branch structure, depthwise separable convolution, and dual-frequency activation function, the model can maintain or even surpass the reconstruction accuracy of traditional heavy networks while significantly reducing the number of parameters.

[0036] Step S2 completes the entire process of building the network structure, module selection, feature extraction, dual-frequency decoupling, and scale constraints. This makes the adaptive heterogeneous reconstruction network a lightweight core feature encoding network suitable for real-time industrial security inspection scenarios. It provides a high-quality, low-redundancy, and highly available feature foundation for subsequent epipolar cross-attention accurate registration and end-to-end 3D Gaussian primitive prediction. It is the core key link to achieve lightweight dual-view reconstruction.

[0037] In step S3, to address the registration difficulties in dual-view transmission images caused by differences in imaging geometry, target aliasing, deformation, and occlusion, such as... Figure 3 As shown, an epipolar cross-attention framework is constructed, and the accuracy of dual-view feature matching is improved step by step according to the three-level processing mechanism of "overview-alignment-compensation". By combining and coupling global attention, pixel center attention and epipolar attention, pixel-level accurate registration of dual-view images under complex conditions is achieved, providing a stable and reliable feature correspondence for subsequent 3D reconstruction. This solves the technical defects of poor robustness of traditional methods and large registration error of deep learning methods from the feature matching level.

[0038] In the specific implementation process, the high-quality dual-view feature maps output by the adaptive heterogeneous reconstruction network are first used as the input to the epipolar cross-attention framework, denoted as follows: and Then, feature alignment and optimization are performed step by step according to the three-level processing flow: The first level is the overview level, which generates global tokens through a lightweight global attention mechanism. It automatically captures the contextual dependencies of key pixels in the image across the entire image, constructs global structural constraints for the scene, ensures overall geometric consistency during the reconstruction process, and avoids overall reconstruction failure due to local mismatches.

[0039] This layer does not introduce a large amount of computational overhead, but can significantly improve the stability and anti-interference ability of the registration framework.

[0040] The second level is the alignment level, which is the core step in achieving accurate registration using the epipolar cross-attention framework. It generates two types of feature representations: pixel-centered tokens and epipolar constraint tokens. The pixel-centered tokens are from the first-view feature... Key pixels with high information content are selected, and local geometric features are extracted using a lightweight Transformer attention mechanism to capture stable matching points such as corners, edges, and textures. Epipolar constraint labeling strictly follows epipolar geometric constraints in the second-view feature extraction. Attention calculation is performed on the corresponding epipolar line to limit the feature matching range to the vicinity of the epipolar line, which greatly reduces the probability of mismatch. At the same time, the feature representation is further refined by combining depth information, so that the dual-view features can achieve accurate alignment under geometric constraints, effectively dealing with complex situations such as target aliasing, deformation, and occlusion. The third level is the compensation level, which introduces the epipolar OmniScopeAttention mechanism. It performs weighted fusion of the outputs of global attention, pixel center attention, and epipolar attention. It obtains the weights of each path by calculating the similarity of the query key, and then achieves multi-scale attention coupling through unified SoftMax operation. This strengthens the information interaction and complementarity between features of different granularities, and performs fine correction and compensation for the registration errors generated in the first two levels, further improving the registration accuracy and robustness.

[0041] In the feature fusion and output stage, the epipolar cross-attention framework performs feature fusion and filtering on the three types of tokens through deep convolution operations, retaining effective matching features and eliminating redundant and noisy features. Finally, it outputs a set of accurately registered and highly stable dual-view keypoint pairs through R-SoftMax, namely the registered first-view features and the registered second-view features, denoted as . and This provides accurate feature correspondences for subsequent 3D reconstruction.

[0042] The epipolar cross-attention framework is designed entirely according to the principle of lightweightness. All attention modules adopt a small-scale, low-computation, and high-efficiency structure, which does not rely on large Transformer models. It can run quickly on embedded devices and edge devices. At the same time, it combines epipolar geometric priors and data-driven attention mechanisms to make the registration results both geometrically rigorous and learning robust, and can continuously output stable and reliable registration results in complex industrial security inspection scenarios.

[0043] The entire process S3, from the design of a three-level mechanism, the generation of multiple types of tokens, the introduction of epipolar constraints, attention coupling to the output of precise key points, forms a complete, efficient, and robust dual-view registration process. It effectively solves the core problems of high difficulty, high error, and susceptibility to interference in dual-view transmission image registration, and provides an essential guarantee for accurate registration for subsequent end-to-end 3D Gaussian primitive prediction and high-quality 3D reconstruction. It is an important technical support for the high-precision reconstruction of this invention.

[0044] In step S4, based on the precise registration key points output by the epipolar cross-attention framework and the dual-frequency features extracted by the adaptive heterogeneous reconstruction network, end-to-end 3D Gaussian meta-parameter prediction is achieved through a fast Gaussian prediction network. Combined with a dual-frequency adaptive distribution strategy and a lightweight rendering mechanism, the inference speed is maximized while ensuring reconstruction accuracy, and the efficient mapping from dual-view 2D images to 3D structures is completed, ultimately achieving real-time, high-precision, and low-power 3D reconstruction in industrial scenarios.

[0045] like Figure 4 As shown, in the specific implementation process, the dual-view keypoint features registered by the epipolar cross-attention framework are first combined with the high-frequency keypoints extracted by the adaptive heterogeneous reconstruction network. Low-frequency key points The input is a fast Gaussian prediction network. The network first performs preliminary fusion of multi-frequency features through the EpiOS-Self attention module and the deep convolution module (Dw-Cov), effectively combining high-frequency detailed features with low-frequency global features, providing a comprehensive and compact feature representation for Gaussian meta-parameter prediction.

[0046] To improve feature representation efficiency and information flow capability, the Fast Gaussian Prediction Network introduces a StarOperation mechanism, which divides the input features into two parallel branches. Within each branch, the initial dimension of the keypoint features is adjusted. Adjusted to amplification factor In this invention, it is set It is used to enrich the feature interaction space without losing feature information and enhance the joint modeling capability of local and global features.

[0047] like Figure 5As shown in the figure, experiments have verified that this mechanism can effectively reduce rendering time and improve overall reconstruction efficiency. After feature enhancement, the Fast Gaussian Prediction Network enters the core Gaussian meta-parameter prediction stage, with the goal of outputting a set of Gaussian meta-parameters that can completely represent the 3D scene.

[0048] in, For the first Each three-dimensional Gaussian element is a basic unit that constitutes the reconstructed three-dimensional scene. A total of Gaussian elements collectively represent a complete 3D scene; For the first The three-dimensional center coordinates of each Gaussian element are position vectors in three-dimensional space, which determine the spatial position of the Gaussian distribution in the reconstructed scene. For the first The transparency parameter of a Gaussian unit, with a value range of [0,1], controls the visibility of the Gaussian unit during the rendering process and is used to simulate the semi-transparent or occluded effect of an object. For the first The shape covariance matrix of Gaussian elements is a 3×3 symmetric positive definite matrix, which defines the spatial ellipsoidal shape (size, rotation, anisotropy) of the Gaussian distribution and is used to characterize the local geometry of the object. For the first The appearance feature parameters of a Gaussian element can represent color, texture or other renderable attributes, and are used to define the visual appearance of the Gaussian element during rendering. This is an index variable for Gaussian elements, taking values ​​1, 2, ... ; The total number of Gaussian elements in the 3D scene is determined by the Fast Gaussian Prediction Network based on scene complexity adaptive prediction.

[0049] To address the characteristics of industrial transmission image reconstruction, the network focuses on the Gaussian center location. and shape parameters By performing refined parametric modeling, problems such as insufficient geometric texture extraction, blurred object outlines, and lack of structural details in traditional methods are effectively alleviated, and the clarity and structural accuracy of 3D reconstruction are significantly improved.

[0050] like Figure 6As shown, in the Gaussian pixel spatial distribution stage, an adaptive distribution strategy based on dual-frequency features is adopted. Gaussian pixels are densely distributed in the edge, texture, and detail-rich regions corresponding to high-frequency features to accurately capture minute structures and subtle changes. Gaussian pixels are sparsely distributed in the smooth regions and global contour regions corresponding to low-frequency features. This reduces redundant calculations and lowers rendering overhead while ensuring structural integrity. In the Gaussian rendering stage, the rendering granularity is adaptively adjusted according to the frequency attributes of the Gaussian pixels. A fine rendering mode is used in the high-frequency region to make the Gaussian ellipsoid smaller and denser, restoring subtle structural changes in the image. A simplified rendering mode is used in the low-frequency region to make the Gaussian ellipsoid larger and more sparsely distributed. This significantly improves the rendering speed without sacrificing visual effects. Through this dual-frequency adaptive rendering mechanism, this invention significantly reduces computation and latency while ensuring reconstruction accuracy, meeting the needs of real-time industrial detection.

[0051] The entire S4 process, from multi-frequency feature fusion, star-shaped operation enhancement, Gaussian meta-parameter prediction, dual-frequency adaptive distribution to lightweight and fast rendering, forms an end-to-end, high-efficiency, and high-precision 3D reconstruction and rendering workflow. It completely solves the problems of large computational load, slow speed, and difficult deployment of traditional 3D reconstruction methods. This invention enables stable, real-time, and high-precision dual-view 3D reconstruction on resource-constrained industrial terminals, mobile devices, and embedded platforms, providing a brand-new lightweight reconstruction solution for fields such as security inspection, industrial inspection, and medical imaging.

[0052] Second Embodiment Based on the above method, the second embodiment of the present invention provides an industrial image dual-view reconstruction system based on a lightweight network. For example... Figure 7 As shown, the industrial image dual-view reconstruction system includes one or more processors and a memory. The memory stores one or more programs, which, when executed by the processor, can implement the industrial image dual-view reconstruction method described in the above embodiments.

[0053] The processor controls the overall operation of the industrial image dual-view reconstruction system to complete all or part of the steps described above. The processor can be a central processing unit (CPU), graphics processing unit (GPU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), digital signal processing (DSP) chip, etc. The memory stores various types of data to support the operation of the industrial image dual-view reconstruction system. This data may include instructions for any application or method operating on the industrial image dual-view reconstruction system, as well as application-related data. The memory can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, etc.

[0054] In an exemplary embodiment, the industrial image dual-view reconstruction system can be implemented by a computer chip or physical entity, or by a product with certain functions, to perform the above-described method and achieve the same technical effect as described above. A typical embodiment is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0055] In another exemplary embodiment, the present invention also provides a computer-readable storage medium including program instructions that, when executed by a processor, implement the steps of the method in any of the above embodiments. For example, the computer-readable storage medium may be the memory including the program instructions described above, which can be executed by a processor to complete the above-described industrial image dual-view reconstruction method and achieve the same technical effects as the above-described industrial image dual-view reconstruction method.

[0056] It should be noted that the above embodiments are merely illustrative examples. The technical solutions of each embodiment can be combined, and all are within the protection scope of this invention.

[0057] The above provides a detailed description of the industrial image dual-view reconstruction method and system based on lightweight networks provided by this invention. Any obvious modifications made by those skilled in the art without departing from the essence of this invention will constitute an infringement of the patent rights of this invention and will incur corresponding legal liability.

Claims

1. A method for dual-view reconstruction of industrial images based on lightweight networks, characterized in that... Includes the following steps: S1: Simultaneously acquire dual-view transmission images of the target object using dual X-ray sources, obtain the projection matrix corresponding to each image, and construct a dataset for sparse view 3D reconstruction based on the dual-view transmission images and projection matrices, and perform preprocessing on the dataset. S2: Based on the dual-frequency feature distribution law of industrial transmission images, an adaptive heterogeneous reconstruction network is constructed. Through a dual-branch serial architecture, global structure perception and local detail feature extraction of dual-view images are realized, and cross-view feature fusion and scale-invariant feature encoding are completed. S3: Based on epipolar geometric constraints and multi-level attention mechanisms, an epipolar cross-attention framework is constructed to perform global overview perception, pixel-level center alignment and epipolar constraint matching on the dual-view features output by the adaptive heterogeneous reconstruction network, and feature registration is achieved through multi-path attention coupling. S4: Based on the epipolar cross-attention framework for registering key points and the dual-frequency features of the adaptive heterogeneous reconstruction network, a fast Gaussian prediction network is constructed to achieve end-to-end 3D Gaussian meta-parameter prediction. The Gaussian meta-space layout is optimized by combining the dual-frequency feature distribution strategy, while a lightweight rendering mechanism and inference efficiency optimization are introduced.

2. The industrial image dual-view reconstruction method as described in claim 1, characterized in that... In step S1, two X-ray sources are simultaneously exposed at a preset angle to generate the first industrial transmission image acquired from a dual-view perspective. and the second industrial transmission image acquired from dual perspectives And record the projection matrix of the ray source. The corresponding formula is: in, For the first A standardized representation of a 3D reconstructed scene; For the first Image tensors from a single perspective; It is a scaling factor, and , It is a positive real number; For the first Projection matrix of each ray source; This is an index variable for the scene and viewpoint, with values ​​1, 2, ... ; This represents the total number of viewpoints in the scene.

3. The industrial image dual-view reconstruction method as described in claim 1, characterized in that... In step S2, the first branch of the serial dual-branch structure is a coarse structure perception branch, which is used to extract low-frequency macroscopic features of the target and achieve feature dimensionality reduction and global perception through continuous lightweight downsampling layers, deep convolutional layers and bottleneck structures; the second branch is a local detail compensation branch, which is used to enhance the ability to express local details and make up for the high-frequency information lost by the coarse branch in the dimensionality reduction process.

4. The industrial image dual-view reconstruction method as described in claim 3, characterized in that... To automatically separate and enhance high-frequency information and low-frequency macroscopic features, a dual-frequency activation function is introduced, with the corresponding formula as follows: ， in, The interaction weights of high-frequency and low-frequency features between dual-view images; These are the learnable initial weight coefficients; For depthwise separable convolution; For Fast Fourier Transform; This is the first industrial transmission image acquired from a dual-view perspective. This is the second industrial transmission image acquired from a dual-view perspective. This is the output value of the dual-frequency activation function; ω represents the input feature values ​​of the activation function; ω represents the basic weight coefficients. The smoothing coefficient is a learnable factor. It is a natural constant.

5. The industrial image dual-view reconstruction method as described in claim 1, characterized in that... In step S3, the epipolar cross attention framework is executed according to a three-level mechanism of "overview-alignment-compensation", which generates global perception markers, pixel center markers and epipolar constraint markers in sequence, and registers key point pairs with the R-SoftMax output through depth convolution.

6. The industrial image dual-view reconstruction method as described in claim 1, characterized in that... In step S4, the initial dimension is determined through a star-shaped operation mechanism. Adjusted to This is used to improve feature interaction efficiency and reduce rendering time; among which Amplification factor.

7. The industrial image dual-view reconstruction method as described in claim 1, characterized in that... In step S4, the three-dimensional Gaussian primitive is represented as: in, For the first A three-dimensional Gaussian element; For the first The three-dimensional center coordinates of each Gaussian element; For the first The transparency parameter of each Gaussian element has a value range of [0,1]. For the first The shape covariance matrix of Gaussian elements; For the first Appearance feature parameters of each high-resolution element; This is an index variable for Gaussian elements, taking values ​​1, 2, ... ; This represents the total number of Gaussian elements in the 3D scene.

8. The industrial image dual-view reconstruction method as described in claim 7, characterized in that... The three-dimensional Gaussian elements are densely distributed in the high-frequency region and sparsely distributed in the low-frequency region, and the granularity is adaptively adjusted according to the frequency during rendering.

9. An industrial image dual-view reconstruction system based on lightweight networks, characterized in that... It includes a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, which, when executed by the processor, enables the processor to implement the industrial image dual-view reconstruction method according to any one of claims 1 to 8.

Images