Intelligent Motherboard Display Driver System Based on 3D Display and Augmented Reality

Abstract

The present invention relates to the technical fields of image processing and 3D image generation, and provides an intelligent mainboard display driving system based on 3D display and augmented reality; first, the system acquires image data and camera parameter data of a home environment through an image acquisition module, and constructs a fractional-domain enhanced convolutional denoising network by combining fractional-domain transformation technology and convolutional neural network technology in the image denoising stage to remove noise in the image; then, the system uses an improved generative adversarial network technology for image enhancement; the system also calculates a corrected image disparity map through a multi-scale fusion Census cost enhanced SGM algorithm to generate an accurate 3D model; through the above steps, the present invention significantly improves the image quality and three-dimensional sense in the decoration effect display of AR devices, and enhances the visual experience of users and the visualization effect of decoration design.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and 3D image generation, and particularly to an intelligent mainboard display driving system based on 3D display and augmented reality. Background Art

[0002] An intelligent mainboard display driving system is a combined system integrating an intelligent processing module and display control technology, mainly applied to fields requiring high-quality 3D display and augmented reality, especially suitable for scenarios with high requirements for image accuracy, details, and vividness. When the existing 3D display and augmented reality technologies are applied to the display of decoration effects, they usually face bottlenecks in image quality and processing speed, especially in image denoising, enhancement, and 3D model generation. In the process of traditional image denoising, the importance of image frequency domain features is often ignored, resulting in poor denoising effects, inability to effectively retain image details, and lack of pertinence in noise processing. In addition, the existing image enhancement technologies have weak detail enhancement capabilities and mainly rely on simple loss functions, resulting in the clarity and naturalness of the generated images not meeting high-quality requirements. In 3D image generation, the calculation accuracy of the disparity map is often insufficient, and the capture of details is not sensitive enough, resulting in errors in the stereoscopic correction of the generated 3D model. Therefore, there is an urgent need for an intelligent display driving system that can balance high performance and processing accuracy in the processes of image denoising, enhancement, and 3D image generation to improve the image quality and model accuracy in the display of decoration effects. Through this system, users can preview decoration design schemes more clearly and vividly through 3D display and augmented reality technologies, thereby improving decision-making efficiency and significantly enhancing the user experience. Summary of the Invention

[0003] The present invention provides an intelligent mainboard display driving system based on 3D display and augmented reality, aiming to solve the problem of low performance of the system in image denoising, enhancement, and 3D model generation due to the limitations of image processing algorithms in the prior art. The existing system ignores the effective extraction of frequency domain features during image denoising, the enhancement effect is not ideal, and the ability to capture details during 3D image generation is insufficient, affecting the display accuracy. The present invention adopts a fractional domain enhanced convolutional denoising network that combines fractional domain transformation technology and convolutional neural network to improve the denoising accuracy and retain more image details. Through an improved generative adversarial network and an optimized loss function, the clarity and naturalness of the image are further enhanced. In 3D image generation, a multi-scale fusion Census cost enhanced SGM algorithm is adopted to make the calculation of the disparity map more accurate, effectively improving the stereoscopic correction and the generation effect of the 3D model. This system provides higher image clarity and stereoscopic display effects in the display of decoration effects. When users view through AR devices, they can obtain a more realistic visual experience, significantly improving the accuracy and practicality of the display of decoration design schemes.

[0004] The present invention provides an intelligent motherboard display driving system based on 3D display and augmented reality. The system includes: an image acquisition module, an image denoising module, an image enhancement module, a 3D image generation module, an augmented reality processing module, and a driving control module;

[0005] The image acquisition module simultaneously acquires home environment images from different perspectives through a binocular camera to obtain home environment image pair data, and acquires the camera parameter data of the binocular camera;

[0006] The image denoising module combines the fractional domain transform technology and the convolutional neural network technology to construct a fractional domain enhanced convolutional denoising network, and uses the fractional domain enhanced convolutional denoising network to denoise the home environment image pair data to obtain denoised image pair data;

[0007] The image enhancement module combines the generative adversarial network technology and an improved loss function calculation method to construct an improved generative adversarial network, and uses the improved generative adversarial network to enhance the denoised image pair data to obtain enhanced image pair data. The improved generative adversarial network includes generator enhancement, discriminator confrontation, and loss calculation;

[0008] The 3D image generation module performs stereo correction on the enhanced image pair data to obtain corrected image pair data, then uses the multi-scale fusion Census cost enhanced SGM algorithm to calculate the disparity map between the corrected image pair data, and constructs a three-dimensional model of the home environment according to the disparity map and the camera parameter data;

[0009] The augmented reality processing module fuses the three-dimensional model of the home environment with the real scene, and the user can view virtual furniture, wall colors, and decorations through an AR device;

[0010] The driving control module coordinates the work of the 3D image generation module and the augmented reality processing module.

[0011] Further, the process of the image denoising module using the fractional domain enhanced convolutional denoising network to denoise the home environment image pair data to obtain denoised image pair data specifically includes the following steps:

[0012] Step N1: Build a model: Build a fractional domain enhanced convolutional denoising network model, and receive the home environment image pair data as the input data of the fractional domain enhanced convolutional denoising network model;

[0013] Step N2: Perform convolution operation: Perform convolution operation on the home environment image pair data to extract the basic features of the image, including edge, texture, and simple structure information, to obtain shallow feature data;

[0014] Step N3: Apply fractional-order frequency filtering: Combine fractional calculus and Fourier transform to obtain a fractional-order frequency filter. Capture the important frequency information of the shallow feature data through the fractional-order frequency filter to obtain frequency feature data. The formula used is as follows:

[0015] ;

[0016] where, represents the frequency feature data, represents the fractional-order frequency filter, represents the order of the fractional order, represents the shallow feature data;

[0017] Step N4: Extract deep features: Gradually extract the complex image features of the frequency feature data through six convolutional blocks, and output the deep feature data. Each convolutional block includes a convolution operation, batch normalization, FReLU activation function, and skip connection mechanism;

[0018] Step N41: Perform convolution operation: Receive the frequency feature data, perform local convolution operation through the convolution kernel, and extract the complex image features in the frequency feature data to obtain the preliminary convolution feature data;

[0019] Step N42: Batch normalization: Perform batch normalization on the preliminary convolution feature data to eliminate the numerical differences between different channels and obtain the normalized feature data;

[0020] Step N43: Apply FReLU activation function: Perform non-linear processing on the normalized feature data through the FReLU activation function to enhance the network's ability to represent complex non-linear features in the image and obtain the non-linear feature data;

[0021] Step N44: Implement skip connection mechanism: Add the frequency feature data and the non-linear feature data to retain the key information and enhance the representation ability of the deep features at the same time, and obtain the deep feature data.

[0022] Step N5: Capture directional features: Combine fractional calculus and directional filters to obtain a fractional-order directional filter. Capture the feature information in different directions of the deep feature data through the fractional-order directional filter to obtain enhanced feature data. The formula used is as follows:

[0023] ;

[0024] where, represents the enhanced feature data, represents the fractional-order directional filter, represents the deep feature data;

[0025] Step N6: Reconstruct the image: Reconstruct the enhanced feature data into a denoised high-definition image to generate denoised image pair data.

[0026] Further, the enhancement of the generator in the image enhancement module specifically includes: Inputting the denoised image pair data into the generator network for feature extraction and enhancement processing to obtain initial enhanced image data.

[0027] Further, the adversarial training of the discriminator in the image enhancement module specifically includes: Inputting the initial enhanced image data into the discriminator network, and through the adversarial training of the generator network and the discriminator network, continuously optimizing the generator network to finally generate enhanced image pair data.

[0028] Further, the loss calculation in the image enhancement module specifically includes: Using a combination of MSE loss, SSIM loss, adversarial loss, and regularization loss to construct a comprehensive loss function, and guiding the optimization of the generator network through the comprehensive loss function.

[0029] Further, the process of using a combination of MSE loss, perceptual loss, adversarial loss, and regularization loss to construct a comprehensive loss function specifically includes the following steps:

[0030] Step S1: Initialize the weight parameters of MSE loss, SSIM loss, adversarial loss, and regularization loss;

[0031] Step S2: During the iteration process, calculate the loss values of MSE loss, SSIM loss, adversarial loss, and regularization loss in the current round, and calculate the loss change rate by comparing the loss values of MSE loss, SSIM loss, adversarial loss, and regularization loss in the previous round. Dynamically adjust the weight parameters according to the loss change rate to obtain the adjusted weight parameters. The formula used is as follows:

[0032] ;

[0033] Where, represents the weight parameter of the th loss term at the th iteration, represents the weight parameter of the th loss term at the th iteration, represents the loss value of the th loss term at the th iteration, represents the loss value of the th loss term at the th iteration, represents the total number of loss terms, which are four, namely MSE loss, perceptual loss, adversarial loss, and regularization loss. Indicates the loss item index, Indicates all The total change in losses of all loss items in the current round and the previous round;

[0034] Step S3: Recombine according to the adjusted weight parameters to construct a comprehensive loss function.

[0035] ;

[0036] Among them, Indicates the comprehensive loss function, Indicates the MSE loss weight, Indicates the MSE loss item; Indicates the SSIM loss weight, Indicates the SSIM loss item; Indicates the adversarial loss weight, Adversarial loss item; Indicates the weight of the regularization loss, Indicates the regularization loss item;

[0037] Furthermore, in the 3D image generation module, the process of calculating the disparity map of the corrected image pair data by the multi-scale fusion Census cost enhanced SGM algorithm specifically includes the following steps:

[0038] Step T1: Multi-scale Census feature extraction layer: Perform Census transformation on the corrected image pair data, and extract the Census features of the pixel neighborhood at three scales of 3×3, 5×5, and 7×7 to generate multi-scale Census feature data;

[0039] Step T2: Multi-scale feature fusion layer: Weightedly fuse the multi-scale Census feature data to capture detail information at different levels to obtain fused Census feature data. The formula used is as follows:

[0040] ;

[0041] Among them, Indicates the fused Census feature data, Indicates the 3×3 scale Census feature data, Indicates the 5×5 scale Census feature data, Indicates the 7×7 scale Census feature data, , and Indicates the weighted coefficients of the scale features;

[0042] Step T3: Cost calculation layer: Calculate the matching cost of the fused Census feature data using the Hamming distance to obtain the matching cost data;

[0043] Step T4: SGM cost aggregation layer: Use the SGM algorithm to globally optimize the matching cost data in the horizontal, vertical, and diagonal directions, and accumulate the matching cost through dynamic programming to obtain the aggregated cost data;

[0044] Step T41: Horizontal cost aggregation: First, perform a traversal process on the matching cost data in the horizontal direction, and perform cumulative calculations in two directions from left to right and from right to left to obtain the horizontal direction cost aggregation data;

[0045] Step T42: Vertical cost aggregation: Then, perform a traversal process in the vertical direction, and perform cumulative calculations in two directions from top to bottom and from bottom to top to obtain the vertical direction cost aggregation data;

[0046] Step T43: Diagonal cost aggregation: Finally, perform a traversal in the diagonal direction, and perform cost accumulation in two directions from top left to bottom right and from top right to bottom left to obtain the diagonal direction cost aggregation data;

[0047] Step T44: Dynamic programming optimization: Perform cost accumulation on the horizontal direction cost aggregation data, vertical direction cost aggregation data, and diagonal direction cost aggregation data to obtain the cost aggregation result, and use dynamic programming technology to globally optimize the cost aggregation result to eliminate path conflicts and obtain the aggregated cost data.

[0048] Step T41: Horizontal cost aggregation: First, perform a traversal process on the matching cost data in the horizontal direction, and perform cumulative calculations in two directions from left to right and from right to left to obtain the horizontal direction cost aggregation data;

[0049] Step T42: Vertical cost aggregation: Then, perform a traversal process in the vertical direction, and perform cumulative calculations in two directions from top to bottom and from bottom to top to obtain the vertical direction cost aggregation data;

[0050] Step T43: Diagonal cost aggregation: Finally, perform a traversal in the diagonal direction, and perform cost accumulation in two directions from top left to bottom right and from top right to bottom left to obtain the diagonal direction cost aggregation data;

[0051] Step T44: Dynamic programming optimization: Perform cost accumulation on the horizontal direction cost aggregation data, vertical direction cost aggregation data, and diagonal direction cost aggregation data to obtain the cost aggregation result, and use dynamic programming technology to globally optimize the cost aggregation result to eliminate path conflicts and obtain the aggregated cost data.

[0052] Step T5: Disparity Selection Layer: Select the minimum matching cost from the aggregated cost data, determine the corresponding disparity value, and generate a disparity map.

[0053] Further, the process of inputting the denoised image pair data into the generator network for feature extraction and enhancement processing to obtain the initial enhanced image data specifically includes the following steps:

[0054] Step B1: Extract Preliminary Features: Perform preliminary feature extraction on the denoised image pair data to obtain preliminary feature data;

[0055] Step B2: Apply Parametric ReLU Activation Function: Perform non-linear activation on the preliminary feature data to endow the network with stronger non-linear expression ability and obtain the activated feature data;

[0056] Step B3: Perform E-Sum Accumulation: Process the activated feature data through six residual blocks and perform accumulation through the E-Sum module to achieve feature fusion and obtain the fused feature data;

[0057] Step B4: Perform Upsampling Processing: Perform upsampling processing on the fused feature data to increase the resolution of the image and obtain high-resolution feature data;

[0058] Step B5: Perform Downsampling Processing: Perform downsampling processing on the high-resolution feature data to further enhance the image details and obtain the enhanced feature data;

[0059] Step B6: Generate Initial Enhanced Image Data: Generate the initial enhanced image data by passing the enhanced feature data through the convolutional layer.

[0060] Adopting the above solution, the beneficial effects obtained by the present invention are as follows:

[0061] An intelligent mainboard display driving system based on 3D display and augmented reality provided by the present invention significantly improves the image denoising ability by introducing a fractional domain enhanced convolutional denoising network that combines fractional domain transformation technology and convolutional neural network technology; existing denoising methods have poor ability to retain image details and are prone to image detail loss, especially in complex environments, and the noise processing effect is relatively limited; the present invention extracts the frequency domain features in the image through fractional domain transformation technology and combines the powerful feature extraction ability of convolutional neural network, not only effectively removes the noise in the image, but also greatly enhances the image detail retention ability, and finally makes the decoration effect displayed on the AR device clearer, and users can view the decoration design details more accurately, improving the user experience.

[0062] The present invention also adopts the improved generative adversarial network technology and realizes the improvement of the image enhancement process by optimizing the calculation method of the loss function. Most of the existing image enhancement methods rely on simple loss functions and it is difficult to achieve a balance between details and global quality at the same time, resulting in insufficient visual clarity of the generated images. Through the mutual confrontation between the generator and the discriminator of the generative adversarial network, the present invention improves the accuracy of image enhancement, and combines the comprehensive calculation methods of MSE loss, SSIM loss, adversarial loss and regularization loss to optimize the global and detail expressiveness of the images, ensuring that users can obtain a more realistic and delicate visual experience when viewing the decoration effect through AR devices.

[0063] In addition, in the aspect of 3D image generation, the present invention adopts the multi-scale fusion Census cost enhanced SGM algorithm, which greatly improves the accuracy of disparity map calculation. Traditional disparity map calculation methods have weak detail capture ability when dealing with complex scenes, which easily leads to large errors in 3D models and affects the overall display effect. The present invention extracts Census features at multiple scales and fuses them, and combines the optimized SGM algorithm to globally optimize the disparity map, thereby generating a more accurate 3D model. The application of this technology not only solves the deficiencies of traditional methods in disparity calculation, but also enhances the stereoscopic display ability of AR devices for decoration effects, provides a more realistic sense of space, further improves the visualization effect of decoration design and the decision-making efficiency of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of a module of an intelligent mainboard display driving system based on 3D display and reality augmentation proposed by the present invention;

[0065] Figure 2 It is a network architecture diagram of the image denoising module proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0066] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0067] Embodiment 1, according to Figure 1 , the present invention provides an intelligent mainboard display driving system based on 3D display and reality augmentation, and the system includes: an image acquisition module, an image denoising module, an image enhancement module, a 3D image generation module, an augmented reality processing module and a driving control module;

[0068] The image acquisition module simultaneously acquires home environment images from different perspectives through a binocular camera, obtains home environment image pair data, and acquires the camera parameter data of the binocular camera.

[0069] The image denoising module combines the fractional domain transformation technology and the convolutional neural network technology to construct a fractional domain enhanced convolutional denoising network, and uses the fractional domain enhanced convolutional denoising network to denoise the home environment image pair data to obtain denoised image pair data.

[0070] The image enhancement module combines the generative adversarial network technology and an improved loss function calculation method to construct an improved generative adversarial network, and uses the improved generative adversarial network to enhance the denoised image pair data to obtain enhanced image pair data. The improved generative adversarial network includes generator enhancement, discriminator confrontation, and loss calculation.

[0071] The 3D image generation module performs stereo correction on the enhanced image pair data to obtain corrected image pair data, then uses the multi-scale fusion Census cost enhanced SGM algorithm to calculate the disparity map between the corrected image pair data, and constructs a three-dimensional model of the home environment based on the disparity map and the camera parameter data.

[0072] The augmented reality processing module fuses the three-dimensional model of the home environment with the real scene, and users can view virtual furniture, wall colors, and decorations through AR devices.

[0073] The drive control module coordinates the work of the 3D image generation module and the augmented reality processing module.

[0074] Embodiment 2, according to Figure 2 , this embodiment is based on the above embodiment. The process of the image denoising module using the fractional domain enhanced convolutional denoising network to denoise the home environment image pair data to obtain denoised image pair data specifically includes the following steps:

[0075] Step N1: Build a model: Build a fractional domain enhanced convolutional denoising network model, and receive the home environment image pair data as the input data of the fractional domain enhanced convolutional denoising network model.

[0076] Step N2: Perform convolution operation: Perform convolution operation on the home environment image pair data to extract the basic features of the image, including edge, texture, and simple structure information, to obtain shallow feature data.

[0077] Step N3: Apply fractional order frequency filtering: Combine fractional order calculus and Fourier transform to obtain a fractional order frequency filter, and capture the important frequency information of the shallow feature data through the fractional order frequency filter to obtain frequency feature data. The formula used is as follows:

[0078] ;

[0079] Among them, represents frequency feature data, represents a fractional-order frequency filter, represents the order of the fractional order, represents shallow feature data;

[0080] Step N4: Extract deep features: Gradually extract complex image features of the frequency feature data through six convolutional blocks, and output deep feature data. Each convolutional block includes a convolution operation, batch normalization, FReLU activation function, and skip connection mechanism;

[0081] Step N41: Perform convolution operation: Receive the frequency feature data, perform local convolution operation through the convolution kernel, extract the complex image features in the frequency feature data, and obtain preliminary convolution feature data;

[0082] Step N42: Batch normalization: Perform batch normalization on the preliminary convolution feature data to eliminate the numerical differences between different channels and obtain normalized feature data;

[0083] Step N43: Apply the FReLU activation function: Non-linearly process the normalized feature data through the FReLU activation function to enhance the network's ability to represent complex non-linear features in the image and obtain non-linear feature data;

[0084] Step N44: Implement the skip connection mechanism: Add the frequency feature data and the non-linear feature data to retain key information and at the same time enhance the representation ability of the deep features to obtain deep feature data.

[0085] Step N5: Capture directional features: Combine fractional calculus and directional filters to obtain a fractional-order directional filter. Capture feature information in different directions of the deep feature data through the fractional-order directional filter to obtain enhanced feature data. The formula used is as follows:

[0086] ;

[0087] Among them, represents the enhanced feature data, represents the fractional-order directional filter, represents the deep feature data;

[0088] Step N6: Reconstruct the image: Reconstruct the enhanced feature data into a high-definition image after denoising to generate denoised image pair data.

[0089] Example 3. This example is based on the above example. The enhancement of the generator in the image enhancement module specifically includes: inputting the denoised image pair data into the generator network for feature extraction and enhancement processing to obtain initial enhanced image data.

[0090] Example 4. This example is based on the above example. The adversarial training of the discriminator in the image enhancement module specifically includes: inputting the initial enhanced image data into the discriminator network, and continuously optimizing the generator network through the adversarial training of the generator network and the discriminator network, and finally generating enhanced image pair data.

[0091] Example 5. This example is based on the above example. The loss calculation in the image enhancement module specifically includes: constructing a comprehensive loss function using a combination of MSE loss, SSIM loss, adversarial loss, and regularization loss, and guiding the optimization of the generator network through the comprehensive loss function.

[0092] Example 6. This example is based on the above example. The process of constructing a comprehensive loss function using a combination of MSE loss, perceptual loss, adversarial loss, and regularization loss specifically includes the following steps:

[0093] Step S1: Initialize the weight parameters of MSE loss, SSIM loss, adversarial loss, and regularization loss;

[0094] Step S2: During the iteration process, calculate the loss values of MSE loss, SSIM loss, adversarial loss, and regularization loss in the current round, and calculate the loss change rate by comparing the loss values of MSE loss, SSIM loss, adversarial loss, and regularization loss in the previous round. Dynamically adjust the weight parameters according to the loss change rate to obtain the adjusted weight parameters. The formula used is as follows:

[0095] ;

[0096] Among them, represents the weight parameter of the th loss term at the th iteration, represents the weight parameter of the th loss term at the th iteration, represents the loss value of the th loss term at the th iteration, represents the loss value of the th loss term at the th iteration, represents the total number of loss terms, which are four, namely MSE loss, perceptual loss, adversarial loss, and regularization loss, represents the loss term index, Denote all the total change in losses of all loss terms in the current round and the previous round;

[0097] Step S3: Recombine according to the adjusted weight parameters to construct a comprehensive loss function.

[0098] ;

[0099] Among them, denotes the comprehensive loss function, denotes the MSE loss weight, denotes the MSE loss term; denotes the SSIM loss weight, denotes the SSIM loss term; denotes the adversarial loss weight, the adversarial loss term; denotes the weight of the regularization loss, denotes the regularization loss term;

[0100] Example 7. Based on the above example, in the 3D image generation module, the process of calculating the disparity map of the corrected image pair data by the multi-scale fusion Census cost enhanced SGM algorithm specifically includes the following steps:

[0101] Step T1: Multi-scale Census feature extraction layer: Perform Census transformation on the corrected image pair data, and extract the Census features of the pixel neighborhood at three scales of 3×3, 5×5, and 7×7 to generate multi-scale Census feature data;

[0102] Step T2: Multi-scale feature fusion layer: Perform weighted fusion on the multi-scale Census feature data to capture detail information at different levels, and obtain the fused Census feature data. The formula used is as follows:

[0103] ;

[0104] Among them, denotes the fused Census feature data, denotes the Census feature data at the 3×3 scale, denotes the Census feature data at the 5×5 scale, denotes the Census feature data at the 7×7 scale, , and denote the weighted coefficients of the scale features;

[0105] Step T3: Cost calculation layer: Calculate the matching cost of the fused Census feature data using the Hamming distance to obtain the matching cost data;

[0106] Step T4: SGM cost aggregation layer: Use the SGM algorithm to globally optimize the matching cost data in the horizontal, vertical, and diagonal directions, and accumulate the matching cost through dynamic programming to obtain the aggregated cost data;

[0107] Step T41: Horizontal cost aggregation: First, perform a traversal process on the matching cost data in the horizontal direction, and perform cumulative calculations in two directions from left to right and from right to left to obtain the horizontal direction cost aggregation data;

[0108] Step T42: Vertical cost aggregation: Then, perform a traversal process in the vertical direction, and perform cumulative calculations in two directions from top to bottom and from bottom to top to obtain the vertical direction cost aggregation data;

[0109] Step T43: Diagonal cost aggregation: Finally, perform a traversal in the diagonal direction, and perform cost accumulation in two directions from top left to bottom right and from top right to bottom left to obtain the diagonal direction cost aggregation data;

[0110] Step T44: Dynamic programming optimization: Perform cost accumulation on the horizontal direction cost aggregation data, vertical direction cost aggregation data, and diagonal direction cost aggregation data to obtain the cost aggregation result, and use dynamic programming technology to globally optimize the cost aggregation result to eliminate path conflicts and obtain the aggregated cost data.

[0111] Step T5: Disparity selection layer: Select the minimum matching cost from the aggregated cost data, determine the corresponding disparity value, and generate a disparity map.

[0112] Embodiment 8, based on the above embodiment, the process of inputting the denoised image pair data into the generator network for feature extraction and enhancement processing to obtain the initial enhanced image data specifically includes the following steps:

[0113] Step B1: Extract preliminary features: Perform preliminary feature extraction on the denoised image pair data to obtain preliminary feature data;

[0114] Step B2: Apply the parametric ReLU activation function: Perform non-linear activation on the preliminary feature data to endow the network with stronger non-linear expression ability to obtain the activated feature data;

[0115] Step B3: Perform E-Sum accumulation: Process the activated feature data through six residual blocks and perform accumulation through the E-Sum module to achieve feature fusion and obtain the fused feature data;

[0116] Step B4: Perform upsampling processing: Perform upsampling processing on the fused feature data to increase the resolution of the image and obtain high-resolution feature data;

[0117] Step B5: Perform downsampling processing: Perform downsampling processing on the high-resolution feature data to further enhance the image details and obtain enhanced feature data;

[0118] Step B6: Generate initial enhanced image data: Generate initial enhanced image data from the enhanced feature data through a convolutional layer.

[0119] Embodiment Nine, this embodiment is based on the above embodiment. The augmented reality processing module fuses the three-dimensional model of the home environment and the real scene, and the user can view virtual furniture, wall colors, and decorations through an AR device;

[0120] In this embodiment, the user places virtual sofas, coffee tables, and bookshelves in the blank area of the living room through an AR device, adjusts the position and size of the furniture through drag gestures, and views their visual effects and spatial layouts in different positions; selects a pink wall color and views the effect of applying the pink wall color to the real living room wall in the AR device; hangs virtual picture frames and wall clocks on the wall.

Claims

1. An intelligent mainboard display driving system based on 3D display and augmented reality, characterized in that: It includes an image acquisition module, an image denoising module, an image enhancement module, and a 3D image generation module; The image acquisition module simultaneously acquires home environment images from different perspectives through a binocular camera, obtains home environment image pair data, and acquires the camera parameter data of the binocular camera; The image denoising module combines the fractional domain transformation technology and the convolutional neural network technology to construct a fractional domain enhanced convolutional denoising network, and uses the fractional domain enhanced convolutional denoising network to denoise the home environment image pair data to obtain denoised image pair data; The image enhancement module combines the generative adversarial network technology and an improved loss function calculation method to construct an improved generative adversarial network, and uses the improved generative adversarial network to enhance the denoised image pair data to obtain enhanced image pair data. The improved generative adversarial network includes generator enhancement, discriminator adversarial, and loss calculation; The 3D image generation module performs stereo correction on the enhanced image pair data to obtain corrected image pair data, then uses the multi-scale fusion Census cost enhanced SGM algorithm to calculate the disparity map between the corrected image pair data, and constructs a three-dimensional model of the home environment according to the disparity map and the camera parameter data; The process of the image denoising module using the fractional domain enhanced convolutional denoising network to denoise the home environment image pair data to obtain denoised image pair data specifically includes the following steps: Step N1: Build a model: Build a fractional domain enhanced convolutional denoising network model and receive the home environment image pair data as the input data of the fractional domain enhanced convolutional denoising network model; Step N2: Perform convolution operation: Perform a convolution operation on the home environment image pair data to extract the basic features of the image, including edge, texture, and simple structure information, to obtain shallow feature data; Step N3: Apply fractional order frequency filtering: Combine fractional order calculus and Fourier transform to obtain a fractional order frequency filter, and capture the important frequency information of the shallow feature data through the fractional order frequency filter to obtain frequency feature data; Step N4: Extract deep features: Gradually extract the complex image features of the frequency feature data through six convolutional blocks and output deep feature data. Each convolutional block includes a convolution operation, batch normalization, an FReLU activation function, and a skip connection mechanism; Step N5: Capture directional features: Combine fractional order calculus and a directional filter to obtain a fractional order directional filter, and capture the feature information in different directions of the deep feature data through the fractional order directional filter to obtain enhanced feature data; Step N6: Reconstruct the image: Reconstruct the enhanced feature data into a high-definition image after denoising to generate denoised image pair data.

2. The intelligent mainboard display driving system based on 3D display and augmented reality according to claim 1, wherein: The generator enhancement in the image enhancement module specifically includes: Inputting the denoised image pair data into the generator network for feature extraction and enhancement processing to obtain initial enhanced image data.

3. The intelligent mainboard display driving system based on 3D display and augmented reality according to claim 2, characterized in that: The discriminator adversarial in the image enhancement module specifically includes: Inputting the initial enhanced image data into the discriminator network, and continuously optimizing the generator network through the adversarial training of the generator network and the discriminator network, and finally generating enhanced image pair data.

4. An intelligent mainboard display driving system based on 3D display and augmented reality according to claim 3, characterized in that: The loss calculation in the image enhancement module specifically includes: constructing a comprehensive loss function using a combination of MSE loss, SSIM loss, adversarial loss, and regularization loss, and guiding the optimization of the generator network through the comprehensive loss function.

5. The intelligent mainboard display driving system based on 3D display and augmented reality according to claim 4, characterized in that: The process of constructing a comprehensive loss function using a combination of MSE loss, perceptual loss, adversarial loss, and regularization loss specifically includes the following steps: Step S1: Initialize the weight parameters of MSE loss, SSIM loss, adversarial loss, and regularization loss; Step S2: During the iteration process, calculate the loss values of MSE loss, SSIM loss, adversarial loss, and regularization loss in the current round, and calculate the loss change rate by comparing the loss values of MSE loss, SSIM loss, adversarial loss, and regularization loss in the previous round. Dynamically adjust the weight parameters according to the loss change rate to obtain the adjusted weight parameters; Step S3: Recombine according to the adjusted weight parameters to construct a comprehensive loss function.

6. The intelligent mainboard display driving system based on 3D display and augmented reality according to claim 1, wherein: In the 3D image generation module, the process of calculating the disparity map of the corrected image pair data by the multi-scale fusion Census cost enhanced SGM algorithm specifically includes the following steps: Step T1: Multi-scale Census feature extraction layer: Perform Census transformation on the corrected image pair data, extract the Census features of the pixel neighborhood at three scales of 3×3, 5×5, and 7×7, and generate multi-scale Census feature data; Step T2: Multi-scale feature fusion layer: Weightedly fuse the multi-scale Census feature data to capture different levels of detail information and obtain the fused Census feature data; Step T3: Cost calculation layer: Calculate the matching cost of the fused Census feature data using the Hamming distance to obtain the matching cost data; Step T4: SGM cost aggregation layer: Use the SGM algorithm to globally optimize the matching cost data in the horizontal, vertical, and diagonal directions, and accumulate the matching cost through dynamic programming to obtain the aggregated cost data; Step T5: Disparity selection layer: Select the minimum matching cost from the aggregated cost data, determine the corresponding disparity value, and generate the disparity map.

Images