Intelligent mainboard display driving system based on 3D display and reality augmentation

Abstract

The invention relates to the technical field of image processing and 3D image generation, and provides an intelligent mainboard display driving system based on 3D display and reality augmentation. According to the system, firstly, image data of a home environment and camera parameter data are acquired through an image acquisition module, and a fractional domain enhanced convolutional denoising network is constructed in combination with a fractional domain transformation technology and a convolutional neural network technology in an image denoising stage to remove noise in an image; then, the system adopts an improved generative adversarial network technology to carry out image enhancement; the system also calculates a corrected image disparity map through a multi-scale fusion Census cost enhancement SGM algorithm to generate an accurate 3D model; through the steps, the image quality and stereoscopic impression of the AR equipment in decoration effect display are remarkably improved, and the visual experience of a user and the visual effect of decoration design are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and 3D image generation, and in particular to an intelligent mainboard display driving system based on 3D display and reality enhancement. Background Art

[0002] The intelligent motherboard display driver system is a combined system that integrates intelligent processing modules and display control technology. It is mainly used in fields that require high-quality 3D display and augmented reality, especially in scenes that require high image accuracy, details and realism. Existing 3D display and augmented reality technologies are usually faced with bottlenecks in image quality and processing speed when applied to decoration effect display, especially in image denoising and enhancement and 3D model generation; traditional methods often ignore the importance of image frequency domain features in the image denoising process, resulting in poor denoising effect, inability to effectively retain image details, and lack of targeted noise processing; in addition, existing image enhancement technologies have limited enhancement of details. The strong ability is weak and it mainly relies on simple loss functions, resulting in the clarity and naturalness of the generated images failing to meet high-quality requirements; in 3D image generation, the calculation accuracy of the disparity map is often insufficient, and the capture of details is not sensitive enough, resulting in errors in the stereo correction of the generated 3D model; therefore, there is an urgent need for an intelligent display drive system that can take into account both high-efficiency performance and processing accuracy in the process of image denoising, enhancement and 3D image generation, so as to improve the image quality and model accuracy when displaying decoration effects; through this system, users can use 3D display and augmented reality technology to preview decoration design plans more clearly and realistically, thereby improving decision-making efficiency and significantly improving user experience. Summary of the invention

[0003] The present invention provides an intelligent mainboard display driving system based on 3D display and augmented reality, aiming to solve the problem in the prior art that the system has low performance in image denoising, enhancement and 3D model generation due to the limitations of image processing algorithms; the existing system ignores the effective extraction of frequency domain features during image denoising, the enhancement effect is not ideal, and the ability to capture details is insufficient during 3D image generation, which affects the display accuracy; the present invention adopts a fractional domain enhanced convolution denoising network combining fractional domain transformation technology with a convolutional neural network, improves the denoising accuracy, and retains more image details; through an improved generative adversarial network and an optimized loss function, the clarity and naturalness of the image are further enhanced; in 3D image generation, a multi-scale fused Census cost enhancement SGM algorithm is adopted to make the disparity map calculation more accurate, effectively improving the stereo correction and 3D model generation effects; the system provides higher image clarity and stereo display effect in the display of decoration effects, and users can obtain a more realistic visual experience when viewing through an AR device, which significantly improves the accuracy and practicality of the display of decoration design schemes.

[0004] The present invention provides an intelligent mainboard display driving system based on 3D display and reality enhancement, the system comprising: an image acquisition module, an image denoising module, an image enhancement module, a 3D image generation module, an augmented reality processing module and a driving control module;

[0005] The image acquisition module simultaneously acquires home environment images from different viewing angles through a binocular camera to obtain home environment image pair data and acquires camera parameter data of the binocular camera;

[0006] The image denoising module combines the fractional domain transformation technology with the convolutional neural network technology to construct a fractional domain enhanced convolution denoising network, and uses the fractional domain enhanced convolution denoising network to denoise the home environment image pair data to obtain denoised image pair data;

[0007] The image enhancement module combines the generative adversarial network technology with the improved loss function calculation method to construct an improved version of the generative adversarial network, and uses the improved version of the generative adversarial network to enhance the denoised image pair data to obtain enhanced image pair data. The improved version of the generative adversarial network includes generator enhancement, discriminator confrontation and loss calculation;

[0008] The 3D image generation module performs stereo correction on the enhanced image pair data to obtain the corrected image pair data, and then uses the multi-scale fusion Census cost enhancement SGM algorithm to calculate the disparity map between the corrected image pair data, and constructs a three-dimensional model of the home environment according to the disparity map and camera parameter data;

[0009] The augmented reality processing module integrates the three-dimensional model of the home environment with the real scene, and users can view virtual furniture, wall colors and decorations through AR devices;

[0010] The driving control module coordinates the work of the 3D image generation module and the augmented reality processing module.

[0011] Furthermore, the image denoising module uses a fractional domain enhanced convolutional denoising network to denoise the home environment image pair data to obtain a denoised image pair data, which specifically includes the following steps:

[0012] Step N1: constructing a model: constructing a fractional domain enhanced convolutional denoising network model, receiving home environment image pair data as input data of the fractional domain enhanced convolutional denoising network model;

[0013] Step N2: Perform convolution operation: Perform convolution operation on the home environment image data to extract basic features of the image, including edge, texture and simple structure information, and obtain shallow feature data;

[0014] Step N3: Apply fractional frequency filtering: Combine fractional calculus and Fourier transform to obtain a fractional frequency filter. The fractional frequency filter is used to capture important frequency information of shallow feature data and obtain frequency feature data. The formula used is as follows:

[0015] ;

[0016] in, represents frequency characteristic data, represents a fractional-order frequency filter, represents the order of the fractional order, Represents shallow feature data;

[0017] Step N4: Extract deep features: gradually extract complex image features of frequency feature data through six layers of convolution blocks, and output deep feature data. Each layer of convolution blocks includes convolution operation, batch normalization, FReLU activation function and skip connection mechanism;

[0018] Step N41: performing a convolution operation: receiving frequency feature data, performing a local convolution operation through a convolution kernel, extracting complex image features from the frequency feature data, and obtaining preliminary convolution feature data;

[0019] Step N42: batch normalization: performing batch normalization processing on the preliminary convolution feature data to eliminate the numerical differences between different channels and obtain normalized feature data;

[0020] Step N43: Applying the FReLU activation function: performing nonlinear processing on the normalized feature data through the FReLU activation function, thereby enhancing the network's ability to represent complex nonlinear features in the image and obtaining nonlinear feature data;

[0021] Step N44: Implement a skip connection mechanism: add the frequency feature data to the nonlinear feature data, retain key information, and enhance the representation capability of deep features to obtain deep feature data.

[0022] Step N5: Capturing directional features: Combining fractional calculus and directional filters, a fractional directional filter is obtained. The feature information in different directions is captured by the fractional directional filter on the deep feature data to obtain enhanced feature data. The formula used is as follows:

[0023] ;

[0024] in, represents enhanced feature data, represents a fractional-order directional filter, Represents deep feature data;

[0025] Step N6: Reconstruct the image: reconstruct the enhanced feature data and convert it into a denoised high-definition image to generate denoised image pair data.

[0026] Furthermore, the generator enhancement in the image enhancement module specifically includes: inputting the denoised image pair data into the generator network, performing feature extraction and enhancement processing, and obtaining initial enhanced image data.

[0027] Furthermore, the discriminator adversarial training in the image enhancement module specifically includes: inputting the initial enhanced image data into the discriminator network, continuously optimizing the generator network through adversarial training between the generator network and the discriminator network, and finally generating enhanced image pair data.

[0028] Furthermore, the loss calculation in the image enhancement module specifically includes: constructing a comprehensive loss function using a combination of MSE loss, SSIM loss, adversarial loss and regularization loss, and guiding the optimization of the generator network through the comprehensive loss function.

[0029] Furthermore, the process of constructing a comprehensive loss function using a combination of MSE loss, perceptual loss, adversarial loss and regularization loss specifically includes the following steps:

[0030] Step S1: Initialize the weight parameters of MSE loss, SSIM loss, adversarial loss and regularization loss;

[0031] Step S2: During the iteration process, the loss values ​​of the MSE loss, SSIM loss, adversarial loss and regularization loss of the current round are calculated, and the loss change rate is calculated by comparing the loss values ​​of the MSE loss, SSIM loss, adversarial loss and regularization loss of the previous round. The weight parameter is dynamically adjusted according to the loss change rate to obtain the adjusted weight parameter. The formula used is as follows:

[0032] ;

[0033] in, Indicates The loss item is The weight parameter at the iteration, Indicates The loss item is The weight parameter at the iteration, Indicates The loss item is The loss value of the iteration, Indicates The loss item is The loss value of the iteration, Represents the total number of loss items, there are four of them are MSE loss, perceptual loss, adversarial loss and regularization loss, represents the loss term index, Indicates all The sum of the loss changes of the loss items in the current round and the previous round;

[0034] Step S3: Recombine according to the adjusted weight parameters to construct a comprehensive loss function.

[0035] ;

[0036] in, represents the comprehensive loss function, represents the MSE loss weight, represents the MSE loss term; represents the SSIM loss weight, represents the SSIM loss term; represents the adversarial loss weight, against loss terms; represents the weight of the regularization loss, represents the regularization loss term;

[0037] Furthermore, in the 3D image generation module, the process of calculating the disparity map between the corrected image pair data by the multi-scale fusion Census cost enhancement SGM algorithm specifically includes the following steps:

[0038] Step T1: Multi-scale Census feature extraction layer: Perform Census transformation on the corrected image data, extract Census features of pixel neighborhoods at three scales of 3×3, 5×5, and 7×7, and generate multi-scale Census feature data;

[0039] Step T2: Multi-scale feature fusion layer: Multi-scale Census feature data is weightedly fused to capture detail information at different levels and obtain fused Census feature data. The formula used is as follows:

[0040] ;

[0041] in, Indicates the fusion of Census feature data. Represents 3×3 scale Census feature data, Represents 5×5 scale Census feature data, Represents 7×7 scale Census feature data, , and The weighting coefficient representing the scale feature;

[0042] Step T3: Cost calculation layer: Use the Hamming distance to calculate the matching cost of the fused Census feature data to obtain the matching cost data;

[0043] Step T4: SGM cost aggregation layer: Use the SGM algorithm to globally optimize the matching cost data in the horizontal, vertical and diagonal directions, accumulate the matching costs through dynamic programming, and obtain the aggregated cost data;

[0044] Step T41: Horizontal cost aggregation: firstly, the matching cost data is traversed horizontally, and accumulated from left to right and from right to left to obtain horizontal cost aggregation data;

[0045] Step T42: vertical cost aggregation: then perform traversal processing in the vertical direction, perform accumulation calculations from top to bottom and from bottom to top, and obtain vertical cost aggregation data;

[0046] Step T43: Diagonal cost aggregation: Finally, perform a diagonal traversal, accumulate costs from the upper left to the lower right and from the upper right to the lower left, and obtain diagonal cost aggregation data;

[0047] Step T44: Dynamic programming optimization: accumulate the horizontal cost aggregation data, vertical cost aggregation data and diagonal cost aggregation data to obtain cost aggregation results, use dynamic programming technology to globally optimize the cost aggregation results, eliminate path conflicts, and obtain aggregated cost data.

[0048] Step T41: Horizontal cost aggregation: firstly, the matching cost data is traversed horizontally, and accumulated from left to right and from right to left to obtain horizontal cost aggregation data;

[0049] Step T42: vertical cost aggregation: then perform traversal processing in the vertical direction, perform accumulation calculations from top to bottom and from bottom to top, and obtain vertical cost aggregation data;

[0050] Step T43: Diagonal cost aggregation: Finally, perform a diagonal traversal, accumulate costs from the upper left to the lower right and from the upper right to the lower left, and obtain diagonal cost aggregation data;

[0051] Step T44: Dynamic programming optimization: accumulate the horizontal cost aggregation data, vertical cost aggregation data and diagonal cost aggregation data to obtain cost aggregation results, use dynamic programming technology to globally optimize the cost aggregation results, eliminate path conflicts, and obtain aggregated cost data.

[0052] Step T5: Disparity selection layer: Select the minimum matching cost from the aggregated cost data, determine the corresponding disparity value, and generate a disparity map.

[0053] Furthermore, the denoised image pair data is input into the generator network for feature extraction and enhancement processing to obtain the initial enhanced image data, which specifically includes the following steps:

[0054] Step B1: Extracting preliminary features: extracting preliminary features from the denoised image data to obtain preliminary feature data;

[0055] Step B2: Apply the parameterized ReLU activation function: perform nonlinear activation on the preliminary feature data to give the network stronger nonlinear expression capabilities and obtain the activated feature data;

[0056] Step B3: perform E-Sum accumulation: the activated feature data is processed through six residual blocks and accumulated through the E-Sum module to achieve feature fusion and obtain fused feature data;

[0057] Step B4: performing upsampling processing: performing upsampling processing on the fused feature data to improve the resolution of the image and obtain high-resolution feature data;

[0058] Step B5: performing downsampling processing: performing downsampling processing on the high-resolution feature data to further enhance the image details and obtain enhanced feature data;

[0059] Step B6: Generate initial enhanced image data: Generate initial enhanced image data by passing the enhanced feature data through a convolutional layer.

[0060] By adopting the above scheme, the beneficial effects achieved by the present invention are as follows:

[0061] The present invention provides an intelligent mainboard display driving system based on 3D display and augmented reality. The fractional domain enhanced convolution denoising network that introduces the fractional domain transformation technology and the convolutional neural network technology is introduced to significantly improve the image denoising ability. The existing denoising methods have poor ability to retain image details and are prone to cause loss of image details. Especially in complex environments, the noise processing effect is relatively limited. The present invention extracts frequency domain features in the image through the fractional domain transformation technology, combined with the powerful feature extraction capability of the convolutional neural network, which not only effectively removes the noise in the image, but also greatly enhances the image detail retention capability, and ultimately makes the decoration effect displayed on the AR device clearer, and the user can view the decoration design details more accurately, thereby improving the user experience.

[0062] The present invention also adopts an improved generative adversarial network technology, and realizes the improvement of the image enhancement process by optimizing the loss function calculation method; most of the existing image enhancement methods rely on simple loss functions, which make it difficult to balance the details and global quality at the same time, resulting in the generated images being visually unclear; the present invention improves the accuracy of image enhancement by the mutual confrontation between the generator and the discriminator of the generative adversarial network, and combines the comprehensive calculation method of MSE loss, SSIM loss, adversarial loss and regularization loss to optimize the global and detailed expressiveness of the image, ensuring that users obtain a more realistic and delicate visual experience when viewing the decoration effect through AR devices.

[0063] In addition, in terms of 3D image generation, the present invention adopts a multi-scale fused Census cost enhanced SGM algorithm, which greatly improves the accuracy of disparity map calculation; traditional disparity map calculation methods have weak ability to capture details when processing complex scenes, which easily leads to large errors in 3D models and affects the overall display effect; the present invention extracts and fuses Census features at multiple scales, combines the optimized SGM algorithm, and globally optimizes the disparity map to generate a more accurate 3D model; the application of this technology not only solves the shortcomings of traditional methods in disparity calculation, but also enhances the three-dimensional display capability of AR devices for decoration effects, provides a more realistic sense of space, and further improves the visualization effect of decoration design and the decision-making efficiency of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a schematic diagram of a module of an intelligent mainboard display driving system based on 3D display and augmented reality proposed by the present invention;

[0065] Figure 2 This is a network architecture diagram of the image denoising module proposed in the present invention. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0067] Embodiment 1, according to Figure 1 , the present invention provides an intelligent mainboard display driving system based on 3D display and reality enhancement, the system comprising: an image acquisition module, an image denoising module, an image enhancement module, a 3D image generation module, an augmented reality processing module and a driving control module;

[0068] The image acquisition module simultaneously acquires home environment images from different viewing angles through a binocular camera to obtain home environment image pair data and acquires camera parameter data of the binocular camera;

[0069] The image denoising module combines the fractional domain transformation technology with the convolutional neural network technology to construct a fractional domain enhanced convolution denoising network, and uses the fractional domain enhanced convolution denoising network to denoise the home environment image pair data to obtain denoised image pair data;

[0070] The image enhancement module combines the generative adversarial network technology with the improved loss function calculation method to construct an improved version of the generative adversarial network, and uses the improved version of the generative adversarial network to enhance the denoised image pair data to obtain enhanced image pair data. The improved version of the generative adversarial network includes generator enhancement, discriminator confrontation and loss calculation;

[0071] The 3D image generation module performs stereo correction on the enhanced image pair data to obtain the corrected image pair data, and then uses the multi-scale fusion Census cost enhancement SGM algorithm to calculate the disparity map between the corrected image pair data, and constructs a three-dimensional model of the home environment based on the disparity map and camera parameter data.

[0072] The augmented reality processing module integrates the three-dimensional model of the home environment with the real scene, and users can view virtual furniture, wall colors and decorations through AR devices;

[0073] The driving control module coordinates the work of the 3D image generation module and the augmented reality processing module.

[0074] Embodiment 2, according to Figure 2 This embodiment is based on the above embodiment. The image denoising module uses a fractional domain enhanced convolutional denoising network to denoise the home environment image pair data to obtain the denoised image pair data. The process specifically includes the following steps:

[0075] Step N1: constructing a model: constructing a fractional domain enhanced convolutional denoising network model, receiving home environment image pair data as input data of the fractional domain enhanced convolutional denoising network model;

[0076] Step N2: Perform convolution operation: Perform convolution operation on the home environment image data to extract basic features of the image, including edge, texture and simple structure information, and obtain shallow feature data;

[0077] Step N3: Apply fractional frequency filtering: Combine fractional calculus and Fourier transform to obtain a fractional frequency filter. The fractional frequency filter is used to capture important frequency information of shallow feature data and obtain frequency feature data. The formula used is as follows:

[0078] ;

[0079] in, represents frequency characteristic data, represents a fractional-order frequency filter, represents the order of the fractional order, Represents shallow feature data;

[0080] Step N4: Extract deep features: gradually extract complex image features of frequency feature data through six layers of convolution blocks, and output deep feature data. Each layer of convolution blocks includes convolution operation, batch normalization, FReLU activation function and skip connection mechanism;

[0081] Step N41: performing a convolution operation: receiving frequency feature data, performing a local convolution operation through a convolution kernel, extracting complex image features from the frequency feature data, and obtaining preliminary convolution feature data;

[0082] Step N42: batch normalization: performing batch normalization processing on the preliminary convolution feature data to eliminate the numerical differences between different channels and obtain normalized feature data;

[0083] Step N43: Applying the FReLU activation function: performing nonlinear processing on the normalized feature data through the FReLU activation function, thereby enhancing the network's ability to represent complex nonlinear features in the image and obtaining nonlinear feature data;

[0084] Step N44: Implement a skip connection mechanism: add the frequency feature data to the nonlinear feature data, retain key information, and enhance the representation capability of deep features to obtain deep feature data.

[0085] Step N5: Capturing directional features: Combining fractional calculus and directional filters, a fractional directional filter is obtained. The feature information in different directions is captured by the fractional directional filter on the deep feature data to obtain enhanced feature data. The formula used is as follows:

[0086] ;

[0087] in, represents enhanced feature data, represents a fractional-order directional filter, Represents deep feature data;

[0088] Step N6: Reconstruct the image: reconstruct the enhanced feature data and convert it into a denoised high-definition image to generate denoised image pair data.

[0089] Embodiment 3: This embodiment is based on the above embodiment, and the generator enhancement in the image enhancement module specifically includes: inputting the denoised image pair data into the generator network, performing feature extraction and enhancement processing, and obtaining initial enhanced image data.

[0090] Embodiment 4, this embodiment is based on the above embodiment, and the discriminator confrontation in the image enhancement module specifically includes: inputting the initial enhanced image data into the discriminator network, continuously optimizing the generator network through adversarial training of the generator network and the discriminator network, and finally generating enhanced image pair data.

[0091] Embodiment 5, this embodiment is based on the above embodiment, and the loss calculation in the image enhancement module specifically includes: using a combination of MSE loss, SSIM loss, adversarial loss and regularization loss to construct a comprehensive loss function, and guiding the optimization of the generator network through the comprehensive loss function.

[0092] Embodiment 6, based on the above embodiment, the process of constructing a comprehensive loss function using a combination of MSE loss, perceptual loss, adversarial loss and regularization loss specifically includes the following steps:

[0093] Step S1: Initialize the weight parameters of MSE loss, SSIM loss, adversarial loss and regularization loss;

[0094] Step S2: During the iteration process, the loss values ​​of the MSE loss, SSIM loss, adversarial loss and regularization loss of the current round are calculated, and the loss change rate is calculated by comparing the loss values ​​of the MSE loss, SSIM loss, adversarial loss and regularization loss of the previous round. The weight parameter is dynamically adjusted according to the loss change rate to obtain the adjusted weight parameter. The formula used is as follows:

[0095] ;

[0096] in, Indicates The loss item is The weight parameter at the iteration, Indicates The loss item is The weight parameter at the iteration, Indicates The loss item is The loss value of the iteration, Indicates The loss item is The loss value of the iteration, Represents the total number of loss items, there are four of them are MSE loss, perceptual loss, adversarial loss and regularization loss, represents the loss term index, Indicates all The sum of the loss changes of the loss items in the current round and the previous round;

[0097] Step S3: Recombine according to the adjusted weight parameters to construct a comprehensive loss function.

[0098] ;

[0099] in, represents the comprehensive loss function, represents the MSE loss weight, represents the MSE loss term; represents the SSIM loss weight, represents the SSIM loss term; represents the adversarial loss weight, against loss terms; represents the weight of the regularization loss, represents the regularization loss term;

[0100] Embodiment 7, based on the above embodiment, in the 3D image generation module, the multi-scale fusion Census cost enhancement SGM algorithm calculates the disparity map between the corrected image pair data, specifically comprising the following steps:

[0101] Step T1: Multi-scale Census feature extraction layer: Perform Census transformation on the corrected image data, extract Census features of pixel neighborhoods at three scales of 3×3, 5×5, and 7×7, and generate multi-scale Census feature data;

[0102] Step T2: Multi-scale feature fusion layer: Multi-scale Census feature data is weightedly fused to capture detail information at different levels and obtain fused Census feature data. The formula used is as follows:

[0103] ;

[0104] in, Indicates the fusion of Census feature data. Represents 3×3 scale Census feature data, Represents 5×5 scale Census feature data, Represents 7×7 scale Census feature data, , and The weighting coefficient representing the scale feature;

[0105] Step T3: Cost calculation layer: Use the Hamming distance to calculate the matching cost of the fused Census feature data to obtain the matching cost data;

[0106] Step T4: SGM cost aggregation layer: Use the SGM algorithm to globally optimize the matching cost data in the horizontal, vertical and diagonal directions, accumulate the matching costs through dynamic programming, and obtain the aggregated cost data;

[0107] Step T41: Horizontal cost aggregation: firstly, the matching cost data is traversed horizontally, and accumulated from left to right and from right to left to obtain horizontal cost aggregation data;

[0108] Step T42: vertical cost aggregation: then perform traversal processing in the vertical direction, perform accumulation calculations from top to bottom and from bottom to top, and obtain vertical cost aggregation data;

[0109] Step T43: Diagonal cost aggregation: Finally, perform a diagonal traversal, accumulate costs from the upper left to the lower right and from the upper right to the lower left, and obtain diagonal cost aggregation data;

[0110] Step T44: Dynamic programming optimization: accumulate the horizontal cost aggregation data, vertical cost aggregation data and diagonal cost aggregation data to obtain cost aggregation results, use dynamic programming technology to globally optimize the cost aggregation results, eliminate path conflicts, and obtain aggregated cost data.

[0111] Step T5: Disparity selection layer: Select the minimum matching cost from the aggregated cost data, determine the corresponding disparity value, and generate a disparity map.

[0112] Embodiment 8, based on the above embodiment, this embodiment inputs the denoised image pair data into the generator network, performs feature extraction and enhancement processing, and obtains the process of initial enhanced image data, which specifically includes the following steps:

[0113] Step B1: Extracting preliminary features: extracting preliminary features from the denoised image data to obtain preliminary feature data;

[0114] Step B2: Apply the parameterized ReLU activation function: perform nonlinear activation on the preliminary feature data to give the network stronger nonlinear expression capabilities and obtain the activated feature data;

[0115] Step B3: perform E-Sum accumulation: the activated feature data is processed through six residual blocks and accumulated through the E-Sum module to achieve feature fusion and obtain fused feature data;

[0116] Step B4: performing upsampling processing: performing upsampling processing on the fused feature data to improve the resolution of the image and obtain high-resolution feature data;

[0117] Step B5: performing downsampling processing: performing downsampling processing on the high-resolution feature data to further enhance the image details and obtain enhanced feature data;

[0118] Step B6: Generate initial enhanced image data: Generate initial enhanced image data by passing the enhanced feature data through a convolutional layer.

[0119] Embodiment 9, based on the above embodiment, the augmented reality processing module integrates the three-dimensional model of the home environment with the real scene, and the user can view virtual furniture, wall color and decorations through the AR device;

[0120] In this embodiment, the user uses the AR device to place a virtual sofa, coffee table, and bookshelf in a blank area of ​​the living room, and adjusts the position and size of the furniture through drag gestures to view their visual effects and spatial layout in different positions; selects pink wall color, and views the effect of the pink wall color applied to the real living room wall in the AR device; and hangs virtual picture frames and wall clocks on the wall.

Claims

1. An intelligent motherboard display driving system based on 3D display and augmented reality, characterized in that: It includes image acquisition module, image denoising module, image enhancement module and 3D image generation module; The image acquisition module simultaneously acquires home environment images from different viewing angles through a binocular camera to obtain home environment image pair data and acquires camera parameter data of the binocular camera; The image denoising module combines the fractional domain transformation technology with the convolutional neural network technology to construct a fractional domain enhanced convolution denoising network, and uses the fractional domain enhanced convolution denoising network to denoise the home environment image pair data to obtain denoised image pair data; The image enhancement module combines the generative adversarial network technology with the improved loss function calculation method to construct an improved version of the generative adversarial network, and uses the improved version of the generative adversarial network to enhance the denoised image pair data to obtain enhanced image pair data. The improved version of the generative adversarial network includes generator enhancement, discriminator confrontation and loss calculation; The 3D image generation module performs stereo correction on the enhanced image pair data to obtain the corrected image pair data, and then uses the multi-scale fusion Census cost enhancement SGM algorithm to calculate the disparity map between the corrected image pair data, and constructs a three-dimensional model of the home environment based on the disparity map and camera parameter data.

2. The intelligent motherboard display driving system based on 3D display and augmented reality according to claim 1, characterized in that: The image denoising module uses a fractional domain enhanced convolutional denoising network to denoise the home environment image pair data to obtain the denoised image pair data, which specifically includes the following steps: Step N1: constructing a model: constructing a fractional domain enhanced convolutional denoising network model, receiving home environment image pair data as input data of the fractional domain enhanced convolutional denoising network model; Step N2: Perform convolution operation: Perform convolution operation on the home environment image data to extract basic features of the image, including edge, texture and simple structure information, and obtain shallow feature data; Step N3: Apply fractional frequency filtering: Combine fractional calculus and Fourier transform to obtain a fractional frequency filter, and use the fractional frequency filter to capture important frequency information of shallow feature data to obtain frequency feature data; Step N4: Extract deep features: gradually extract complex image features of frequency feature data through six layers of convolution blocks, and output deep feature data. Each layer of convolution blocks includes convolution operation, batch normalization, FReLU activation function and skip connection mechanism; Step N5: Capturing directional features: combining fractional calculus and directional filters to obtain fractional directional filters, and using fractional directional filters to capture feature information in different directions on deep feature data to obtain enhanced feature data; Step N6: Reconstruct the image: reconstruct the enhanced feature data and convert it into a denoised high-definition image to generate denoised image pair data.

3. The intelligent motherboard display driving system based on 3D display and augmented reality according to claim 1, characterized in that: The generator enhancement in the image enhancement module specifically includes: inputting the denoised image pair data into the generator network, performing feature extraction and enhancement processing, and obtaining initial enhanced image data.

4. The intelligent motherboard display driving system based on 3D display and augmented reality according to claim 3, characterized in that: The discriminator confrontation in the image enhancement module specifically includes: inputting the initial enhanced image data into the discriminator network, continuously optimizing the generator network through adversarial training between the generator network and the discriminator network, and finally generating enhanced image pair data.

5. The intelligent motherboard display driving system based on 3D display and augmented reality according to claim 4, characterized in that: The loss calculation in the image enhancement module specifically includes: constructing a comprehensive loss function using a combination of MSE loss, SSIM loss, adversarial loss and regularization loss, and guiding the optimization of the generator network through the comprehensive loss function.

6. The intelligent motherboard display driving system based on 3D display and augmented reality according to claim 5, characterized in that: The process of constructing a comprehensive loss function using a combination of MSE loss, perceptual loss, adversarial loss and regularization loss specifically includes the following steps: Step S1: Initialize the weight parameters of MSE loss, SSIM loss, adversarial loss and regularization loss; Step S2: During the iteration process, the loss values ​​of the MSE loss, SSIM loss, adversarial loss and regularization loss of the current round are calculated, and the loss change rate is calculated by comparing the loss values ​​of the MSE loss, SSIM loss, adversarial loss and regularization loss of the previous round. The weight parameters are dynamically adjusted according to the loss change rate to obtain the adjusted weight parameters; Step S3: Recombine according to the adjusted weight parameters to construct a comprehensive loss function.

7. The intelligent motherboard display driving system based on 3D display and augmented reality according to claim 1, characterized in that: In the 3D image generation module, the process of calculating the disparity map between the corrected image pair data by the multi-scale fusion Census cost enhancement SGM algorithm specifically includes the following steps: Step T1: Multi-scale Census feature extraction layer: Perform Census transformation on the corrected image data, extract Census features of pixel neighborhoods at three scales of 3×3, 5×5, and 7×7, and generate multi-scale Census feature data; Step T2: Multi-scale feature fusion layer: weighted fusion of multi-scale Census feature data to capture detail information at different levels and obtain fused Census feature data; Step T3: Cost calculation layer: Use the Hamming distance to calculate the matching cost of the fused Census feature data to obtain the matching cost data; Step T4: SGM cost aggregation layer: Use the SGM algorithm to globally optimize the matching cost data in the horizontal, vertical and diagonal directions, accumulate the matching costs through dynamic programming, and obtain the aggregated cost data; Step T5: Disparity selection layer: Select the minimum matching cost from the aggregated cost data, determine the corresponding disparity value, and generate a disparity map.

Images