Sublimation-based high-reliability light field super-resolution edge calculation method and system

Abstract

The invention relates to the technical field of data processing, and discloses a mercuric chloride-based high-reliability light field super-resolution edge calculation method and a mercuric chloride-based high-reliability light field super-resolution edge calculation system. The method comprises the following steps: collecting RAW data through a light field camera and carrying out DnCNN denoising; using Transform-CNN (Convolutional Neural Network) to calibrate and generate an error coefficient and a correction matrix; the FPGA accelerates the OfficientNet and processes the OfficientNet and the FFT to output a score and a frequency domain result; the mercuration NPU executes ESRGAN to form a deep learning image; performing U-Net fusion to obtain an 8K image; and carrying out quality evaluation and dynamically allocating resources to construct a redundancy mechanism. According to the method, the high-reliability light field super-resolution edge calculation method based on mercuric chloride is constructed, innovative mechanisms such as adaptive quality evaluation, dual-path super-resolution processing and intelligent fusion, heterogeneous resource dynamic scheduling and the like are integrated, and the reliability and adaptability of light field super-resolution processing in the edge environment are effectively improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a high-reliability light field super-resolution edge computing method and system based on Ascend. Background Art

[0002] In recent years, with the rapid development of computer vision, virtual reality, autonomous driving and other fields, the demand for high-quality images has been growing. As a new imaging technology, light field cameras can simultaneously record the direction and intensity information of light in the scene, providing a rich data foundation for subsequent image processing. Compared with traditional two-dimensional images, the four-dimensional light field data (spatial coordinates and angular coordinates) captured by light field imaging technology provides a richer scene representation, making three-dimensional scene reconstruction and depth perception possible. Light field super-resolution reconstruction technology aims to use these rich light field data to break through the resolution limitations of traditional imaging systems and obtain higher-definition images. It has broad application prospects in medical imaging, security monitoring, industrial detection and other fields. The development routes of related technologies include traditional methods (based on frequency domain processing, interpolation and regularization) and deep learning methods (convolutional neural networks, generative adversarial networks, etc.), but most research focuses on cloud processing environments.

[0003] However, existing light field super-resolution technologies face many challenges in edge computing environments. First, light field data is huge, and traditional cloud processing models bring high network latency and bandwidth pressure; second, edge device computing resources are limited, making it difficult to directly deploy complex deep learning models; third, the quality of light field data varies greatly in different scenarios, and a single processing algorithm is difficult to adapt to diverse application scenarios; fourth, existing methods generally lack a reliability guarantee mechanism and cannot cope with common problems in edge scenarios such as data anomalies and equipment overloads; finally, existing technologies mostly take processing quality as the only goal, ignoring comprehensive performance requirements such as real-time, stability, and resource efficiency in actual applications. Especially in the context of the increasing popularity of new AI chips such as Ascend, how to give full play to the advantages of heterogeneous computing resources and build a highly reliable edge light field super-resolution computing framework has become a key technical issue that needs to be solved urgently. Summary of the invention

[0004] The present application provides a high-reliability light field super-resolution edge computing method and system based on Ascend, which is used to construct a high-reliability light field super-resolution edge computing method based on Ascend, integrating innovative mechanisms such as adaptive quality assessment, dual-path super-resolution processing and intelligent fusion, and dynamic scheduling of heterogeneous resources, thereby effectively improving the reliability and adaptability of light field super-resolution processing in edge environments.

[0005] In the first aspect, the present application provides a high-reliability light field super-resolution edge computing method based on Ascend, and the high-reliability light field super-resolution edge computing method based on Ascend includes: collecting RAW format light field data through a light field camera, performing DnCNN denoising processing on the light field data, and obtaining pre-processed data in HDF5 format; based on the pre-processed data, calibrating through a Transformer-CNN model to generate a depth error compensation coefficient and a distortion correction matrix; using the calibrated light field data, through FPGA acceleration EfficientNet model and FFT processing, outputting a reliability score and a frequency domain super-resolution result; based on the reliability score and the frequency domain super-resolution result, the Ascend NPU executes ESRGAN model calculation to form a deep learning super-resolution image; the frequency domain super-resolution result and the deep learning super-resolution image are weighted by quality score through a U-Net fusion network to obtain 8K light field super-resolution imaging; PSNR and SSIM evaluation are performed on the 8K light field super-resolution imaging, and the task scheduling module dynamically allocates Ascend NPU and FPGA resources to build an edge computing redundancy mechanism.

[0006] In a second aspect, the present application provides a high-reliability light field super-resolution edge computing system based on Ascend, and the high-reliability light field super-resolution edge computing system based on Ascend includes: A denoising module is used to collect RAW format light field data through a light field camera, perform DnCNN denoising on the light field data, and obtain pre-processed data in HDF5 format; A calibration module, used to calibrate the preprocessed data through a Transformer-CNN model to generate a depth error compensation coefficient and a distortion correction matrix; The processing module is used to use the calibrated light field data, accelerate the EfficientNet model and FFT processing through FPGA, and output the reliability score and frequency domain super-resolution results; A computing module, configured to execute ESRGAN model calculation by the Ascend NPU based on the reliability score and the frequency domain super-resolution result to form a deep learning super-resolution image; A weighting module is used to weight the frequency domain super-resolution result and the deep learning super-resolution image through a U-Net fusion network to obtain 8K light field super-resolution imaging; The allocation module is used to perform PSNR and SSIM evaluation on the super-resolution imaging of the 8K light field. The task scheduling module dynamically allocates Ascend NPU and FPGA resources to build an edge computing redundancy mechanism.

[0007] A third aspect of the present invention provides a computer device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the computer device executes the above-mentioned Ascend-based high-reliability light field super-resolution edge computing method.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the above-mentioned Ascend-based high-reliability light field super-resolution edge computing method.

[0009] In the technical solution provided by the present application, the present invention collects RAW format light field data through a light field camera and performs DnCNN denoising processing to obtain structured HDF5 format pre-processed data, thereby ensuring the data quality and consistency of subsequent processing; the pre-processed data is calibrated based on the Transformer-CNN hybrid architecture model to generate accurate depth error compensation coefficients and distortion correction matrices, effectively eliminating systematic errors in the light field imaging process; the EfficientNet model and FFT processing technology accelerated by FPGA are used to achieve hardware-level parallel processing capabilities, greatly improving the real-time performance of quality assessment and frequency domain processing, and outputting The reliability score and frequency domain super-resolution results provide double guarantees for subsequent processing; the ESRGAN model calculation is performed based on the Ascend NPU, which fully utilizes the advantages of NPU in deep learning reasoning to form a deep learning super-resolution image with rich texture details; the frequency domain results and deep learning images are innovatively weighted by the U-Net fusion network for quality scoring, which realizes the complementary enhancement of the advantages of different processing paths and solves the problem that a single algorithm is difficult to adapt to complex scenes; the PSNR, SSIM evaluation and task scheduling module dynamic resource allocation strategy are adopted to build a complete edge computing redundancy mechanism, which significantly improves the fault tolerance and stability of the system. The present invention has made many innovations in the algorithm design level according to the characteristics of light field data: the DnCNN denoising model optimizes the noise distribution characteristics of light field images and effectively retains the angle information; the Transformer-CNN hybrid architecture gives full play to the complementary advantages of Transformer in global relationship modeling and CNN in local feature extraction; the dynamic loss function adjustment mechanism based on reliability scoring enables the ESRGAN model to adaptively balance fidelity and perceptual quality according to data quality; the channel attention mechanism of the U-Net fusion network realizes the optimal fusion of frequency domain processing and deep learning results. These algorithmic features work together to enable the system to automatically select the best processing strategy for light field data of different qualities in resource-constrained edge environments, significantly improving the quality consistency of super-resolution results. Overall, the present invention not only overcomes the problems of resource limitations and insufficient reliability in edge environments in the prior art, but also achieves high reliability, high adaptability and high efficiency of light field super-resolution systems through the collaborative optimization of heterogeneous computing resources and the dynamic adjustment of intelligent processing strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0011] Figure 1 A schematic diagram of an embodiment of a high-reliability light field super-resolution edge computing method based on Ascend in an embodiment of the present application; Figure 2 A schematic diagram of an embodiment of a high-reliability light field super-resolution edge computing system based on Ascend in an embodiment of the present application; Figure 3 It is a schematic block diagram of the structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The embodiments of the present application provide a highly reliable light field super-resolution edge computing method and system based on Ascend. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0013] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of a high-reliability light field super-resolution edge computing method based on Ascend in the embodiments of the present application includes: Step S101, collecting RAW format light field data through a light field camera, performing DnCNN denoising processing on the light field data, and obtaining pre-processed data in HDF5 format; Step S102: Calibrate the preprocessed data using the Transformer-CNN model to generate a depth error compensation coefficient and a distortion correction matrix; Step S103, using the calibrated light field data, the FPGA accelerates the EfficientNet model and FFT processing, and outputs the reliability score and frequency domain super-resolution results; Step S104: Based on the reliability score and the frequency domain super-resolution result, the Ascend NPU performs ESRGAN model calculation to form a deep learning super-resolution image; Step S105: The frequency domain super-resolution result and the deep learning super-resolution image are weighted by quality scoring through a U-Net fusion network to obtain 8K light field super-resolution imaging; Step S106: Perform PSNR and SSIM evaluation on the super-resolution imaging of the 8K light field, and dynamically allocate Ascend NPU and FPGA resources by the task scheduling module to build an edge computing redundancy mechanism.

[0014] It is understandable that the execution subject of this application can be a high-reliability light field super-resolution edge computing system based on Ascend, or a terminal or a server, which is not limited here. The embodiment of this application is described by taking the server as the execution subject as an example.

[0015] Specifically, RAW format light field data is collected through high-precision light field cameras such as Lytro or Raytrix series, which contain multi-view and depth information with 4K resolution. During the collection process, the operating environment parameters of the light field camera are monitored. When the temperature exceeds 60°C or the power fluctuation exceeds ±5%, resulting in a signal-to-noise ratio of less than 20dB, it will automatically switch to the backup light field camera to ensure the continuity of data input. The collected RAW format light field data is then transmitted to the edge server based on Huawei Atlas 500, and the DnCNN denoising process is performed by the RK3588 edge computing chip. DnCNN is a deep convolutional neural network that is specially optimized for the noise characteristics of light field images, and the single-frame processing delay is controlled within 10ms. During the processing process, abnormal sub-images with a view angle offset of more than 2 pixels are screened out based on the epipolar constraint rule, and low-quality areas are filtered out by a depth confidence threshold of 0.8. After the processing, the light field data will be attached with metadata tags such as dynamic range and contrast statistics, and finally encapsulated into HDF5 format pre-processed data.

[0016] Calibration is performed based on the preprocessed data in HDF5 format. Here, the preprocessed data is parsed into a sub-aperture image sequence of an N×N array to form a multi-view representation of the light field. The light field parameters are then calibrated using a hybrid architecture Transformer-CNN model, where the Vision Transformer branch is responsible for global perspective relationship analysis and extracting spatial consistency features; while the ResNet-18 variant CNN branch focuses on local distortion positioning and identifying high-frequency distortion areas. After the two features are fused, a perspective correlation map is generated to calculate the depth error compensation coefficient and the distortion correction matrix. These two key parameters are used to correct the optical distortion and depth error in the light field data.

[0017] Using the calibrated light field data, dual-channel parallel processing is performed on the Xilinx UltraScale+ FPGA. First, the quality of the light field data is evaluated through the FPGA-accelerated EfficientNet-B3 feature extraction layer, which compresses the feature calculation time from 15ms on the CPU to 5ms. The quality evaluation builds a weighted adjustable scoring function based on PSNR, SSIM and edge sharpness indicators to calculate the reliability score. At the same time, the light field data is divided into image blocks of n×n pixels, and a fast Fourier transform (FFT) is performed to convert the spatial domain data into a frequency domain representation. Interpolation upsampling is performed in the frequency domain, and then reconstructed through an inverse FFT transform to generate a frequency domain super-resolution result. The dynamic resource control module inside the FPGA flexibly allocates computing resources according to the processing load. By default, 80% of the computing resources are used for quality assessment acceleration and 20% for frequency domain processing. The reliability score and the frequency domain super-resolution result are transmitted to the Ascend NPU processing unit to build a light field enhancement processing flow. An improved version of the ESRGAN network is deployed on the Ascend NPU, and a residual dense block structure is used to enhance the feature extraction capability. During the processing, the reliability score is integrated into the loss function calculation to form a weighted loss function that combines pixel loss, perceptual loss and adversarial loss. Multi-layer feature extraction is performed on the frequency domain results through residual dense blocks, the light field angle information is retained, and the phase consistency loss constraint algorithm is used to reduce texture artifacts in multi-view data, and finally a deep learning super-resolution image containing high-frequency details and accurate depth information is generated.

[0018] The frequency domain super-resolution results are fused with the deep learning super-resolution images. First, the high-frequency feature map is extracted from the frequency domain results, and the texture feature map is extracted from the deep learning image. Both are input into the fusion network with a lightweight U-Net structure. In the encoder stage of the fusion network, the key information is identified through the channel attention mechanism; at the same time, the weight coefficient λ is dynamically adjusted based on the reliability score. When the score is greater than 0.9, the weight of the deep learning branch is set to 0.8, and when the score is less than or equal to 0.6, the weight is reduced to 0.3, which depends more on the fidelity of the frequency domain results. The decoder of the fusion network reconstructs the processed features into 8K resolution light field super-resolution imaging.

[0019] Perform multi-dimensional quality assessment on super-resolution imaging, including PSNR assessment (threshold 32dB), SSIM assessment (threshold 0.92) and edge sharpness assessment (threshold 90). If the standard is not met, the recalculation process is triggered, and the calibrated data is called from the storage module to re-execute the super-resolution processing, and retry up to 3 times. At the same time, the task scheduling module dynamically allocates Ascend NPU and FPGA resources through the priority queue mechanism. In normal mode, 70% of the NPU computing power is allocated for real-time super-resolution inference and 30% for incremental model training. The system also continuously monitors the hardware operating parameters. When the temperature exceeds 85°C or the utilization rate exceeds 90%, the hardware redundancy switching mechanism is started to migrate the computing tasks to the backup computing nodes to build a complete edge computing redundancy mechanism.

[0020] For example, after collecting RAW format light field data of an indoor scene, DnCNN denoising processing increased the noise signal-to-noise ratio from the original 18dB to 28dB, and the epipolar constraint filtering removed about 8% of abnormal sub-images. After the Transformer-CNN model was calibrated, the distortion correction matrix corrected the radial distortion of 1.2 pixels in the edge area. In the FPGA acceleration processing stage, the quality assessment calculation obtained a reliability score of 0.85, and the frequency domain super-resolution processing retained high-frequency details but had a slight ringing effect. After the Ascend NPU executed the ESRGAN model, the generated deep learning super-resolution image performed well in texture details but was slightly blurred in the depth transition area. The U-Net fusion network set a weight coefficient of 0.7 based on the reliability score of 0.85, effectively combining the edge clarity of the frequency domain results with the texture richness of the deep learning results. Finally, the PSNR of 8K super-resolution imaging reached 33.5dB, the SSIM reached 0.94, and the edge sharpness reached 95, meeting the quality requirements without recalculation. The entire processing flow is completed on the edge device, and the total computing time is controlled within 200ms, giving full play to the efficiency of Ascend NPU in deep learning reasoning and the advantages of FPGA in parallel computing.

[0021] In the embodiment of the present application, the present invention collects RAW format light field data through a light field camera and performs DnCNN denoising processing to obtain structured HDF5 format pre-processed data, thereby ensuring the data quality and consistency of subsequent processing; the pre-processed data is calibrated based on the Transformer-CNN hybrid architecture model to generate accurate depth error compensation coefficients and distortion correction matrices, effectively eliminating systematic errors in the light field imaging process; the EfficientNet model and FFT processing technology accelerated by FPGA are used to achieve hardware-level parallel processing capabilities, greatly improving the real-time performance of quality assessment and frequency domain processing, and outputting The reliability score and frequency domain super-resolution results provide double guarantees for subsequent processing; the ESRGAN model calculation is performed based on the Ascend NPU, which fully utilizes the advantages of NPU in deep learning reasoning to form a deep learning super-resolution image with rich texture details; the frequency domain results and deep learning images are innovatively weighted by the U-Net fusion network for quality scoring, which realizes the complementary enhancement of the advantages of different processing paths and solves the problem that a single algorithm is difficult to adapt to complex scenes; the PSNR, SSIM evaluation and task scheduling module dynamic resource allocation strategy are adopted to build a complete edge computing redundancy mechanism, which significantly improves the fault tolerance and stability of the system. The present invention has made many innovations in the algorithm design level according to the characteristics of light field data: the DnCNN denoising model optimizes the noise distribution characteristics of light field images and effectively retains the angle information; the Transformer-CNN hybrid architecture gives full play to the complementary advantages of Transformer in global relationship modeling and CNN in local feature extraction; the dynamic loss function adjustment mechanism based on reliability scoring enables the ESRGAN model to adaptively balance fidelity and perceptual quality according to data quality; the channel attention mechanism of the U-Net fusion network realizes the optimal fusion of frequency domain processing and deep learning results. These algorithmic features work together to enable the system to automatically select the best processing strategy for light field data of different qualities in resource-constrained edge environments, significantly improving the quality consistency of super-resolution results. Overall, the present invention not only overcomes the problems of resource limitations and insufficient reliability in edge environments in the prior art, but also achieves high reliability, high adaptability and high efficiency of light field super-resolution systems through the collaborative optimization of heterogeneous computing resources and the dynamic adjustment of intelligent processing strategies.

[0022] In a specific embodiment, the process of executing step S101 may specifically include the following steps: (1) Collect 4K resolution RAW format light field data, and the sensor records the light direction and intensity information to form multi-view light field raw data; (2) Monitor the operating environment parameters of the light field camera. When the temperature exceeds 60°C or the power supply fluctuation exceeds ±5%, resulting in a signal-to-noise ratio lower than 20 dB, switch the acquisition task to the backup light field camera. (3) The RAW format light field data is transmitted to the edge server based on Huawei Atlas 500, and the RK3588 edge computing chip performs the computing task; (4) Applying the DnCNN deep convolutional neural network model to the RAW format light field data to eliminate sensor noise and control the single-frame processing delay to less than 10 ms; (5) analyzing the sub-images in the light field data according to the epipolar constraint rule, and filtering out abnormal data regions where the viewing angle offset exceeds 2 pixels; (6) Performing quality screening on the light field data according to a depth confidence threshold of 0.8, removing low-quality areas, and ensuring data reliability; (7) Adding metadata tags of dynamic range and contrast statistics to the light field data and encapsulating them into pre-processed data in HDF5 format.

[0023] Specifically, 4K resolution RAW format light field data is collected. This process uses professional light field cameras such as Lytro or Raytrix series, whose built-in microlens array can simultaneously record the spatial position and angle information of light. The camera sensor captures not only the light intensity on the two-dimensional plane, but also the light incidence angle data, thereby forming multi-view light field raw data containing depth information. Each original RAW data point contains four-dimensional information composed of spatial coordinates (x, y) and angular coordinates (u, v), which records the complete path characteristics of light in the scene. The light field camera is easily affected by environmental interference during the data collection process, which affects the imaging quality. Therefore, real-time monitoring of the camera operating environment parameters is crucial. The system continuously monitors key indicators such as temperature and power supply stability through built-in sensors. When the temperature exceeds 60°C or the power supply fluctuates by more than ±5%, the performance of the internal circuit of the camera decreases, causing the signal-to-noise ratio of the collected data to drop below 20dB. At this time, the image noise increases significantly and the details are seriously lost. After the monitoring program detects these abnormal parameters, it immediately triggers the backup camera switching mechanism to seamlessly transfer the data collection task to the backup light field camera with normal environmental parameters to ensure the continuity and reliability of data collection. During the switching process, the spatial positions and parameter configurations of the two cameras remain synchronized to avoid perspective jumps.

[0024] The collected RAW format light field data is sent to the edge server based on Huawei Atlas 500 through a high-speed data transmission channel. Atlas 500 is an edge computing platform launched by Huawei with powerful AI computing capabilities. The edge server is equipped with the RK3588 edge computing chip, which is a system-level chip that integrates an eight-core CPU, a high-performance GPU and an NPU, and is optimized for edge computing scenarios. After receiving the light field RAW data, the RK3588 loads the data into the memory and assigns computing tasks, improving processing efficiency through multi-core parallel processing.

[0025] The first task performed by the RK3588 chip is to apply the DnCNN deep convolutional neural network model to denoise the RAW format light field data. DnCNN is a residual learning framework designed specifically for image denoising, consisting of multiple convolutional layers, batch normalization layers, and ReLU activation functions. The model first decomposes the RAW data into different sub-aperture views, and then learns the noise features independently for each view. DnCNN learns the noise distribution features through residual connections instead of directly learning the original image, so that the model can more accurately identify and remove sensor noise, especially for photon shot noise and readout noise under low light conditions in light field data. The NPU unit of RK3588 performs hardware acceleration on the DnCNN model, controlling the single-frame processing delay to less than 10ms to meet real-time processing requirements. After denoising is completed, the analysis and processing stage based on the epipolar constraint rule is entered. The sub-images in the light field data should satisfy the epipolar geometric constraints, that is, the corresponding points should be distributed along the epipolar lines under different viewing angles. The processor calculates the corresponding positions of the feature points in each sub-image in adjacent viewing angles to form a disparity map. When the disparity value in a certain area deviates from the theoretical expectation by more than 2 pixels, it is marked as abnormal data. This abnormality is usually caused by factors such as sensor defects, light scattering or object reflection, which will seriously affect the accuracy of subsequent super-resolution processing. The system automatically detects and filters these abnormal areas and retains valid data that meets geometric consistency.

[0026] Next, the depth confidence evaluation mechanism is used for quality screening. Light field data can calculate the depth of the scene through parallax analysis, but the accuracy of depth estimation in different areas varies. The depth confidence algorithm is used to quantitatively evaluate the reliability of depth estimation in each area. The calculation factors include parallax consistency, texture richness, and gradient changes. When the depth confidence of an area is lower than the set threshold of 0.8, it indicates that the depth information of the area is unreliable, which usually occurs on reflective surfaces, transparent objects, or areas with missing textures. The system marks and removes these low-confidence areas to ensure the reliability of subsequent data processing.

[0027] Finally, metadata tags are added to the processed light field data, including dynamic range information (recording the difference between the maximum and minimum pixel values) and contrast statistics (describing the degree of light and dark changes in the image), which provide important references for subsequent super-resolution processing. The processed data is packaged in HDF5 format, which is a file format for storing and organizing large amounts of data. It supports multi-dimensional array storage and complex metadata attachment, and is particularly suitable for structured storage of light field data. During the packaging process, light field sub-images, depth maps, quality tags, and metadata are organized uniformly to form structured HDF5 format pre-processed data. When capturing light fields for an indoor scene with complex textures and objects at different depths, the Lytro Illum camera generates 5MB of RAW format light field data, including 9×9 array sub-images. During the processing, the edge server detected that the signal-to-noise ratio in some areas was only 16dB due to window reflections, which was far below the acceptable threshold, and immediately switched to the backup camera to re-collect data. The DnCNN denoising process identified and removed sensor noise patterns, increasing the overall signal-to-noise ratio to 32dB. Polar constraint analysis found that there was a viewing angle offset of about 3 pixels in the edge area of ​​the window, and the system automatically marked and filtered these areas. Depth confidence evaluation showed that the confidence of the smooth glass surface and the low-texture wall in the distance was 0.65 and 0.72, respectively, both below the 0.8 threshold, so these areas were marked as low reliability areas. The final generated HDF5 format preprocessed data contains clear multi-view information, accurate depth data, and complete quality assessment tags, providing high-quality input data for subsequent super-resolution processing based on Ascend NPU. The entire preprocessing process fully utilizes the parallel processing capabilities of the RK3588 edge computing chip, with a total time consumption of only 85ms, which effectively supports real-time light field super-resolution calculations.

[0028] In a specific embodiment, the process of executing step S102 may specifically include the following steps: (1) Parsing the HDF5 format preprocessed data into a sub-aperture image sequence of an N×N array to construct a multi-view representation of the light field; (2) Performing global perspective relationship analysis on the sub-aperture image sequence through the Vision Transformer branch to extract light field spatial consistency features; (3) Using a ResNet-18 variant CNN branch to locate local distortion of the sub-aperture image sequence and identify high-frequency distortion areas; (4) combining the spatial consistency feature with information of the high-frequency distortion area to generate a perspective correlation map; (5) calculating a depth error compensation coefficient based on the perspective association map and adjusting the light field depth data; (6) Constructing a distortion correction matrix based on the spatial consistency features and information about the high-frequency distortion area to eliminate optical distortion in the light field data.

[0029] Specifically, the calibration processing of HDF5 format preprocessed data is a key link in the high-reliability light field super-resolution edge computing method. First, the HDF5 format preprocessed data needs to be parsed into a structured N×N array sub-aperture image sequence. This parsing process extracts light field information stored in different data sets through the HDF5 file parser, including image data, disparity maps, and quality tags from each perspective. Typical light field data is usually organized as a 5×5, 7×7, or 9×9 sub-aperture array, and each sub-aperture corresponds to an image from a specific perspective. The parsed data forms a complete four-dimensional light field representation, that is, two-dimensional spatial position plus two-dimensional angle information, constructing a complete light field multi-perspective representation, providing a basic data structure for subsequent processing.

[0030] Next, the Vision Transformer branch is used to analyze the global view relationship of the sub-aperture image sequence. Vision Transformer is a deep learning architecture based on the self-attention mechanism. It divides the input image into fixed-size patches and captures the long-range dependencies between these patches through the self-attention layer. In light field processing, Vision Transformer treats each sub-aperture image as a sequence input. The self-attention mechanism enables the model to consider the information of all perspectives at the same time and effectively capture the global relationship between different perspectives. Through multi-head self-attention calculation, the model generates feature maps representing the spatial consistency of the light field. These feature maps contain information about the geometric consistency that light should maintain at different perspectives, which is crucial for identifying systematic deviations caused by inaccurate calibration. The ResNet-18 variant CNN branch is used to locate local distortions in the sub-aperture image sequence. ResNet-18 is a convolutional neural network based on residual connections. It solves the gradient vanishing problem in deep network training through skip connections. In this method, ResNet-18 is modified to adapt to the light field data structure and focuses on extracting local spatial features. This branch processes each sub-aperture image separately, and identifies local areas with severe distortion through stacked convolution operations and feature map comparison, paying special attention to locations prone to high-frequency distortion, such as lens edge areas and areas with uneven lighting. Through multi-scale feature extraction, the CNN branch can accurately locate problems such as optical distortion, chromatic aberration, and pixel offset, and output the precise location map of high-frequency distortion areas and the quantified value of the distortion degree.

[0031] The spatial consistency features are then combined with the information of the high-frequency distortion area to generate a perspective association map. This process weights the global features output by the Vision Transformer and the local features output by the ResNet-18 variant through a feature fusion network. The fusion uses an attention mechanism to assign different weights to different areas according to the feature confidence to ensure the balanced integration of global geometric consistency and local distortion information. The generated perspective association map is a multi-channel tensor, where each channel corresponds to the mapping relationship between a pair of perspectives, accurately describing the spatial correspondence and distortion degree between different perspectives in the light field, and providing an accurate reference for subsequent calibration steps.

[0032] The depth information of light field data is usually obtained by calculating the disparity between sub-aperture images, but due to factors such as calibration errors and lens distortion, the initial depth information often has systematic deviations. By analyzing the difference between the relative displacement in the perspective association map and the theoretical expected displacement, the depth correction value of each spatial position is calculated. This process first constructs the disparity-depth mapping relationship under the ideal light field model, and then generates spatially varying depth error compensation coefficients based on the deviation between the actual mapping and the ideal mapping. These coefficients are applied to the original depth map to adjust the depth value of each pixel and correct the depth estimation error caused by the lens parameter deviation.

[0033] Finally, a distortion correction matrix is ​​constructed based on the spatial consistency features and the information of the high-frequency distortion area. Optical distortion mainly includes radial distortion and tangential distortion, which will cause the straight lines in the image to appear bent or deformed. By analyzing the distortion patterns in the view-related mapping diagram, the system constructs a specific correction matrix for each sub-aperture image. These matrices are 3×3 or higher-dimensional transformation matrices that describe the mapping relationship from the distorted image to the ideal image. Applying these correction matrices to the original sub-aperture images can eliminate various optical distortions in the light field data, including barrel distortion, pincushion distortion, and complex mixed distortions, thereby obtaining a geometrically accurate light field representation.

[0034] Taking the campus security scene as an example, when the system processes a set of light field data containing buildings near and far and complex environments, the HDF5 format data is first parsed into a 7×7 sub-aperture image array. Vision Transformer analysis found that the perspective relationship in the central area is stable, but there is a systematic offset in the edge area; the ResNet-18 variant identifies the local distortion area in the lower right corner due to light reflection. The perspective correlation map generated by combining the information of the two shows that there is a systematic deviation of about 1.5 pixels in the parallax calculation at the edge of the building. Based on this finding, the system calculates the spatially varying depth error compensation coefficient and successfully corrects the problem of underestimation of the depth of distant buildings. At the same time, the constructed distortion correction matrix effectively eliminates the radial distortion of about 2% caused by the lens, so that the straight edge of the building maintains the geometric straight line characteristics in all perspectives. The calibrated light field data achieves accurate depth estimation and spatial reconstruction on the edge computing device.

[0035] In a specific embodiment, the process of executing step S103 may specifically include the following steps: (1) applying the depth error compensation coefficient and the distortion correction matrix to the calibrated light field data to generate a corrected light field image; (2) deploying a lightweight EfficientNet-B3 feature extraction layer on a Xilinx UltraScale+ FPGA to process the corrected light field image in parallel through logic units; (3) constructing a weight-adjustable scoring function based on PSNR, SSIM and edge sharpness indicators to calculate the reliability score of the corrected light field image; (4) dividing the corrected light field image into image blocks of n×n pixels in preparation for frequency domain processing; (5) performing a fast Fourier transform on the image block to convert the spatial domain light field data into a frequency domain representation; (6) The frequency domain representation is interpolated and up-sampled in the frequency domain, and then reconstructed by inverse Fourier transform to obtain a frequency domain super-resolution result.

[0036] Specifically, the calibration data is subjected to FPGA-accelerated quality assessment and frequency domain super-resolution processing. First, the depth error compensation coefficient and distortion correction matrix calculated in the previous step are applied to the calibrated light field data. This process is implemented through matrix transformation operations, and the corresponding distortion correction matrix is ​​applied to each sub-aperture image for spatial remapping, while the light field depth information is adjusted according to the depth error compensation coefficient. In the specific operation, the distortion correction matrix acts on each pixel coordinate to calculate its new position in the corrected image, and the corrected pixel value is obtained through bilinear or bicubic interpolation; the depth error compensation coefficient directly acts on the value of the depth map to correct the depth estimation deviation. These operations produce geometrically consistent and depth-accurate corrected light field images, laying the foundation for subsequent evaluation and frequency domain processing. The lightweight EfficientNet-B3 feature extraction layer is deployed on Xilinx UltraScale+ FPGA. EfficientNet is an efficient convolutional neural network architecture that balances network depth, width, and resolution through a compound scaling method. The B3 variant provides a good balance between accuracy and efficiency. Deploying it on FPGA requires network pruning and quantization optimization to reduce computational complexity while retaining key feature extraction capabilities. Xilinx UltraScale+ FPGA provides highly parallel computing power through programmable logic units, which is particularly suitable for accelerating convolution operations. In FPGA implementation, the convolution layer is mapped to the programmable logic unit, DSP slicing is used to accelerate multiplication and accumulation operations, and the image data stream is processed through a pipeline architecture. This parallel processing method can reduce the feature extraction calculation time of EfficientNet-B3 from 15ms of ordinary CPU to 5ms, greatly improving real-time processing capabilities.

[0037] Based on the extracted features, the system constructs a weighted scoring function to calculate the reliability score of the corrected light field image. The scoring function takes into account three key quality indicators: the peak signal-to-noise ratio (PSNR) measures the difference between the image and the noise-free reference; the structural similarity (SSIM) evaluates the structural, brightness and contrast similarity of the image; and the edge sharpness index quantifies the clarity of the image edge. The scoring function adopts a weighted summation form, that is, Score = α×PSNR_norm + β×SSIM + γ×Edge_Sharpness, where α, β, and γ are adjustable weight coefficients that are adaptively adjusted according to different scenarios, and PSNR_norm is the normalized PSNR value. The system calculates the score of each sub-aperture image and comprehensively obtains the reliability score of the entire light field data, which usually ranges from 0 to 1, and a high value indicates reliable data quality.

[0038] In parallel with the quality assessment, the system prepares the corrected light field image for frequency domain super-resolution processing. First, the corrected light field image is segmented into image blocks of size n×n pixels, with a typical block size of 64×64 or 128×128 pixels. Block processing helps reduce memory requirements, improve computational efficiency, and allow adaptive processing for local characteristics. When segmenting, appropriate overlapping areas are considered (usually 10-20% of the block size) to avoid discontinuities when processing block boundaries. For each light field subview, the system generates a set of regularly arranged image blocks to form a two-dimensional block matrix structure. Each block retains its position information in the original image to facilitate subsequent reconstruction.

[0039] A fast Fourier transform (FFT) is performed on each image block to convert the spatial domain data into a frequency domain representation. FFT is an algorithm that efficiently calculates discrete Fourier transforms and can decompose spatial domain signals into sinusoidal components of different frequencies. FPGA hardware accelerates the FFT operation and efficiently calculates two-dimensional FFT through butterfly operation units. The transformed frequency domain data contains an amplitude spectrum and a phase spectrum. The amplitude spectrum represents the intensity of each frequency component, and the phase spectrum represents the corresponding phase information. The advantage of frequency domain analysis is that it is easier to distinguish the low-frequency structural information and high-frequency detail information of the image, which facilitates targeted processing. The frequency domain representation is interpolated and upsampled in the frequency domain. Frequency domain upsampling is achieved by inserting zero padding in the center area of ​​the spectrum, and then applying frequency domain constraints to ensure that the newly generated high-frequency components transition smoothly with the edges of the original spectrum. This method retains all the information of the original spectrum while reserving space for new high-frequency details. After processing, the frequency domain data is converted back to the spatial domain through an inverse fast Fourier transform (IFFT) to obtain a higher resolution image block. Finally, each image block is reassembled according to its original position, and the overlapping area is weighted averaged when necessary to obtain the frequency domain super-resolution result.

[0040] Taking the autonomous driving scenario as an example, when processing complex light field data containing vehicles and pedestrians near and far, the system first applies the depth error compensation coefficient to correct the problem of underestimated depth of distant objects, and applies the distortion correction matrix to eliminate about 3% of edge distortion to generate a geometrically accurate corrected light field image. EfficientNet-B3 deployed on the FPGA processes 128 sub-image blocks in parallel and extracts key features for quality assessment. The system calculates that the PSNR of the vehicle area is 38dB, the SSIM is 0.96, and the edge sharpness is 92. The scoring function with weights of 0.3, 0.4, and 0.3 is applied, and the final reliability score is 0.92, indicating that the data quality is good. For frequency domain processing, the corrected light field image is divided into 64×64 pixel image blocks. After performing FFT, the sampling points are doubled in the frequency domain. Clear license plate details and pedestrian facial features are reconstructed through IFFT, which provides high-quality frequency domain processing results and accurate reliability assessment for subsequent deep learning super-resolution based on Ascend NPU.

[0041] In a specific embodiment, the process of executing step S104 may specifically include the following steps: (1) Transmitting the reliability score and the frequency domain super-resolution result to the Ascend NPU processing unit to construct a light field enhancement processing flow; (2) Deploy the ESRGAN network with a residual dense block structure on the Ascend NPU and receive the light field enhancement processing stream as input data; (3) constructing a weighted loss function based on the reliability score, and incorporating the reliability score into the calculation process of pixel loss, perceptual loss, and adversarial loss; (4) performing multi-layer feature extraction on the frequency domain super-resolution result through a residual dense block to retain the angle information of the light field image; (5) Processing the multi-layer features using a phase consistency loss constraint algorithm to reduce texture artifacts in the light field multi-view data; (6) Upsample and reconstruct the processed feature map to generate a deep learning super-resolution image containing high-frequency details and accurate depth information.

[0042] Specifically, the reliability score and frequency domain super-resolution result obtained by FPGA processing in the previous stage are transmitted to the Ascend NPU processing unit. Ascend NPU is a neural network processor designed by Huawei, which adopts the Da Vinci architecture and is optimized for deep learning inference. It has efficient matrix computing capabilities and low power consumption. The data transmission process is implemented through a high-speed PCIe bus or on-chip interconnection to ensure data integrity and transmission efficiency. The transmitted data includes frequency domain super-resolution results (including preliminary enhanced light field images) and reliability scores (0-1 interval values ​​indicating data credibility). These data together constitute the light field enhancement processing flow as input for deep learning processing. After receiving the data, Ascend NPU loads it into the on-chip memory and organizes the processing flow according to the predefined data flow graph.

[0043] The second step is to deploy the ESRGAN network with a residual dense block structure on the Ascend NPU. ESRGAN (EnhancedSuper-Resolution Generative Adversarial Network) is an advanced super-resolution generative adversarial network that introduces residual dense blocks (RDBs) and related improvements based on the original SRGAN. The deployment process requires first converting the pre-trained model to a format supported by the Ascend NPU, and then enabling the network to run efficiently on the NPU through steps such as operator mapping, graph optimization, and quantization compression. The residual dense block is the core structure of the network, which consists of multiple convolutional layers. The output of each layer is connected to the output of all previous layers to form a dense connection pattern, which enhances the feature extraction capability and gradient flow. The ESRGAN network architecture includes a feature extraction front end, a main body of multiple residual dense blocks, and a reconstruction back end. It receives the light field enhancement processing flow as input data, and each residual dense block is mapped to different computing units of the Ascend NPU to achieve parallel processing. A weighted loss function is constructed based on the reliability score. Traditional ESRGAN uses fixed weights to combine multiple loss functions, but the quality of light field data varies greatly and requires adaptive adjustment. This method incorporates reliability score into the loss function calculation, combining pixel loss, perceptual loss and adversarial loss. Pixel loss measures the direct difference between the reconstructed image and the target image; perceptual loss measures the perceptual similarity through the feature difference extracted by the pre-trained VGG network; adversarial loss comes from the discriminator's evaluation of the authenticity of the generated image. Reliability score is used as a dynamic weight adjustment factor. When the score value is high, the influence of perceptual loss and adversarial loss is enhanced to promote the generation of texture details; when the score value is low, the weights of these two items are reduced, and the stability of pixel loss is more relied on. This dynamic loss function enables the network to intelligently adjust the reconstruction strategy according to the reliability of the input data.

[0044] The fourth step is to perform multi-layer feature extraction on the frequency domain super-resolution results through residual dense blocks. Each residual dense block contains multiple convolutional layers, and the output feature map of each layer is connected to all subsequent layers to form a densely connected network. Ascend NPU efficiently performs convolution operations through computing units. The feature extraction process is divided into multiple scales: first, shallow features are extracted to capture basic structures and edges; then mid-level features extract textures and local patterns; finally, deep features capture abstract semantic information. For light field data, feature extraction needs to specifically retain angle information, so the network maintains perspective consistency when processing multi-view sub-images, and stores perspective relationship information by designing special feature channels to ensure that angle information is not lost during reconstruction.

[0045] Then, the phase consistency loss constraint algorithm is used to process multiple layers of features to reduce texture artifacts in light field multi-view data. Light field super-resolution can easily produce inconsistent textures in different viewpoints, especially on complex surfaces. The phase consistency loss constraint algorithm ensures that texture generation across viewpoints remains consistent by analyzing the phase spectrum relationship of features from different viewpoints. The algorithm first converts each viewpoint feature to the frequency domain, extracts the phase information, and then forces the phase difference of similar regions between different viewpoints to be minimized. This constraint mechanism keeps the texture patterns coherent under different viewpoints and reduces the artifacts caused by processing each viewpoint independently, especially in complex scenes such as surface reflections and transparent objects.

[0046] The processed feature map is upsampled and reconstructed to generate a deep learning super-resolution image. The upsampling process uses a sub-pixel convolution layer, which provides finer detail reconstruction capabilities than traditional bilinear interpolation. The reconstruction module fuses multiple layers of features and retains the original information through residual connections. During the reconstruction process, special attention is paid to the preservation of light field depth information, and depth perception loss is used to ensure that the super-resolution result is consistent with the original depth. Figure 1 The final output deep learning super-resolution image retains the light field angle information while having enhanced spatial resolution, clear high-frequency details and accurate depth information.

[0047] Taking intelligent monitoring applications as an example, when processing light field scenes containing multiple moving objects, the area with a reliability score of 0.88 (main moving objects) is given a higher perceptual loss weight by the Ascend NPU to generate more texture details; while the area with a score of 0.62 (shadow and reflective areas) relies more on pixel loss to maintain the stability of detail generation. The multi-layer features extracted by the residual dense block retain the detailed changes of the monitored object under different viewing angles, and the phase consistency constraint ensures that the surface texture of the vehicle remains consistent under different viewing angles. The final reconstructed 8K super-resolution image can clearly present the license plate number and human features 10 meters away, while accurately retaining the scene depth information, supporting subsequent 3D reconstruction and object recognition tasks.

[0048] In a specific embodiment, the process of executing step S105 may specifically include the following steps: (1) extracting high-frequency feature maps from the frequency domain super-resolution results to retain spatial detail information; (2) extracting a texture feature map from the deep learning super-resolution image to obtain details of deep learning reconstruction; (3) Inputting the high-frequency feature map and the texture feature map into a fusion network with a lightweight U-Net structure; (4) in the encoder stage of the fusion network, identifying key information in the high-frequency feature map and the texture feature map through a channel attention mechanism; (5) dynamically adjusting the weight coefficient λ based on the reliability score, setting the deep learning branch weight to 0.8 when the score is greater than 0.9, and reducing the deep learning branch weight to 0.3 when the score is less than or equal to 0.6; (6) The fused features are reconstructed into 8K resolution light field super-resolution imaging through the decoder of the fusion network.

[0049] Specifically, high-frequency feature maps are extracted from the frequency domain super-resolution results. This process processes the frequency domain super-resolution results through a feature extraction network, which contains multiple convolutional layers and high-pass filters, specifically designed to capture high-frequency components in the image. The high-frequency feature map focuses on retaining edge, texture, and detail information, which are critical to image clarity. Frequency domain path processing can effectively preserve the spatial structure and geometric characteristics of the original image, especially in areas with clear boundaries and regular textures. While retaining high-frequency information, the extraction process filters out artifacts and noise that may be introduced in the frequency domain processing, ensuring that the feature map has high information content and low noise.

[0050] Extract texture feature maps from deep learning super-resolution images. Super-resolution images generated by the ESRGAN network through the deep learning path often perform well in terms of texture details and perceptual quality. The texture feature extraction network focuses on capturing the detailed texture, material properties, and local structural features reconstructed by deep learning. These texture features often go beyond the details visible in the original low-resolution data and are generated through the prior knowledge learned by the deep network. The extraction process adopts a multi-scale feature extraction strategy to ensure that texture information at different scales is captured, from macroscopic structures to microscopic details, providing a comprehensive texture description. Compared with frequency domain features, deep learning texture features perform better in complex and irregular texture areas, but the geometric accuracy may not be as good as frequency domain features. The extracted high-frequency feature map and texture feature map are input into a fusion network with a lightweight U-Net structure. U-Net is an encoder-decoder neural network originally designed for medical image segmentation. It is widely used in various image processing tasks due to its efficient feature extraction and reconstruction capabilities. Lightweight U-Net adapts to resource constraints in edge computing environments by reducing network depth and number of channels. The fusion network receives two feature maps as input and unifies them into the network processing flow through the designed multi-channel input layer. To adapt to the processing characteristics of Ascend NPU, the network structure is optimized to reduce memory requirements and improve computing efficiency while maintaining the effectiveness of feature fusion.

[0051] In the encoder stage of the fusion network, the key information in the high-frequency feature map and the texture feature map is identified through the channel attention mechanism. Channel attention is an adaptive feature weighting method that assigns weights according to the importance of different channel features. In specific implementation, the encoder first extracts the hierarchical representation of the input features through downsampling convolution, and then calculates global statistical information (such as mean and standard deviation) for each feature channel to learn the channel weights. Important channels (channels containing key information) receive higher weights, and unimportant channels have lower weights. This mechanism enables the network to automatically focus on the clear edges in high-frequency features and the rich details in texture features, selectively retaining the dominant parts of the two features to form a complementary and enhanced feature representation. Dynamically adjusting the weight coefficient λ based on the reliability score is a key adaptive mechanism. This mechanism uses the reliability score calculated in the early stage to dynamically determine the contribution ratio of the frequency domain and deep learning paths. The specific rules are set as follows: when the score is greater than 0.9, the weight of the deep learning branch is set to 0.8, indicating that the data is highly reliable and more dependent on the rich texture generated by deep learning; when the score is less than or equal to 0.6, the weight of the deep learning branch is reduced to 0.3, indicating that the data reliability is low and more dependent on the fidelity of the frequency domain path. When the reliability score is between 0.6 and 0.9, the weight is calculated by linear interpolation. This dynamic adjustment strategy ensures that the system can intelligently switch processing strategies based on the quality of input data to achieve the best balance between fidelity and perceived quality.

[0052] Finally, the fused features are reconstructed into 8K resolution light field super-resolution imaging through the decoder of the fusion network. The decoder contains a series of upsampling convolutional layers to gradually restore the spatial resolution of the features. Each upsampling layer is followed by a residual block to enhance the network's expressiveness while alleviating the gradient vanishing problem. During the decoding process, the jump connection from the encoder provides detailed spatial information to supplement the details that may be lost during the upsampling process. The final output layer uses sub-pixel convolution to achieve accurate detail reconstruction and generate 8K resolution (7680×4320 pixels) light field super-resolution imaging. The imaging result combines the geometric accuracy of frequency domain processing and the rich texture of deep learning, while retaining the angle and depth information of the light field data.

[0053] Taking product inspection applications as an example, when processing light field data containing fine circuit boards, the system first extracts clear circuit line edges and geometric shapes from the frequency domain results, and extracts complex component textures and marking details from the deep learning results. The fusion network uses the channel attention mechanism to enhance the frequency domain features in the circuit line area and the deep learning features in the component identification area. Since the reliability score of this batch of data is 0.85, the system sets the deep learning branch weight to 0.7 to appropriately balance the two features. The resulting super-resolution imaging can clearly present 0.1mm wide circuit lines and tiny text marks on components, while accurately retaining the three-dimensional depth information of the circuit board, enabling the inspection system to accurately identify defects and locate their spatial positions, greatly improving product quality inspection efficiency and accuracy.

[0054] In a specific embodiment, the process of executing step S106 may specifically include the following steps: (1) Calculate the peak signal-to-noise ratio (PSNR) value of the super-resolution imaging of the 8K light field, and set 32 ​​dB as the quality threshold; (2) measuring the structural similarity SSIM index of the super-resolution imaging of the 8K light field, and setting 0.92 as the quality acceptance threshold; (3) Evaluate the edge sharpness index of the super-resolution imaging of the 8K light field, and set 90 as the quality acceptance threshold; (4) Comprehensively analyzing the PSNR value, SSIM index and edge sharpness index, if the set threshold is not reached, triggering a recalculation process, calling the calibrated data from the storage module to re-perform super-resolution processing; (5) Ascend NPU computing resources are allocated through a priority queue mechanism. In normal mode, 70% of computing power is allocated for real-time super-resolution inference and 30% for incremental model training. (6) Monitor the operating parameters of FPGA and NPU. When the temperature exceeds 85°C or the utilization rate exceeds 90%, start the hardware redundancy switching mechanism and migrate the computing tasks to the backup computing nodes to ensure the continuous and stable operation of the edge computing system.

[0055] Specifically, the sixth stage of the Ascend-based high-reliability light field super-resolution edge computing method is quality assessment and resource management to ensure system output reliability and operational stability. First, the peak signal-to-noise ratio (PSNR) of the generated super-resolution imaging is calculated. PSNR is a basic indicator for evaluating image quality. It calculates the mean square error (MSE) between the reconstructed image and the reference image, and then converts it into a signal-to-noise ratio on a logarithmic scale. In the light field super-resolution scenario, the reference image usually uses a preset standard sample or a high-quality result processed in the previous stage. The PSNR calculation covers all sub-view images, and its average value is taken as the PSNR evaluation of the overall light field data. The system sets 32dB as the quality threshold, which is based on a large number of experimental verifications and can ensure that details are recognizable and there is no obvious noise interference. A PSNR below the threshold usually means that key details are lost or too many artifacts are introduced during the reconstruction process.

[0056] Next, the structural similarity (SSIM) index of super-resolution imaging is measured. SSIM is a quality assessment method that is more in line with human visual perception than PSNR. It generates a score between 0 and 1 by comparing the brightness, contrast and structural similarity of the image, with 1 indicating exact similarity. The SSIM calculation takes into account the local characteristics of the image, calculates the SSIM value for each sub-view image, and then weighted averages it to obtain the overall score. The system sets 0.92 as the SSIM quality threshold, which ensures that the reconstructed image is highly similar in structure to the ideal result and maintains the consistency of key visual features. SSIM below the threshold usually indicates that the image structure is deformed or the texture features are unnatural, even if the PSNR may meet the standard.

[0057] The third step is to evaluate the edge sharpness index of super-resolution imaging. Edge sharpness is a special index to measure the clarity of an image, which is calculated by measuring the gradient amplitude of the transition area of ​​the image edge. The calculation process first uses an edge detection operator (such as Sobel or Canny) to locate the edge of the image, then measures the average intensity of the pixel gradient at the edge, and finally normalizes it to a score value of 0-100. The system sets 90 as the edge sharpness quality qualified threshold, which ensures that the image edges are sharp and clear, and the details are clearly outlined. Edge sharpness below the threshold usually means that the image is blurred or lacks details, especially in high-frequency detail areas such as text and lines.

[0058] The PSNR value, SSIM index and edge sharpness index are comprehensively analyzed to form a quality decision mechanism. The system requires that the three indicators reach their respective thresholds at the same time to determine the output as qualified. If any indicator does not meet the standard, the recalculation process is triggered: the previously saved calibrated data (second-stage processing results) is called from the storage module, and the super-resolution processing is re-executed. The recalculation process may use different parameter configurations or processing algorithms. The system attempts to recalculate up to 3 times, adjusting the relevant parameters each time to improve the success rate. If it still does not meet the standard, it is marked as a low-quality output and a detailed analysis report is recorded. This cyclic optimization mechanism ensures the high quality standards of the system output and prevents unqualified results from flowing to downstream applications.

[0059] The task scheduling module allocates Ascend NPU computing resources through a priority queue mechanism. This mechanism prioritizes and queues all processing tasks, with real-time tasks usually receiving the highest priority. In normal working mode, the system allocates 70% of the computing power of Ascend NPU to real-time super-resolution inference tasks to ensure timely response of main functions; the other 30% of the computing power is used for incremental model training to continuously optimize network parameters to adapt to new scenarios. Incremental training adopts an online learning strategy, using processed high-quality samples as training data to continuously improve model performance without interfering with real-time tasks. When the pressure of real-time tasks increases, the system will dynamically adjust this allocation ratio to ensure that core functions are given priority in resource protection.

[0060] Finally, the system continuously monitors the operating parameters of FPGA and NPU and implements a hardware redundancy guarantee mechanism. Monitoring indicators include key parameters such as processor temperature, resource utilization, power status, and processing delay. When the temperature is detected to be above 85°C or the resource utilization continues to exceed 90%, the system determines that the hardware is in an overloaded state and starts the hardware redundancy switching mechanism. This mechanism migrates the current processing tasks and status data to the backup computing node, which can be a redundant FPGA / NPU module or other nodes in the edge server cluster. The task migration process is implemented through a state synchronization protocol to ensure that processing continuity is not affected. The hardware redundancy mechanism is linked with dynamic resource scheduling to form a complete edge computing system reliability guarantee framework, which can keep the system running continuously and stably even in harsh environments or under high load conditions.

[0061] Taking medical imaging applications as an example, when the system processes the patient's light field fundus image, the generated 8K super-resolution imaging is evaluated for quality and obtains a PSNR value of 33.8dB (exceeding the 32dB threshold), an SSIM value of 0.94 (exceeding the 0.92 threshold), and an edge sharpness value of 95 (exceeding the 90 threshold). All three indicators meet the standards, indicating that the reconstruction quality meets the medical diagnosis requirements. At the same time, monitoring shows that the NPU temperature reaches 82°C, close to but not exceeding the 85°C threshold. The system controls heat dissipation by dynamically adjusting the fan speed. During the peak diagnosis period, the system adjusts the NPU computing power allocation to 85% for real-time reasoning and 15% for model training, giving priority to clinical diagnosis needs. This example demonstrates the complete processing flow of the high-reliability light field super-resolution edge computing method based on Ascend in terms of quality assurance and resource management, ensuring the provision of high-quality and highly reliable light field super-resolution imaging services in resource-constrained edge environments.

[0062] The above describes the high-reliability light field super-resolution edge computing method based on Ascend in the embodiment of the present application. The following describes the high-reliability light field super-resolution edge computing system based on Ascend in the embodiment of the present application. Figure 2 In the embodiments of the present application, an embodiment of a high-reliability light field super-resolution edge computing system based on Ascend includes: A denoising module is used to collect RAW format light field data through a light field camera, perform DnCNN denoising on the light field data, and obtain pre-processed data in HDF5 format; A calibration module, used to calibrate the preprocessed data through a Transformer-CNN model to generate a depth error compensation coefficient and a distortion correction matrix; The processing module is used to use the calibrated light field data, accelerate the EfficientNet model and FFT processing through FPGA, and output the reliability score and frequency domain super-resolution results; A computing module, configured to execute ESRGAN model calculation by the Ascend NPU based on the reliability score and the frequency domain super-resolution result to form a deep learning super-resolution image; A weighting module is used to weight the frequency domain super-resolution result and the deep learning super-resolution image through a U-Net fusion network to obtain 8K light field super-resolution imaging; The allocation module is used to perform PSNR and SSIM evaluation on the super-resolution imaging of the 8K light field. The task scheduling module dynamically allocates Ascend NPU and FPGA resources to build an edge computing redundancy mechanism.

[0063] Through the collaborative cooperation of the above-mentioned components, the present invention collects RAW format light field data through a light field camera and performs DnCNN denoising processing to obtain structured HDF5 format pre-processed data, thereby ensuring the data quality and consistency of subsequent processing; the pre-processed data is calibrated based on the Transformer-CNN hybrid architecture model to generate accurate depth error compensation coefficients and distortion correction matrices, effectively eliminating systematic errors in the light field imaging process; the EfficientNet model accelerated by FPGA and FFT processing technology are used to achieve hardware-level parallel processing capabilities, greatly improving the real-time performance of quality assessment and frequency domain processing, and at the same time The output reliability score and frequency domain super-resolution results provide double guarantees for subsequent processing; the ESRGAN model calculation is performed based on the Ascend NPU, which fully utilizes the advantages of NPU in deep learning reasoning to form a deep learning super-resolution image with rich texture details; the frequency domain results and deep learning images are innovatively weighted by the U-Net fusion network for quality scoring, which realizes the complementary enhancement of the advantages of different processing paths and solves the problem that a single algorithm is difficult to adapt to complex scenes; the PSNR, SSIM evaluation and task scheduling module dynamic resource allocation strategy are adopted to build a complete edge computing redundancy mechanism, which significantly improves the fault tolerance and stability of the system. The present invention has made many innovations in the algorithm design level according to the characteristics of light field data: the DnCNN denoising model optimizes the noise distribution characteristics of light field images and effectively retains the angle information; the Transformer-CNN hybrid architecture gives full play to the complementary advantages of Transformer in global relationship modeling and CNN in local feature extraction; the dynamic loss function adjustment mechanism based on reliability scoring enables the ESRGAN model to adaptively balance fidelity and perceptual quality according to data quality; the channel attention mechanism of the U-Net fusion network realizes the optimal fusion of frequency domain processing and deep learning results. These algorithmic features work together to enable the system to automatically select the best processing strategy for light field data of different qualities in resource-constrained edge environments, significantly improving the quality consistency of super-resolution results. Overall, the present invention not only overcomes the problems of resource limitations and insufficient reliability in edge environments in the prior art, but also achieves high reliability, high adaptability and high efficiency of light field super-resolution systems through the collaborative optimization of heterogeneous computing resources and the dynamic adjustment of intelligent processing strategies.

[0064] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0065] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0066] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0067] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0068] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0069] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0070] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A highly reliable light field super-resolution edge computing method based on Ascend, characterized in that: The Ascend-based high-reliability light field super-resolution edge computing method includes: Collecting RAW format light field data through a light field camera, performing DnCNN denoising on the light field data, and obtaining pre-processed data in HDF5 format; According to the preprocessed data, calibration is performed through the Transformer-CNN model to generate a depth error compensation coefficient and a distortion correction matrix; Using the calibrated light field data, the FPGA-accelerated EfficientNet model and FFT processing are used to output reliability scores and frequency domain super-resolution results. Based on the reliability score and frequency domain super-resolution results, the Ascend NPU performs ESRGAN model calculation to form a deep learning super-resolution image. The frequency domain super-resolution result and the deep learning super-resolution image are weighted by quality scoring through a U-Net fusion network to obtain 8K light field super-resolution imaging; The PSNR and SSIM are evaluated for the super-resolution imaging of the 8K light field, and the task scheduling module dynamically allocates Ascend NPU and FPGA resources to build an edge computing redundancy mechanism.

2. The Ascend-based high-reliability light field super-resolution edge computing method according to claim 1, characterized in that: The light field camera is used to collect RAW format light field data, and the light field data is subjected to DnCNN denoising processing to obtain pre-processed data in HDF5 format, including: Collect 4K resolution RAW format light field data, and the sensor records the light direction and intensity information to form multi-view light field raw data; Monitor the operating environment parameters of the light field camera. When the temperature exceeds 60°C or the power supply fluctuation exceeds ±5%, resulting in a signal-to-noise ratio lower than 20dB, switch the acquisition task to the backup light field camera. The RAW format light field data is transmitted to the edge server based on Huawei Atlas 500, and the RK3588 edge computing chip performs the computing task; Applying the DnCNN deep convolutional neural network model to the RAW format light field data to eliminate sensor noise and control the single frame processing delay to less than 10ms; Analyze the sub-images in the light field data according to the epipolar constraint rule, and filter out abnormal data areas with a viewing angle deviation exceeding 2 pixels; The light field data is quality screened according to a depth confidence threshold of 0.8, and low-quality areas are removed to ensure data reliability; Dynamic range and contrast statistics metadata tags are added to the light field data, and the data is packaged into HDF5 format pre-processed data.

3. The Ascend-based high-reliability light field super-resolution edge computing method according to claim 1, characterized in that: The method of calibrating the preprocessed data through the Transformer-CNN model to generate a depth error compensation coefficient and a distortion correction matrix includes: Parsing the HDF5 format preprocessed data into a sub-aperture image sequence of an N×N array, and constructing a light field multi-view representation; Performing global perspective relationship analysis on the sub-aperture image sequence through the Vision Transformer branch to extract light field spatial consistency features; Using a ResNet-18 variant CNN branch to locate local distortion of the sub-aperture image sequence and identify high-frequency distortion areas; Combining the spatial consistency feature with information of the high-frequency distortion area to generate a viewing angle correlation map; Calculating a depth error compensation coefficient based on the perspective association map, and adjusting light field depth data; A distortion correction matrix is ​​constructed according to the spatial consistency feature and information of the high-frequency distortion area to eliminate optical distortion in the light field data.

4. The Ascend-based high-reliability light field super-resolution edge computing method according to claim 1, characterized in that: The calibrated light field data is used to accelerate the EfficientNet model and FFT processing through FPGA to output reliability scores and frequency domain super-resolution results, including: Applying the depth error compensation coefficient and the distortion correction matrix to the calibrated light field data to generate a corrected light field image; Deploy a lightweight EfficientNet-B3 feature extraction layer on a Xilinx UltraScale+ FPGA to parallelize the processing of the corrected light field image through logic units; Constructing a weight-adjustable scoring function based on PSNR, SSIM and edge sharpness indicators to calculate the reliability score of the corrected light field image; Segmenting the corrected light field image into image blocks of size n×n pixels in preparation for frequency domain processing; Performing a fast Fourier transform on the image block to convert the spatial domain light field data into a frequency domain representation; The frequency domain representation is subjected to interpolation up-sampling processing in the frequency domain, and then reconstructed through inverse Fourier transform to obtain a frequency domain super-resolution result.

5. The Ascend-based high-reliability light field super-resolution edge computing method according to claim 1, characterized in that: Based on the reliability score and the frequency domain super-resolution result, the Ascend NPU performs ESRGAN model calculation to form a deep learning super-resolution image, including: Transmitting the reliability score and the frequency domain super-resolution result to the Ascend NPU processing unit to construct a light field enhancement processing flow; Deploy an ESRGAN network with a residual dense block structure on the Ascend NPU and receive the light field enhancement processing stream as input data; Constructing a weighted loss function according to the reliability score, and incorporating the reliability score into the calculation process of pixel loss, perceptual loss, and adversarial loss; Performing multi-layer feature extraction on the frequency domain super-resolution result through a residual dense block to retain the angle information of the light field image; Processing the multi-layer features using a phase consistency loss constraint algorithm to reduce texture artifacts in light field multi-view data; The processed feature map is upsampled and reconstructed to generate a deep learning super-resolution image containing high-frequency details and accurate depth information.

6. The Ascend-based high-reliability light field super-resolution edge computing method according to claim 1, characterized in that: The frequency domain super-resolution result and the deep learning super-resolution image are weighted by quality scoring through a U-Net fusion network to obtain 8K light field super-resolution imaging, including: Extracting high-frequency feature maps from the frequency domain super-resolution results to retain spatial detail information; Extracting a texture feature map from the deep learning super-resolution image to obtain details of deep learning reconstruction; Inputting the high-frequency feature map and the texture feature map into a fusion network with a lightweight U-Net structure; In the encoder stage of the fusion network, key information in the high-frequency feature map and the texture feature map is identified through a channel attention mechanism; Dynamically adjust the weight coefficient λ based on the reliability score, set the deep learning branch weight to 0.8 when the score is greater than 0.9, and reduce the deep learning branch weight to 0.3 when the score is less than or equal to 0.6; The fused features are reconstructed into 8K resolution light field super-resolution imaging through the decoder of the fusion network.

7. The Ascend-based high-reliability light field super-resolution edge computing method according to claim 1, characterized in that: The PSNR and SSIM evaluations are performed on the super-resolution imaging of the 8K light field, and the Ascend NPU and FPGA resources are dynamically allocated by the task scheduling module to build an edge computing redundancy mechanism, including: Calculate the peak signal-to-noise ratio (PSNR) value of the super-resolution imaging of the 8K light field, and set 32 ​​dB as the quality threshold; The structural similarity SSIM index of the super-resolution imaging of the 8K light field is measured, and 0.92 is set as the quality acceptance threshold; Evaluate the edge sharpness index of the super-resolution imaging of the 8K light field, and set 90 as the quality acceptance threshold; The PSNR value, SSIM index and edge sharpness index are comprehensively analyzed. If the set threshold is not reached, a recalculation process is triggered, and the calibrated data is called from the storage module to perform super-resolution processing again; Ascend NPU computing resources are allocated through a priority queue mechanism. In normal mode, 70% of computing power is allocated for real-time super-resolution inference and 30% for incremental model training. Monitor the operating parameters of FPGA and NPU. When the temperature exceeds 85°C or the utilization rate exceeds 90%, start the hardware redundancy switching mechanism to migrate computing tasks to the backup computing nodes to ensure the continuous and stable operation of the edge computing system.

8. A high-reliability light field super-resolution edge computing system based on Ascend, used to implement the high-reliability light field super-resolution edge computing method based on Ascend as described in any one of claims 1-7, characterized in that: The Ascend-based high-reliability light field super-resolution edge computing system includes: A denoising module is used to collect RAW format light field data through a light field camera, perform DnCNN denoising on the light field data, and obtain pre-processed data in HDF5 format; A calibration module, used to calibrate the preprocessed data through a Transformer-CNN model to generate a depth error compensation coefficient and a distortion correction matrix; The processing module is used to use the calibrated light field data, accelerate the EfficientNet model and FFT processing through FPGA, and output the reliability score and frequency domain super-resolution results; A computing module, configured to execute ESRGAN model calculation by the Ascend NPU based on the reliability score and the frequency domain super-resolution result to form a deep learning super-resolution image; A weighting module is used to weight the frequency domain super-resolution result and the deep learning super-resolution image through a U-Net fusion network to obtain 8K light field super-resolution imaging; The allocation module is used to perform PSNR and SSIM evaluation on the super-resolution imaging of the 8K light field. The task scheduling module dynamically allocates Ascend NPU and FPGA resources to build an edge computing redundancy mechanism.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the high-reliability light field super-resolution edge computing method based on Ascend described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the processor executes the Ascend-based high-reliability light field super-resolution edge computing method as described in any one of claims 1 to 7.

Images