Visual enhancement method and system for LED creative spherical display screen

Abstract

The invention relates to the technical field of spherical display screens, and discloses a visual enhancement method and system for an LED creative spherical display screen. The method comprises: inputting an original display image of the LED spherical display screen into a multi-scale Transformer hierarchical CNN network for feature extraction to obtain a multi-dimensional feature atlas; performing feature mapping, projecting pixel points into a spherical coordinate system to calculate display deviations, and generating a comprehensive evaluation matrix; calculating spatial compensation coefficients through an N-order polynomial function to obtain a spatiotemporal compensation matrix; performing gradient calculation on the spatiotemporal compensation matrix to obtain a target vector field; inputting the target vector field into a U‑Net structure generator for adversarial training to obtain compensation optimization parameters; performing compensation calculation in combination with light intensity data, and outputting an enhanced display image. The invention realizes real-time dynamic optimization of the display image of the spherical display screen, and ensures the display quality under different ambient lighting conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of spherical display screens, and in particular to a visual enhancement method and system for an LED creative spherical display screen. Background Art

[0002] Creative spherical display screens are widely used in commercial advertising, exhibitions and other fields due to their unique visual effects and spatial artistic expression. However, due to the special structure of spherical display, traditional LED flat display technology will produce serious visual distortion and display distortion problems when applied to spherical surfaces.

[0003] Spherical LED displays face multiple technical challenges. The first is the geometric distortion problem caused by the curvature of the sphere. Observation from different viewing angles will produce different degrees of image deformation. The second is the non-uniform distribution of LED dots on the sphere, which causes uneven display brightness and color. The third is the impact of changes in ambient light on the display effect, as well as the consistency of image quality when observing from multiple viewing angles. Current visual enhancement methods mainly focus on the optimization of two-dimensional plane displays, and lack specialized algorithms for spherical display characteristics. Existing technologies often use simple geometric transformations or brightness compensation, which cannot effectively solve the complex distortion problems in spherical displays, and are also difficult to cope with dynamic display content and changing viewing environments, which seriously restricts the application effect of spherical LED displays. Summary of the invention

[0004] The present invention provides a visual enhancement method and system for an LED creative spherical display screen, which realizes real-time dynamic optimization of a display image of the spherical display screen and ensures display quality under different ambient lighting conditions.

[0005] In a first aspect, the present invention provides a method for visual enhancement of an LED creative spherical display screen, the method comprising:

[0006] The original display image of the LED spherical display screen is input into the multi-scale Transformer hierarchical CNN network for feature extraction to obtain a multi-dimensional feature atlas;

[0007] Performing feature mapping on the multidimensional feature atlas, projecting pixel points into a spherical coordinate system to calculate display deviation, generating distortion mapping data, and calculating multi-view data based on a spatial consistency function to obtain a comprehensive evaluation matrix;

[0008] The comprehensive evaluation matrix is ​​input into the adaptive compensation model to calculate the spatial compensation coefficient, and combined with the temporal compensation function to obtain the spatiotemporal compensation matrix;

[0009] Performing gradient calculation on the spatiotemporal compensation matrix at M×N grid points to construct a compensation gradient field, and performing convolution operation with the original display image to obtain a target vector field;

[0010] The target vector field is input into the U-Net structure generator, adversarial training is performed in combination with the LED luminous characteristic model and the visual perception model, and the compensation optimization parameters are obtained through scoring by the PatchGAN discriminator;

[0011] In combination with the illumination intensity data, compensation calculation is performed on the original display image according to the compensation optimization parameters to output an enhanced display image.

[0012] In a second aspect, the present invention provides a visual enhancement device for an LED creative spherical display screen, the visual enhancement device for an LED creative spherical display screen comprising:

[0013] A feature extraction module is used to input the original display image of the LED spherical display screen into a multi-scale Transformer hierarchical CNN network for feature extraction to obtain a multi-dimensional feature atlas;

[0014] A feature mapping module is used to perform feature mapping on the multi-dimensional feature atlas, project pixel points into a spherical coordinate system to calculate display deviation, generate distortion mapping data, and calculate multi-view data based on a spatial consistency function to obtain a comprehensive evaluation matrix;

[0015] An adaptive compensation module, used to input the comprehensive evaluation matrix into an adaptive compensation model to calculate the spatial compensation coefficient, and combine it with the temporal compensation function to obtain a spatiotemporal compensation matrix;

[0016] A calculation module, used for performing gradient calculation on the spatiotemporal compensation matrix at M×N grid points, constructing a compensation gradient field, and performing convolution operation with the original display image to obtain a target vector field;

[0017] A compensation optimization module is used to input the target vector field into a U-Net structure generator, perform adversarial training in combination with an LED luminous characteristic model and a visual perception model, and obtain compensation optimization parameters through scoring by a PatchGAN discriminator;

[0018] The output module is used to combine the light intensity data, perform compensation calculation on the original display image according to the compensation optimization parameters, and output an enhanced display image.

[0019] The third aspect of the present invention provides a computer device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the computer device executes the above-mentioned visual enhancement method of the LED creative spherical display screen.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the visual enhancement method of the above-mentioned LED creative spherical display screen.

[0021] In the technical solution provided by the present invention, the multi-dimensional features of the display image are effectively extracted through the multi-scale Transformer hierarchical CNN network structure, which overcomes the limitations of the traditional single-scale feature extraction method and enhances the comprehensiveness and accuracy of feature extraction. The method of combining spherical coordinate mapping and spatial consistency function is adopted to accurately calculate the display distortion, and the display distortion problem under different observation angles is effectively solved through multi-view data fusion technology. Combined with the adaptive compensation model and the timing compensation function, the joint optimization in the spatial and temporal dimensions is realized, and the dynamic distortion in the display process is effectively reduced. By introducing the construction method of the compensation gradient field and the target vector field, the geometric correction of the display image is more accurate and the edge transition is more natural. The adversarial training mechanism of the U-Net structure generator and the PatchGAN discriminator is adopted, combined with the LED luminous characteristic model and the visual perception model, the authenticity and stability of the visual enhancement effect are significantly improved. Through the fusion operation of the light intensity data and the compensation optimization parameters, in conjunction with the frame synchronization control mechanism, the real-time dynamic optimization of the display image is realized, and the display quality under different ambient lighting conditions is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0023] Figure 1 Schematic diagram of the steps of the visual enhancement method of the LED creative spherical display screen in an embodiment of the present invention;

[0024] Figure 2 It is a structural schematic diagram of a visual enhancement device of an LED creative spherical display screen in an embodiment of the present invention;

[0025] Figure 3 It is a schematic block diagram of the structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The embodiment of the present invention provides a visual enhancement method and system for a creative spherical LED display screen. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the visual enhancement method of the LED creative spherical display screen in the embodiment of the present invention includes:

[0028] Step S1, inputting the original display image of the LED spherical display screen into a multi-scale Transformer hierarchical CNN network for feature extraction to obtain a multi-dimensional feature atlas;

[0029] It is understandable that the execution subject of the present invention may be a visual enhancement device of a creative spherical LED display screen, or may be a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0030] Specifically, the original display image is input into a hybrid network structure combining a multi-scale convolutional neural network (CNN) and a Transformer. The original display image is processed by multiple convolutional layers of different scales, including convolution kernels of 3×3, 5×5, and 7×7 sizes. These convolutional layers are used to extract different scale features of the image and obtain preliminary feature data. Through the convolution operation, the network can capture local and fine-grained spatial information from the image. The initial feature data is input into a Transformer encoder with 8 attention heads for processing. The dimension of each attention head is 64, and each attention head captures the relationship and dependency between different positions in the image in a 64-dimensional space. The attention mechanism of the Transformer can help the model understand the mutual influence of various parts of the image in a global scope, avoid relying only on local information, and thus improve the effect of feature extraction. Through the parallel processing of multiple attention heads, the network can fully mine the complex patterns in the image and generate attention feature data. The attention feature data is input into the local feature extraction module, the medium-scale feature extraction module, and the global feature extraction module respectively. According to different spatial scales, different levels of features of the image are extracted. The local feature extraction module captures small-scale features in the image by processing fine-grained local information; the medium-scale feature extraction module uses an appropriate scale range to extract larger-scale structural information in the image; and the global feature extraction module focuses on the features of the entire image and captures global context information. The feature maps generated by each module are local feature maps, medium-scale feature maps, and global feature maps, representing the multi-level information of the image at different spatial scales. The local feature map is processed by 1×1 convolution to adjust the number of channels of the feature map to better match the needs of subsequent processing. This process effectively reduces the amount of calculation and helps optimize feature representation. For the medium-scale feature map, the spatial pyramid pooling technology is used to extract multi-scale information using 2×2 and 4×4 pooling kernels. Spatial pyramid pooling can effectively extract global information of different scales from the image, so that the model can better process input data of different sizes and scales and improve the perception of features at different spatial levels. For the global feature map, the self-attention mechanism is used to process it. By calculating the correlation between each pixel in the feature map, the understanding of global information is strengthened to obtain the third-layer feature map. The self-attention mechanism enables the model to dynamically adjust the importance of each pixel, enhancing the flexibility and accuracy of feature representation. The first-layer feature map, the second-layer feature map, and the third-layer feature map are input into the feature fusion module for processing. In the feature fusion module, the three feature maps of different scales are channel-joined in a cascade manner to form a fused feature data set. By joining feature maps of different scales, the network can retain multi-level spatial information and provide richer feature representation. The fused feature data is normalized to improve the stability of training and the convergence speed of the model.A multi-dimensional feature atlas is generated through a residual connection structure to ensure the effective transmission of feature information and avoid the problem of gradient disappearance or explosion.

[0031] Step S2, feature mapping is performed on the multidimensional feature atlas, pixel points are projected into a spherical coordinate system to calculate display deviation, distortion mapping data is generated, and multi-view data is calculated based on a spatial consistency function to obtain a comprehensive evaluation matrix;

[0032] Specifically, according to the spherical coordinate mapping relationship, the plane coordinate points in the multidimensional feature atlas are converted into a spherical feature point set. The two-dimensional pixel points in the image are mapped onto the three-dimensional spherical surface. Because the display area of ​​the LED display screen is usually spherical, the display content under the plane coordinate system will be distorted on the spherical surface. Through accurate coordinate mapping, it is ensured that the display content of the image can adapt to the spherical shape, and then the display deviation is calculated. The conversion process is realized by the spherical coordinate mapping formula, which depends on the inverse mapping relationship from the spherical projection to the plane, so that each pixel point of the original image can correspond to a position on the spherical surface. Based on the spherical feature point set, the ideal display position matrix and the actual display position matrix are constructed. These two matrices represent the position where each feature point should be displayed under ideal conditions and the actual display position of the feature point during the actual display process. The ideal display position matrix is ​​a uniformly distributed coordinate set, which represents the ideal projection of the image without distortion. The actual display position matrix is ​​generated according to the physical layout and pixel distribution of the LED spherical display screen. By calculating the Euclidean distance between the two, the initial distortion data is obtained, which reflects the deviation of each pixel point due to the nonlinear characteristics of the spherical display. The initial distortion data is processed using a spatial consistency loss function. The function of the spatial consistency loss function is to ensure that the display effect of the image in different spatial regions is consistent, and there will be no obvious visual differences due to different display areas. By calculating the initial distortion data, the spatial distortion gradient is obtained, which reflects the distortion change between different regions during the display process. Based on the spatial distortion gradient, a second-order curvature tensor is constructed to calculate the second-order change of the distortion. Gaussian curvature and mean curvature are important curvature indicators that can provide detailed information about the distortion. Through the calculation of these two curvatures, the display deviation is quantified to obtain accurate distortion mapping data. Multi-view processing is performed on the distortion mapping data. The distortion mapping data is transformed into a three-dimensional projection at five preset viewing angles. These viewing angles include the display effects presented when observing the spherical display screen from different angles. Each viewing angle corresponds to a unique distortion matrix, which reflects the distortion at that viewing angle. Through multi-view calculations, it is ensured that the display effects at different viewing angles are reasonably corrected, and the display effects at a certain viewing angle are avoided to be unnatural or deformed. The distortion matrices of multiple viewing angles are weighted with the normalized weight coefficients. The normalized weight coefficient is allocated according to the importance and influence of each perspective to ensure that the distortion correction effect is balanced under different perspectives. This weighted calculation adjusts the weight according to the visual perception model, making the system more accurate and in line with actual needs for distortion correction under each perspective. Through weighted calculation, the perspective weight data is obtained. The perspective weight data and the distortion mapping data are spliced ​​in the channel dimension to generate a fusion evaluation matrix. This matrix integrates the distortion information and corresponding weights under different perspectives, reflecting the comprehensive correction of the display effect under multiple observation perspectives.By performing comprehensive weighted evaluation on the fusion evaluation matrix, a comprehensive evaluation matrix is ​​obtained.

[0033] Step S3, inputting the comprehensive evaluation matrix into the adaptive compensation model to calculate the spatial compensation coefficient, and combining it with the temporal compensation function to obtain the spatiotemporal compensation matrix;

[0034] Specifically, the comprehensive evaluation matrix is ​​input into the adaptive compensation model. The adaptive compensation model calculates the optimal compensation coefficient by analyzing the multidimensional information contained in the evaluation matrix to eliminate the distortion generated during the display process. The model uses the gradient descent method to optimize the compensation effect. The gradient descent method gradually adjusts the model parameters by minimizing the error function to find the optimal order, thereby determining the polynomial order that best suits the current display requirements. The optimal order is used as the basis for constructing the spatial compensation function, helping the model to more accurately reflect the display distortion during the compensation process. The spatial compensation function is constructed based on the optimal polynomial order. The spatial compensation function corrects each pixel of the displayed image to eliminate the distortion caused by spherical projection. Through the polynomial method, the compensation function can provide accurate numerical calculations for effective display correction. In order to optimize the spatial compensation, the error is calculated with the comprehensive evaluation matrix, and the coefficient of each item is solved by the least squares method to ensure that each parameter of the compensation function can accurately meet the display requirements, and a spatial compensation parameter group is obtained. The timing compensation function is constructed based on periodic data. The timing compensation function considers the changes in the display effect in the time dimension, that is, the dynamic changes of the display screen in different time periods, such as fluctuations in brightness and color. By combining the spatial compensation parameter group and the timing compensation function, a joint compensation function is generated, which comprehensively considers the compensation requirements in both space and time, and provides a unified correction solution for the entire display process. The joint compensation function is the core part of the spatiotemporal compensation matrix. Its function is to realize joint compensation in space and time, and ensure that the image can be accurately displayed at different times and spaces. In order to make the compensation effect meet the needs of the actual display screen, the joint compensation function needs to be discretely sampled in the display area. The continuous compensation function is converted into discrete data suitable for digital display to facilitate actual image processing and display. Through discretization, a compensation sampling matrix is ​​obtained, which contains compensation information corresponding to each pixel in the display area. The compensation sampling matrix is ​​nonlinearly mapped to adjust the compensation data so that it can adapt to different display conditions and the characteristics of actual hardware. Nonlinear mapping enables the compensation data to more finely reflect the subtle changes in the display effect and improve the accuracy of compensation. The compensation data will be resampled according to the resolution of the display screen to ensure that the compensation data matches the actual resolution of the display screen. Through resampling, the compensation information of each pixel can be accurately mapped to the actual display position. The standard compensation data is interpolated and extended using a linear interpolation algorithm. In the time and space dimensions, interpolation and extension help fill the gaps between data points, thereby ensuring the continuity and smoothness of the compensation data in the entire display area. Generate a spatiotemporal compensation matrix.

[0035] Step S4, performing gradient calculation on the spatiotemporal compensation matrix at M×N grid points, constructing a compensation gradient field, and performing convolution operation with the original display image to obtain a target vector field;

[0036] Specifically, the spatiotemporal compensation matrix is ​​divided into M rows and N columns of grid points according to the resolution of the display screen. Each grid point corresponds to a small area on the display screen. For each grid point, a unique index number is assigned to it to generate a grid index matrix. The matrix is ​​used to identify the position of each grid point in the entire spatiotemporal compensation matrix. The spatiotemporal compensation values ​​of adjacent grid points in the grid index matrix are differentially operated, the change amount between adjacent grid points is calculated, and the gradient information of the image in the horizontal and vertical directions is obtained. The horizontal gradient represents the change trend of the image in the horizontal direction, and the vertical gradient reflects the change of the image in the vertical direction. The Sobel operator is constructed based on the initial gradient data. The Sobel operator is an edge detection operator. By performing convolution operations on the gradients in the horizontal and vertical directions, the Sobel operator can extract the most significant edge information in the image and obtain the directional gradient field, which reflects the change intensity of the image in different directions. Based on the directional gradient field, the gradient amplitude of each grid point is calculated. The gradient amplitude reflects the change intensity of the image at the point, which is a combination of the horizontal gradient and the vertical gradient. The gradient amplitude matrix is ​​obtained by calculation, and the change intensity of the image in each area is obtained. The gradient amplitude matrix is ​​subjected to non-maximum suppression processing. Non-maximum suppression can effectively reduce noise and redundant data, retain the most significant changes in the image, and obtain a clearer and more concise sparse gradient field. The double threshold method is used to connect the edges and binarize the sparse gradient field. The double threshold method connects the edges and filters the noise by setting two thresholds, corresponding to the high gradient and low gradient areas respectively. In this way, the important edge information in the image is effectively extracted, while unnecessary details are removed to obtain more accurate edge connection data. The edge connection data is convolved with the original display image to obtain a smoothed vector field. The smoothed vector field can effectively smooth the image by combining the edge connection information and the characteristics of the image itself, thereby reducing unnecessary noise caused by compensation. In order to ensure that the smoothed vector field adapts to the actual resolution of the display, it is resampled according to the resolution of the display. The resampling process can adjust the resolution of the smoothed vector field to match the resolution of the display, ensuring that the compensation effect can be accurately mapped to each pixel in the display area. The resampled smoothed vector field is normalized and its value is mapped into a standard range to ensure that the compensation data will not be too large or too small during calculation and display, thus avoiding numerical overflow or loss of precision, and finally generating the target vector field.

[0037] Step S5, input the target vector field into the U-Net structure generator, perform adversarial training in combination with the LED luminous characteristic model and the visual perception model, and obtain compensation optimization parameters through scoring by the PatchGAN discriminator;

[0038] Specifically, the target vector field is input into the encoder part of the U-Net structure generator, which consists of four convolutional layers and downsampling layers. Multi-scale feature extraction is performed on the input data to obtain multi-scale encoded features. In this process, each convolutional layer gradually abstracts the input features through the local receptive field, and reduces the spatial resolution through the downsampling layer to capture feature information at different scales. The multi-scale encoded features are input into the LED luminous characteristic model to analyze the lighting response characteristics of the LED. The luminous characteristic model can accurately calculate the luminous characteristic data of the LED under different voltage and current conditions by establishing the LED lighting response function. This function simulates the relationship between the brightness and the physical driving conditions when the LED is lit, and helps the generator to truly simulate the LED display effect during the image processing process. Based on the luminous characteristic data, a visual perception model is constructed. The visual perception model establishes a brightness perception function by applying the Weber-Fechner law, which reflects the perception characteristics of the human visual system to different brightness changes. The Weber-Fechner law shows that human perception of brightness is a logarithmic function relationship, which enables the model to better simulate real visual perception behavior. By calculating the brightness perception function, visual perception data is generated, so that the system can fully consider the visual characteristics of the human eye when optimizing the display effect. The visual perception data is input into the decoder part of the U-Net structure generator for feature reconstruction. The decoder part is also composed of four layers of deconvolution layers and upsampling layers, which gradually restores the spatial resolution of the feature information and combines the features transmitted by the encoder part through jump connections to generate the final enhanced image. The generated image is divided into image blocks, and each image block is input into the PatchGAN discriminator for authenticity scoring. The PatchGAN discriminator is an adversarial network that focuses on the authenticity of local features. By scoring image blocks, it can effectively detect whether there are unnatural artifacts or distortions in the generated image. The adversarial loss function combines the performance of the generator and the discriminator to guide the generator to gradually optimize its parameters during the training process to generate more natural and realistic images. Through the back-propagation algorithm, the gradient of the adversarial loss function is passed to the generator network, thereby updating the network parameters of the generator and obtaining the optimized generated data. Based on the optimized generated data, parameter extraction is performed to generate a parameter vector representing the optimization result of the generator. These parameter vectors contain all the important information in the compensation process, and the parameter vectors are quantized and mapped. Quantization coding can compress the parameter vectors into a fixed format, while parameter mapping can convert these codes into specific values ​​suitable for the compensation process, and finally generate compensation optimization parameters.

[0039] Step S6: Combined with the illumination intensity data, compensation calculation is performed on the original display image according to the compensation optimization parameters, and an enhanced display image is output.

[0040] Specifically, the illumination intensity data of the LED spherical display screen is obtained to reflect the brightness distribution of the display screen in different areas. Grayscale mapping is performed on the illumination intensity data to convert the illumination data into a standardized grayscale value range for subsequent processing. Gaussian filtering is applied to the mapped illumination data to eliminate noise and smooth the illumination distribution, thereby generating an illumination mapping matrix to describe the illumination intensity of the display screen in different areas. The illumination mapping matrix is ​​weightedly fused with the compensation optimization parameters to obtain compensation weight data. The compensation optimization parameters are derived from previous adversarial training and contain core information on the correction of the distortion and luminous characteristics of the display image. The weighted fusion process combines the effects of illumination intensity and compensation optimization through mathematical operations, so that the final compensation can adapt to both the physical display environment and visual perception requirements. After obtaining the compensation weight data, the original display image is separated according to the RGB three channels, and the red, green and blue channels of the image are processed independently. The separation operation ensures that the compensation of each color channel can be accurate and in place to avoid color mixing problems. The separated color channels are matrix multiplied with the compensation weight data to generate a color compensation matrix. Nonlinear mapping is performed on each pixel based on the color compensation matrix. According to the specific compensation requirements of each pixel, its brightness and color value are dynamically adjusted. The mapping function combines the luminous characteristics of LED and the visual perception model to ensure that the generated brightness balance data can fit both the physical display characteristics and the visual perception of the human eye. In order to synchronize the compensation process with the operation of the display hardware, the brightness balance data is cached and the system clock is synchronized according to the vertical synchronization frequency of the LED spherical display screen to ensure that the generation of compensation data and the refresh of the display screen are consistent, thereby avoiding delays or tearing of the displayed content. After this processing, a timing control signal is generated. Based on the timing control signal, a drive data stream is generated. The drive data stream is the basic information stream used to control the display screen, which converts the brightness balance data and compensation information into instructions that can be recognized by the hardware. In the process of generating the drive data stream, a row-field double buffering mechanism is used to separate the read and write operations of the data stream. The use of the double buffering mechanism effectively reduces read-write conflicts and improves the efficiency and stability of data transmission. At the same time, the drive data stream is output to the drive circuit according to the target scanning magnification to ensure that the compensation data can be accurately transmitted to each area of ​​the display screen. The drive data stream is converted into a light control signal through a constant current driver chip. The role of the constant current driver chip is to ensure that each LED receives a stable current when it is lit, thereby ensuring uniformity of brightness and consistency of display effects. The light control signal directly drives the lighting of the LED, and by dynamically adjusting the brightness and color, it ultimately achieves enhanced display image output.

[0041] In the embodiment of the present invention, the multi-dimensional features of the display image are effectively extracted through the multi-scale Transformer hierarchical CNN network structure, which overcomes the limitations of the traditional single-scale feature extraction method and enhances the comprehensiveness and accuracy of feature extraction. The method of combining spherical coordinate mapping and spatial consistency function is adopted to accurately calculate the display distortion, and the display distortion problem under different observation angles is effectively solved through multi-view data fusion technology. Combined with the adaptive compensation model and the timing compensation function, the joint optimization in the spatial and temporal dimensions is realized, and the dynamic distortion in the display process is effectively reduced. By introducing the construction method of the compensation gradient field and the target vector field, the geometric correction of the display image is more accurate and the edge transition is more natural. The adversarial training mechanism of the U-Net structure generator and the PatchGAN discriminator is adopted, combined with the LED luminous characteristic model and the visual perception model, the authenticity and stability of the visual enhancement effect are significantly improved. Through the fusion operation of the light intensity data and the compensation optimization parameters, in conjunction with the frame synchronization control mechanism, the real-time dynamic optimization of the display image is realized, and the display quality under different ambient lighting conditions is guaranteed.

[0042] In a specific embodiment, the process of executing step S1 may specifically include the following steps:

[0043] The original display image of the LED spherical display screen is input into the multi-scale Transformer hierarchical CNN network, and the original display image is convolved through 3×3, 5×5, and 7×7 scale convolution layers to obtain initial feature data;

[0044] The initial feature data is input into the Transformer encoder with 8 attention heads for processing, and the dimension of each attention head is 64 to obtain the attention feature data;

[0045] The attention feature data are input into the local feature extraction module, the medium-scale feature extraction module and the global feature extraction module respectively to obtain the local feature map, the medium-scale feature map and the global feature map;

[0046] The local feature map is processed by 1×1 convolution to adjust the number of channels to obtain the first layer feature map; the medium-scale feature map is processed by spatial pyramid pooling, and 2×2 and 4×4 pooling kernels are used to extract multi-scale information to obtain the second layer feature map; the global feature map is processed by self-attention mechanism, and the correlation between each pixel in the feature map is calculated to obtain the third layer feature map;

[0047] The first layer feature map, the second layer feature map and the third layer feature map are input into the feature fusion module, and the channels are spliced ​​in a cascade manner to obtain fused feature data. The fused feature data is normalized, and a multi-dimensional feature atlas is generated through a residual connection structure.

[0048] Specifically, the original display image is input into a multi-scale Transformer hierarchical CNN network to extract multi-level feature information. The input image is convolved with three different scales of convolution kernels: 3×3, 5×5, and 7×7 to capture feature information at different scales. Assume that the size of the input image is ,in and are the height and width of the image, is the number of channels. After processing by three convolution kernels, the convolution operation is expressed as:

[0049] ;

[0050] in represents the size of the convolution kernel, Indicates location The output eigenvalues ​​of is the weight of the convolution kernel, is the pixel value of the input image. Through the operation of three convolution kernels, the initial feature data is obtained ,in Indicates the number of channels of the feature map. The input is processed into a Transformer encoder with 8 attention heads. The Transformer encoder uses the attention mechanism to capture the long-range dependencies between different regions in the image. The attention mechanism is implemented by calculating the weighted relationship between the query, key, and value, and its formula is:

[0051] ;

[0052] in are matrices of queries, keys, and values, respectively, represents the length of the input sequence, is the dimension of the attention head (here 64). For 8 attention heads, the calculation formula for each head is the same, and the results are finally transformed and concatenated to obtain the global attention feature. ,in . Attention feature data The local feature extraction module, the medium-scale feature extraction module and the global feature extraction module are input respectively to generate local feature maps. , medium-scale feature map and global feature map The local feature extraction module focuses on the features in a small range. Perform a 1×1 convolution to adjust the number of channels:

[0053] ;

[0054] in Representation 1 The weight of the convolution kernel. For the medium-scale feature extraction module, multi-scale pooling is performed through the spatial pyramid pooling technique. The spatial pyramid pooling operation applies pooling kernels of different scales (such as 2×2 and 4×4) to each feature block. The formula is:

[0055] ;

[0056] The kernel represents the range of the pooling window. The global feature extraction module calculates the global correlation between pixels in the feature map through the self-attention mechanism. Its formula is similar to the attention calculation in Transformer. The global feature map is obtained by pixel-by-pixel dependency modeling. The local feature map, medium-scale feature map and global feature map are input into the feature fusion module as the first layer, second layer and third layer feature maps respectively. The feature fusion module performs channel splicing in a cascade manner to generate fused feature data. :

[0057] ;

[0058] Where concat represents the concatenation operation on the channel dimension. The fused feature data is then normalized to standardize the feature range and enhance the training stability of the model:

[0059] ;

[0060] in and Channel The fused feature data is processed through the residual connection structure to generate a multi-dimensional feature atlas :

[0061] ;

[0062] This residual structure ensures the effective transfer of information during the fusion process and avoids gradient disappearance.

[0063] In a specific embodiment, the process of executing step S2 may specifically include the following steps:

[0064] According to the spherical coordinate mapping relationship, the plane coordinate points in the multidimensional feature atlas are transformed to obtain a spherical feature point set;

[0065] Based on the spherical feature point set, an ideal display position matrix and an actual display position matrix are constructed, and initial distortion data is obtained by Euclidean distance calculation;

[0066] The initial distortion data is calculated based on the spatial consistency loss function to obtain the spatial distortion gradient, the second-order curvature tensor is constructed according to the spatial distortion gradient, and the distortion mapping data is obtained by calculating the Gaussian curvature and the mean curvature;

[0067] The distortion mapping data is transformed into a three-dimensional projection at five preset viewing angles. Each viewing angle corresponds to a distortion matrix to obtain a multi-view distortion matrix. The multi-view distortion matrix is ​​weighted and calculated with a normalized weight coefficient to obtain viewing angle weight data.

[0068] The perspective weight data and the distortion mapping data are spliced ​​in the channel dimension to generate a fusion evaluation matrix, and the fusion evaluation matrix is ​​comprehensively weighted evaluated to generate a comprehensive evaluation matrix.

[0069] Specifically, according to the spherical coordinate mapping relationship, the plane coordinate points in the multidimensional feature atlas are converted into a spherical feature point set. Mapping to 3D spherical coordinates , the specific formula is:

[0070]

[0071] in is the radius of the sphere, is the pitch angle (depending on coordinate), is the azimuth (depending on Through this mapping, each plane coordinate point is converted into the corresponding three-dimensional spherical feature point, generating a spherical feature point set ,in Is the total number of feature points. Based on the spherical feature point set, the ideal display position matrix and the actual display position matrix are constructed. Ideal display position matrix In an ideal situation, the 3D position of each point is generated by a uniform distribution, such as uniformly sampling the angle range and . Actual display position matrix It comes from the physical mapping of the actual display screen, and is obtained by measuring the optical deviation of the spherical feature points. The difference between the two is calculated through the Euclidean distance to obtain the initial distortion data:

[0072] ;

[0073] in It is The distortion value of each point, are the corresponding components of the ideal and actual coordinates respectively. Initial distortion dataset Reflects the overall deviation distribution of the display. Optimize the initial distortion data based on the spatial consistency loss function. Spatial consistency loss function Defined as:

[0074] ;

[0075] in Indicate point The loss function aims to minimize the distortion difference between neighboring points to maintain spatial consistency. , and obtain the spatial distortion gradient , which is used to describe the change of distortion in local space. Based on the spatial distortion gradient , construct the second-order curvature tensor The curvature tensor is used to describe the curvature change of each point in different directions. Its calculation formula is:

[0076] ;

[0077] in is the parameter direction in the local coordinate system. and mean curvature Further quantification of the distortion properties:

[0078] ;

[0079] Gaussian curvature reflects the overall concavity of the local surface, while the mean curvature reflects the average deformation degree of the local area. and , calculate the distortion mapping data The distortion mapping data is transformed into a three-dimensional projection at five preset viewing angles. These viewing angles correspond to the five main viewing directions of the spherical display screen, such as the front, left, right, top and bottom. The projection transformation of each viewing angle is calculated by the viewing angle matrix calculate:

[0080] ;

[0081] in It is The distortion matrix at each viewing angle is is the projection matrix corresponding to the viewing angle. By combining the distortion matrices of the five viewing angles, a multi-view distortion matrix is ​​formed. In order to quantify the distortion effect under multi-view, the multi-view distortion matrix and the normalized weight coefficient Perform weighted calculation:

[0082] ;

[0083] in Indicates The weight of each perspective satisfies . Weighted view weight data The distortion characteristics under all viewing angles are integrated. With distortion map data Perform concatenation on the channel dimension to generate a fusion evaluation matrix :

[0084] ;

[0085] The fusion evaluation matrix is ​​subjected to comprehensive weighted evaluation to generate a comprehensive evaluation matrix :

[0086] ;

[0087] in is the normalization factor, The weight of each point. Comprehensive evaluation matrix It is the core data used for compensation optimization, including distortion characteristics and local curvature information under multiple perspectives.

[0088] In a specific embodiment, the process of executing step S3 may specifically include the following steps:

[0089] The comprehensive evaluation matrix is ​​input into the adaptive compensation model, and the optimal order is calculated by the gradient descent method to obtain the optimal polynomial order;

[0090] The spatial compensation function is constructed based on the optimal polynomial order to obtain the compensation polynomial basis function;

[0091] The error calculation is performed between the compensation polynomial basis function and the comprehensive evaluation matrix, and each coefficient is solved by the least square method to obtain a spatial compensation parameter group, and a timing compensation function is constructed for the spatial compensation parameter group based on the period;

[0092] The spatial compensation parameter group and the temporal compensation function are combined to obtain a joint compensation function, and the joint compensation function is discretely sampled in the display area to obtain a compensation sampling matrix;

[0093] The compensation sampling matrix is ​​nonlinearly mapped and resampled according to the display resolution to obtain standard compensation data. The standard compensation data is interpolated and expanded in the time dimension and space dimension through a linear interpolation algorithm to generate a spatiotemporal compensation matrix.

[0094] Specifically, the comprehensive evaluation matrix Enter the adaptive compensation model, which uses the gradient descent method to optimize the order of the polynomial to find the optimal compensation expression. Assume that the compensation polynomial is expressed as:

[0095] ;

[0096] in and is the two-dimensional space coordinate, are the polynomial coefficients to be optimized, is the order of the polynomial. The adaptive compensation model is constructed by setting an initial order And the target loss function , and continuously adjust through the gradient descent method , so that the loss function Minimize, the formula is:

[0097] ;

[0098] in It is the first in the comprehensive evaluation matrix The value of a point, is the predicted value of the polynomial at the corresponding point, is the total number of points. Update the order by gradient descent:

[0099] ;

[0100] in is the learning rate, and finally the optimal polynomial order is obtained Based on the optimal order , construct compensation polynomial basis functions ,in These basis functions form the basis of the compensation function, which is used to describe the compensation pattern of the spherical display. The compensation polynomial basis functions are combined with the comprehensive evaluation matrix Perform error calculation and solve the coefficient of each term by the least squares method The goal of the least squares method is to minimize the overall error:

[0101] ;

[0102] Through each Find the partial derivative and set it to zero:

[0103] ;

[0104] Solve for the coefficients These coefficients constitute the spatial compensation parameter group , describing the compensation requirements of different spatial points. A timing compensation function is constructed based on the spatial compensation parameter group to capture the dynamic changes in the time dimension. Assume that the timing compensation function is , which is of the form:

[0105] ;

[0106] in It's time. is the compensation period, and is the timing parameter, is the harmonic order. By combining and , generating a joint compensation function:

[0107] ;

[0108] in Indicates the comprehensive compensation requirements in space and time. Discretely sample the joint compensation function in the display area to generate the compensation sampling matrix ,in and are the horizontal and vertical resolutions of the display, is the number of sampling points in the time dimension. In order to improve the compensation accuracy, Perform nonlinear mapping, such as using a logarithmic function to compress the dynamic range:

[0109] ;

[0110] in To adjust the parameters, obtain the compensation data after nonlinear mapping. Resample the compensation data according to the actual resolution of the display screen to generate standard compensation data In order to ensure the smoothness of time and space dimensions, a linear interpolation algorithm is used to interpolate and expand in the time and space dimensions. The formula is:

[0111] ;

[0112] in and are adjacent interpolation points. After interpolation and expansion, the final spatiotemporal compensation matrix is ​​generated .

[0113] In a specific embodiment, the process of executing step S4 may specifically include the following steps:

[0114] Divide the spatiotemporal compensation matrix into M rows and N columns of grid points according to the display screen resolution, and index each grid point to obtain a grid index matrix;

[0115] Perform differential operation on the spatiotemporal compensation values ​​of adjacent grid points in the grid index matrix, calculate the horizontal gradient and the vertical gradient, and obtain the initial gradient data;

[0116] Construct the Sobel operator based on the initial gradient data, and perform convolution operations on the horizontal and vertical gradients to obtain the directional gradient field;

[0117] Based on the directional gradient field, the gradient amplitude of each grid point is calculated to obtain the gradient amplitude matrix, and the gradient amplitude matrix is ​​subjected to non-maximum suppression processing to obtain a sparse gradient field;

[0118] The sparse gradient field is edge-connected and binarized by a double threshold method to obtain edge-connected data;

[0119] The edge connection data and the original display image are convolved to obtain a smooth vector field, which is then resampled according to the display resolution and normalized to generate a target vector field.

[0120] Specifically, the time-space compensation matrix is ​​adjusted according to the resolution of the display screen. Divide into OK Assume the display resolution is , the size of each grid cell is ,but and . Index each grid point and generate a grid index matrix ,in and Indicates the vertical and horizontal index of the grid point, Used to identify the position of the grid point in the entire matrix. Calculate the difference in the temporal and spatial compensation values ​​between adjacent grid points and perform horizontal and vertical gradient calculations. Horizontal gradient and vertical gradient The difference formulas are:

[0121] ;

[0122] ;

[0123] in Represents a grid point The differential operation can capture the changes in the temporal and spatial compensation values ​​of adjacent grid points, thereby generating the initial gradient data and Based on the initial gradient data, the Sobel operator is constructed to process the gradient. The Sobel operator is a convolution kernel used for image edge detection and is used to enhance gradient changes. The horizontal Sobel operator and the Sobel operator in the vertical direction They are:

[0124] ;

[0125] By and Perform convolution operation with the corresponding Sobel operator to obtain the directional gradient field :

[0126] ;

[0127] Where * represents the convolution operation, is the magnitude of the directional gradient at each grid point. Based on the directional gradient field , calculate the gradient magnitude at each grid point , the formula is:

[0128] ;

[0129] Gradient magnitude matrix Non-maximum suppression is performed to retain the sharp edge and suppress redundant noise. Perform interpolation comparison, and if the amplitude of the current point is not the local maximum, set it to zero. After completing non-maximum suppression, generate a sparse gradient field The double threshold method is applied to the sparse gradient field for edge connection and binarization. The double threshold method uses a high threshold and low threshold , the rules are as follows: If , marked as a strong edge point; if , marked as weak edge points, and decided whether to keep them based on the connectivity of the adjacent strong edge points. This method generates binary edge connection data , for subsequent processing. Connect edge data With the original display image Perform convolution operation to obtain a smooth vector field This process is expressed by the formula:

[0130] ;

[0131] in is the radius of the convolution kernel, Represents the smoothing characteristics of each grid point. The smoothed vector field is calculated according to the resolution of the display screen. Resample to align with the pixels of the display. After resampling, normalize the smoothed vector field:

[0132] ;

[0133] Normalized smoothed vector field Mapping the value of each point to the standard range [0,1] eliminates the impact of brightness or gradient differences in different areas.

[0134] In a specific embodiment, the process of executing step S5 may specifically include the following steps:

[0135] The target vector field is input into the encoder part of the U-Net structure generator, and features are extracted through four convolutional layers and downsampling layers to obtain multi-scale encoding features;

[0136] The multi-scale coding features are input into the LED luminous characteristic model, the LED lighting response function is established and the luminous characteristic data is calculated;

[0137] A visual perception model is constructed based on the luminous characteristic data, and a brightness perception function is established through the Weber-Fechner law to calculate the visual perception data;

[0138] The visual perception data is input into the decoder part of the U-Net structure generator, and the features are reconstructed through four deconvolution layers and upsampling layers to obtain the generated image;

[0139] The generated image is divided into image blocks and input into the PatchGAN discriminator for authenticity scoring to obtain local feature scores. An adversarial loss function is constructed based on the local feature scores. The network parameters are updated through back propagation to obtain optimized generated data.

[0140] Parameters are extracted based on the optimized generated data to obtain a parameter vector, and the parameter vector is quantized and encoded and parameter mapped to generate compensation optimization parameters.

[0141] Specifically, the target vector field Enter the encoder part of the U-Net structure generator. The U-Net encoder consists of four convolutional layers and downsampling layers. The convolution operation of each layer is expressed as:

[0142] ;

[0143] in It is The input feature map of the layer, and They are convolution kernel and bias respectively, * indicates convolution operation, is the activation function (such as ReLU). Downsampling is achieved through maximum pooling, which is expressed as:

[0144] ;

[0145] in is the downsampled coordinate. After four layers of convolution and downsampling, the encoder can extract multi-scale coding features. , whose size decreases layer by layer, while the number of channels increases layer by layer, containing the global and local features of the target vector field. Input into the LED luminous characteristic model to establish the LED lighting response function. The LED lighting response is mainly affected by the driving current and operating voltage The brightness response function is expressed as:

[0146] ;

[0147] in is the luminous brightness, is the luminous efficiency, is the response coefficient of the LED material. By combining the coding feature with the response function, the luminous characteristic data is calculated. , reflecting the luminous characteristics of each pixel. Based on the luminous characteristic data , construct a visual perception model. According to the Weber-Fechner law, the human eye's perception of brightness changes is a logarithmic function relationship, and the brightness perception function is expressed as:

[0148] ;

[0149] in is the perceived brightness value, and is the adjustment parameter. Through this perception model, the visual perception data is calculated , considering the relationship between physical luminous properties and human eye perception, so as to improve the actual display effect of compensation. Enter the decoder part of the U-Net structure generator. The decoder also consists of four deconvolution layers and upsampling layers. The deconvolution operation is expressed as:

[0150] ;

[0151] in It is The output of the layer decoder, is the deconvolution kernel. Upsampling is achieved through interpolation, which is expressed as:

[0152] ;

[0153] in is the upsampling factor. After four layers of deconvolution and upsampling, the output image is finally generated. , is an enhanced image based on perceptual optimization. The generated image Divide the image into blocks, the size of each image block is ,For example The divided image blocks are input into the PatchGAN discriminator. PatchGAN focuses on evaluating the authenticity of local images. Its scoring mechanism calculates the true probability of each image block. Implementation. The loss function of PatchGAN is defined as:

[0154] ;

[0155] in and are the distributions of real images and generated images respectively. The score of the discriminator Used to guide the generator to optimize image quality. Based on the local feature scoring of PatchGAN, an adversarial loss function is constructed. To optimize the generator. The adversarial loss is combined with the reconstruction error of the generator :

[0156] ;

[0157] in is the weight coefficient, which is used to balance the generator's adversarial ability and reconstruction quality. The parameters of the generator are updated through back propagation, and the generator is gradually optimized to obtain the optimized generated data. Based on the optimized generated data, parameter extraction is performed to generate a parameter vector describing the image features. These parameters are mapped into a fixed-length binary sequence through quantization encoding , the formula is:

[0158] ;

[0159] in is the quantized bit depth. The mapped parameters are further analyzed to generate compensation optimization parameters , used to guide real-time compensation of the display.

[0160] In a specific embodiment, the process of executing step S6 may specifically include the following steps:

[0161] Obtain the light intensity data of the LED spherical display screen, perform grayscale mapping and Gaussian filtering on the light intensity data, and obtain a light mapping matrix;

[0162] The illumination mapping matrix and the compensation optimization parameters are weightedly fused to obtain compensation weight data, the original display image is separated according to the RGB three channels, and matrix multiplication operation is performed with the compensation weight data to obtain a color compensation matrix;

[0163] Based on the color compensation matrix, nonlinear mapping is performed on each pixel to obtain brightness balance data. According to the vertical synchronization frequency of the LED spherical display screen, the brightness balance data is cached and synchronized with the system clock to obtain a timing control signal.

[0164] Generate a driving data stream based on the timing control signal, separate the reading and writing of the driving data stream through the row-field double buffer mechanism, and output it to the driving circuit according to the target scanning ratio to obtain the LED driving sequence;

[0165] The LED driving sequence is converted into a light-emitting control signal through a constant current driver chip to output an enhanced display image.

[0166] Specifically, the light intensity data of the LED spherical display screen is obtained. These data are collected in real time by light sensors distributed on the surface of the display screen, and the light intensity values ​​are expressed in a matrix Indicates that and are the height and width of the display screen respectively. Grayscale mapping is performed on the light intensity data, and the original light intensity value is standardized to the grayscale range [0,255]. The formula is:

[0167] ;

[0168] in represents the normalized grayscale value, and are the minimum and maximum values ​​of the light intensity matrix respectively. Gaussian filtering is performed to eliminate noise and smooth the light distribution. The operation of Gaussian filtering is defined as:

[0169] ;

[0170] in is the Gaussian kernel function, Controls the standard deviation of the filter, Determine the size of the kernel. After Gaussian filtering, the light mapping matrix is ​​obtained , reflecting the illumination distribution of the display at different locations. Optimization parameters with compensation Perform weighted fusion to generate compensation weight data The formula for weighted fusion is:

[0171] ;

[0172] in and is the weight coefficient used to balance the contribution of illumination intensity and compensation parameters to display optimization. Separate by RGB three channels, and the data of each channel is represented as . Combine channel data with compensation weight matrix Perform point-by-point multiplication to generate a color compensation matrix :

[0173] ;

[0174] in Indicates the channel index. Based on the color compensation matrix , nonlinear mapping is performed on each pixel to equalize the brightness. The nonlinear mapping adopts the form of logarithmic function, and the formula is:

[0175] ;

[0176] in and It is an adjustment parameter used to control the dynamic range and brightness balance of the mapping. After nonlinear mapping, the brightness balance data is obtained. , ensuring that the brightness of the image is more natural visually. In order to ensure the synchronization of the displayed image, the brightness balance data Buffer and clock synchronization with the vertical synchronization signal of the display. Assume that the vertical synchronization frequency of the display is , then the cache time for each frame of data is . Clock synchronization is achieved by equalizing the data Store in cache queue And read frame by frame according to the rhythm of the vertical sync signal:

[0177] ;

[0178] in Is the timestamp of the current frame. After synchronization, the timing control signal is generated , used to guide the data refresh of the display. Based on the timing control signal , generate driving data flow The driving data stream needs to be read and written separately through the line and field double buffer mechanism to ensure the stability and real-time performance of data transmission. The line buffer and field buffer are used to store the data of each line and each frame respectively. The formula is:

[0179] ;

[0180] in and It is the data of line buffer and field buffer. Through the double buffer mechanism, the driving data stream is scanned at the target magnification. Output to the drive circuit. The drive circuit receives the drive data stream , the data is converted into a light-emitting control signal through a constant current driver chip The constant current driver chip ensures that each LED receives a stable current when it is lit. The formula is:

[0181] ;

[0182] in is the driving current, It is the current limiting resistor of the driving circuit. Light-emitting control signal The LED is driven to light up and an enhanced display image is output.

[0183] The above describes the visual enhancement method of the LED creative spherical display screen in the embodiment of the present invention. The following describes the visual enhancement device of the LED creative spherical display screen in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, a visual enhancement device for a creative spherical LED display screen includes:

[0184] A feature extraction module is used to input the original display image of the LED spherical display screen into a multi-scale Transformer hierarchical CNN network for feature extraction to obtain a multi-dimensional feature atlas;

[0185] The feature mapping module is used to perform feature mapping on the multi-dimensional feature atlas, project the pixel points into the spherical coordinate system to calculate the display deviation, generate distortion mapping data, and calculate the multi-view data based on the spatial consistency function to obtain a comprehensive evaluation matrix;

[0186] The adaptive compensation module is used to input the comprehensive evaluation matrix into the adaptive compensation model to calculate the spatial compensation coefficient and combine it with the time series compensation function to obtain the time-space compensation matrix;

[0187] A calculation module is used to perform gradient calculation on the spatiotemporal compensation matrix at M×N grid points, construct a compensation gradient field, and perform convolution operation with the original display image to obtain a target vector field;

[0188] The compensation optimization module is used to input the target vector field into the U-Net structure generator, perform adversarial training based on the LED luminous characteristic model and the visual perception model, and obtain the compensation optimization parameters through the PatchGAN discriminator score;

[0189] The output module is used to combine the light intensity data, perform compensation calculation on the original display image according to the compensation optimization parameters, and output the enhanced display image.

[0190] Through the cooperation of the above components, the multi-scale Transformer hierarchical CNN network structure can effectively extract the multi-dimensional features of the display image, overcome the limitations of the traditional single-scale feature extraction method, and enhance the comprehensiveness and accuracy of feature extraction. The method of combining spherical coordinate mapping and spatial consistency function is used to accurately calculate the display distortion, and the multi-view data fusion technology is used to effectively solve the display distortion problem under different observation angles. Combined with the adaptive compensation model and the timing compensation function, the joint optimization in the spatial and temporal dimensions is realized, and the dynamic distortion in the display process is effectively reduced. By introducing the construction method of the compensation gradient field and the target vector field, the geometric correction of the display image is more accurate and the edge transition is more natural. The adversarial training mechanism of the U-Net structure generator and the PatchGAN discriminator is adopted, combined with the LED luminous characteristic model and the visual perception model, the authenticity and stability of the visual enhancement effect are significantly improved. Through the fusion operation of the light intensity data and the compensation optimization parameters, combined with the frame synchronization control mechanism, the real-time dynamic optimization of the display image is realized, and the display quality under different ambient lighting conditions is guaranteed.

[0191] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0192] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0193] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0194] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0195] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0196] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0197] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visual enhancement method for a creative spherical LED display screen, characterized in that: The method comprises: The original display image of the LED spherical display screen is input into a multi-scale Transformer hierarchical CNN network for feature extraction to obtain a multi-dimensional feature atlas; specifically, the method includes: inputting the original display image of the LED spherical display screen into a multi-scale Transformer hierarchical CNN network, performing convolution processing on the original display image through 3×3, 5×5, and 7×7 scale convolution layers to obtain initial feature data; inputting the initial feature data into a Transformer encoder with 8 attention heads for processing, each with a dimension of 64, to obtain attention feature data; and inputting the attention feature data into a local feature extraction module, a medium-scale feature extraction module, and a global feature extraction module respectively. , obtain a local feature map, a medium-scale feature map and a global feature map; perform 1×1 convolution processing on the local feature map to adjust the number of channels to obtain a first-layer feature map; perform spatial pyramid pooling processing on the medium-scale feature map, use 2×2 and 4×4 pooling kernels to extract multi-scale information, and obtain a second-layer feature map; perform self-attention mechanism processing on the global feature map, calculate the correlation between each pixel in the feature map, and obtain a third-layer feature map; input the first-layer feature map, the second-layer feature map and the third-layer feature map into the feature fusion module, perform channel splicing in a cascade manner, obtain fused feature data, and normalize the fused feature data, and generate a multi-dimensional feature map set through a residual connection structure; The multidimensional feature atlas is feature mapped, the pixel points are projected into a spherical coordinate system to calculate the display deviation, distortion mapping data is generated, and multi-view data is calculated based on a spatial consistency function to obtain a comprehensive evaluation matrix; specifically, the method comprises: according to the spherical coordinate mapping relationship, the plane coordinate points in the multidimensional feature atlas are transformed to obtain a spherical feature point set; based on the spherical feature point set, an ideal display position matrix and an actual display position matrix are constructed, and initial distortion data are obtained by Euclidean distance calculation; based on the spatial consistency loss function, the initial distortion data is operated to obtain a spatial distortion gradient degree, constructing a second-order curvature tensor according to the spatial distortion gradient, and obtaining distortion mapping data by Gaussian curvature and mean curvature calculation; performing three-dimensional projection transformation on the distortion mapping data under five preset observation angles, each angle corresponding to a distortion matrix, obtaining a multi-angle distortion matrix, and performing weighted calculation on the multi-angle distortion matrix and a normalized weight coefficient to obtain perspective weight data; splicing the perspective weight data and the distortion mapping data in the channel dimension to generate a fusion evaluation matrix, and performing comprehensive weighted evaluation on the fusion evaluation matrix to generate a comprehensive evaluation matrix; The comprehensive evaluation matrix is ​​input into the adaptive compensation model to calculate the spatial compensation coefficient, and combined with the temporal compensation function to obtain the spatiotemporal compensation matrix; Performing gradient calculation on the spatiotemporal compensation matrix at M×N grid points to construct a compensation gradient field, and performing convolution operation with the original display image to obtain a target vector field; The target vector field is input into the U-Net structure generator, adversarial training is performed in combination with the LED luminous characteristic model and the visual perception model, and the compensation optimization parameters are obtained through scoring by the PatchGAN discriminator; In combination with the illumination intensity data, compensation calculation is performed on the original display image according to the compensation optimization parameters to output an enhanced display image.

2. The visual enhancement method of the LED creative spherical display screen according to claim 1, characterized in that: The comprehensive evaluation matrix is ​​input into the adaptive compensation model to calculate the spatial compensation coefficient, and combined with the time series compensation function to obtain the time-space compensation matrix, including: The comprehensive evaluation matrix is ​​input into the adaptive compensation model, and the optimal order is calculated by the gradient descent method to obtain the optimal polynomial order; Constructing a spatial compensation function based on the optimal polynomial order to obtain a compensation polynomial basis function; Performing error calculation on the compensation polynomial basis function and the comprehensive evaluation matrix, solving each coefficient by the least square method to obtain a spatial compensation parameter group, and constructing a timing compensation function for the spatial compensation parameter group based on a period; Combining the spatial compensation parameter group and the temporal compensation function to obtain a joint compensation function, and performing discrete sampling on the joint compensation function in a display area to obtain a compensation sampling matrix; The compensation sampling matrix is ​​nonlinearly mapped and resampled according to the display screen resolution to obtain standard compensation data, and the standard compensation data is interpolated and expanded in the time dimension and the space dimension by a linear interpolation algorithm to generate a spatiotemporal compensation matrix.

3. The visual enhancement method of the LED creative spherical display screen according to claim 2, characterized in that: The step of performing gradient calculation on the spatiotemporal compensation matrix at M×N grid points to construct a compensation gradient field and performing convolution operation with the original display image to obtain a target vector field includes: Dividing the spatiotemporal compensation matrix into M rows and N columns of grid points according to the resolution of the display screen, and indexing each grid point to obtain a grid index matrix; Performing differential operation on the spatiotemporal compensation values ​​of adjacent grid points in the grid index matrix, calculating the horizontal gradient and the vertical gradient, and obtaining initial gradient data; Constructing a Sobel operator based on the initial gradient data, and performing a convolution operation on the horizontal direction gradient and the vertical direction gradient to obtain a directional gradient field; Based on the directional gradient field, the gradient amplitude of each grid point is calculated to obtain a gradient amplitude matrix, and non-maximum suppression processing is performed on the gradient amplitude matrix to obtain a sparse gradient field; Performing edge connection and binarization processing on the sparse gradient field by a double threshold method to obtain edge connection data; A convolution operation is performed on the edge connection data and the original display image to obtain a smooth vector field, and the smooth vector field is resampled according to the resolution of the display screen and normalized to generate a target vector field.

4. The visual enhancement method of the LED creative spherical display screen according to claim 3, characterized in that: The target vector field is input into the U-Net structure generator, adversarial training is performed in combination with the LED luminous characteristic model and the visual perception model, and the compensation optimization parameters are obtained through the PatchGAN discriminator scoring, including: The target vector field is input into the encoder part of the U-Net structure generator, and feature extraction is performed through four convolutional layers and a downsampling layer to obtain multi-scale coding features; Input the multi-scale coding features into the LED luminous characteristic model, establish the LED lighting response function and calculate the luminous characteristic data; Building a visual perception model based on the luminous characteristic data, establishing a brightness perception function through the Weber-Fechner law and calculating visual perception data; Input the visual perception data into the decoder part of the U-Net structure generator, perform feature reconstruction through four deconvolution layers and upsampling layers, and obtain a generated image; Divide the generated image into image blocks, input the blocks into a PatchGAN discriminator for authenticity scoring, obtain a local feature score, construct an adversarial loss function based on the local feature score, update the network parameters through back propagation, and obtain optimized generated data; Parameter extraction is performed based on the optimized generated data to obtain a parameter vector, and quantization encoding and parameter mapping are performed on the parameter vector to generate compensation optimization parameters.

5. The visual enhancement method of the LED creative spherical display screen according to claim 4, characterized in that: The step of combining the illumination intensity data, performing compensation calculation on the original display image according to the compensation optimization parameter, and outputting an enhanced display image comprises: Acquire light intensity data of the LED spherical display screen, perform grayscale mapping and Gaussian filtering on the light intensity data, and obtain a light mapping matrix; The illumination mapping matrix is ​​weightedly fused with the compensation optimization parameters to obtain compensation weight data, the original display image is separated according to the RGB three channels, and a matrix multiplication operation is performed with the compensation weight data to obtain a color compensation matrix; Based on the color compensation matrix, nonlinear mapping is performed on each pixel to obtain brightness balance data, and according to the vertical synchronization frequency of the LED spherical display screen, the brightness balance data is cached and synchronized with the system clock to obtain a timing control signal; Generate a driving data stream based on the timing control signal, perform read-write separation on the driving data stream through a row-field double buffer mechanism, and output it to a driving circuit according to a target scanning ratio to obtain an LED driving sequence; The LED driving sequence is converted into a light-emitting control signal through a constant current driving chip, and an enhanced display image is output.

6. A visual enhancement device for a creative LED spherical display screen, characterized in that: The method for performing the visual enhancement of the LED creative spherical display screen according to any one of claims 1 to 5, wherein the visual enhancement device of the LED creative spherical display screen comprises: The feature extraction module is used to input the original display image of the LED spherical display screen into the multi-scale Transformer hierarchical CNN network for feature extraction to obtain a multi-dimensional feature atlas; specifically, the module comprises: inputting the original display image of the LED spherical display screen into the multi-scale Transformer hierarchical CNN network, performing convolution processing on the original display image through 3×3, 5×5, and 7×7 scale convolution layers to obtain initial feature data; inputting the initial feature data into the Transformer encoder of 8 attention heads for processing, each of which has a dimension of 64, to obtain attention feature data; and inputting the attention feature data into the local feature extraction module, the medium-scale feature extraction module, and the global feature extraction module, respectively. A feature extraction module is provided to obtain a local feature map, a medium-scale feature map and a global feature map; the local feature map is subjected to 1×1 convolution processing to adjust the number of channels to obtain a first-layer feature map; the medium-scale feature map is subjected to spatial pyramid pooling processing, and multi-scale information is extracted using 2×2 and 4×4 pooling kernels to obtain a second-layer feature map; the global feature map is subjected to self-attention mechanism processing, and the correlation between each pixel in the feature map is calculated to obtain a third-layer feature map; the first-layer feature map, the second-layer feature map and the third-layer feature map are input into a feature fusion module, and channel splicing is performed in a cascade manner to obtain fused feature data, and the fused feature data is normalized, and a multi-dimensional feature atlas is generated through a residual connection structure; The feature mapping module is used to perform feature mapping on the multidimensional feature atlas, project pixel points into a spherical coordinate system to calculate display deviation, generate distortion mapping data, and calculate multi-view data based on a spatial consistency function to obtain a comprehensive evaluation matrix; specifically, the module comprises: transforming the plane coordinate points in the multidimensional feature atlas according to the spherical coordinate mapping relationship to obtain a spherical feature point set; constructing an ideal display position matrix and an actual display position matrix based on the spherical feature point set, and obtaining initial distortion data through Euclidean distance calculation; operating the initial distortion data based on a spatial consistency loss function to obtain a spatial distortion matrix. The method comprises the following steps: calculating a spatial distortion gradient, constructing a second-order curvature tensor according to the spatial distortion gradient, and obtaining distortion mapping data by calculating Gaussian curvature and mean curvature; performing a three-dimensional projection transformation on the distortion mapping data under five preset observation angles, wherein each angle corresponds to a distortion matrix, and obtaining a multi-view distortion matrix; performing weighted calculation on the multi-view distortion matrix and a normalized weight coefficient to obtain perspective weight data; splicing the perspective weight data and the distortion mapping data in the channel dimension to generate a fusion evaluation matrix, and performing a comprehensive weighted evaluation on the fusion evaluation matrix to generate a comprehensive evaluation matrix; An adaptive compensation module, used to input the comprehensive evaluation matrix into an adaptive compensation model to calculate the spatial compensation coefficient, and combine it with the temporal compensation function to obtain a spatiotemporal compensation matrix; A calculation module, used for performing gradient calculation on the spatiotemporal compensation matrix at M×N grid points, constructing a compensation gradient field, and performing convolution operation with the original display image to obtain a target vector field; A compensation optimization module is used to input the target vector field into a U-Net structure generator, perform adversarial training in combination with an LED luminous characteristic model and a visual perception model, and obtain compensation optimization parameters through scoring by a PatchGAN discriminator; The output module is used to combine the light intensity data, perform compensation calculation on the original display image according to the compensation optimization parameters, and output an enhanced display image.

7. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and is characterized in that when the processor executes the computer program, the visual enhancement method of the LED creative spherical display screen described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the processor is enabled to execute the visual enhancement method for the LED creative spherical display screen according to any one of claims 1 to 5.

Images