Edge fusion method and system for XR immersive LED display screen

Abstract

The present invention relates to an edge fusion method and system for an XR immersive LED display screen. The method: extract features from the image data of the XR immersive LED display screen to obtain an RGB three-channel spatial domain feature matrix and a frequency domain edge feature matrix; perform feature processing to obtain a multi-core context feature map; perform encoding processing to obtain a position-brightness comprehensive encoding feature matrix; perform multi-scale decomposition to obtain a multi-scale detail feature map; perform pixel clustering on the edge area of ​​the XR immersive LED display screen to obtain an edge topology structure diagram and a brightness transition function; dynamically match the display frame rate through a PID controller, and perform brightness compensation and color correction on the edge area to obtain edge fusion data. The present invention has multi-scale feature extraction capabilities, can adaptively handle edge fusion problems in different scenarios, and ensure that the processing delay meets real-time display requirements, thereby improving the overall performance of the XR immersive display system.

Description

Technical Field

[0001] The present invention relates to the technical field of LED display screens, and in particular to an edge fusion method and system for an XR immersive LED display screen. Background Art

[0002] The seamless splicing and edge fusion of multiple LED displays is a key technical link to ensure the display effect. The traditional edge fusion method mainly relies on manual adjustment and simple brightness transition algorithm, which cannot meet the display requirements in complex scenes.

[0003] There are three main problems with existing edge fusion technology: first, it only extracts features from the spatial domain for edge detection and fusion, ignoring frequency domain information, resulting in unsatisfactory fusion effect when the display content is similar to the background brightness; secondly, it lacks an adaptive feature extraction and fusion mechanism, and cannot dynamically adjust the fusion parameters according to different scenarios; the real-time performance of traditional algorithms is poor, and it is difficult to meet the strict requirements for display delay in XR scenarios. Summary of the invention

[0004] The main purpose of the present invention is to provide an edge fusion method and system for an XR immersive LED display screen. The present invention has multi-scale feature extraction capabilities, can adaptively handle edge fusion problems in different scenarios, and ensure that the processing delay meets real-time display requirements, thereby improving the overall performance of the XR immersive display system.

[0005] To achieve the above object, the present invention provides an edge fusion method for an XR immersive LED display screen, comprising the following steps:

[0006] Perform spatial domain feature extraction and frequency domain feature extraction on the image data of the XR immersive LED display screen to obtain the RGB three-channel spatial domain feature matrix and frequency domain edge feature matrix;

[0007] Based on the spatial domain feature matrix and the frequency domain edge feature matrix, a multi-scale convolution kernel and a global context attention mechanism are used to perform feature processing to obtain a multi-core context feature map;

[0008] Inputting the multi-core context feature map into a position encoding module and a brightness encoding module for encoding processing to obtain a position-brightness comprehensive encoding feature matrix;

[0009] The position-brightness integrated coding feature matrix is ​​decomposed at multiple scales through a feature pyramid structure, and processed by depthwise separable convolution and channel attention to obtain a multi-scale detail feature map;

[0010] According to the brightness distribution and gradient information in the multi-scale detail feature map, an adaptive clustering algorithm is used to perform pixel clustering on the edge area of ​​the XR immersive LED display screen to obtain an edge topology structure map and a brightness transition function;

[0011] Based on the edge topology structure diagram and the brightness transition function, the display frame rate is dynamically matched through a PID controller, and brightness compensation and color correction are performed on the edge area to obtain edge fusion data.

[0012] The present invention also provides an edge fusion system for an XR immersive LED display screen, comprising:

[0013] A feature extraction module is used to perform spatial domain feature extraction and frequency domain feature extraction on the image data of the XR immersive LED display screen to obtain an RGB three-channel spatial domain feature matrix and a frequency domain edge feature matrix;

[0014] A feature processing module, used to perform feature processing based on the spatial domain feature matrix and the frequency domain edge feature matrix using a multi-scale convolution kernel and a global context attention mechanism to obtain a multi-core context feature map;

[0015] A coding processing module, used for inputting the multi-core context feature map into a position coding module and a brightness coding module for coding processing to obtain a position-brightness comprehensive coding feature matrix;

[0016] A multi-scale decomposition module, used to perform multi-scale decomposition on the position-brightness integrated coding feature matrix through a feature pyramid structure, and obtain a multi-scale detail feature map through depth-separable convolution and channel attention processing;

[0017] A pixel clustering module, for performing pixel clustering on the edge area of ​​the XR immersive LED display screen according to the brightness distribution and gradient information in the multi-scale detail feature map and using an adaptive clustering algorithm to obtain an edge topology structure map and a brightness transition function;

[0018] The dynamic matching module is used to dynamically match the display frame rate through a PID controller based on the edge topology structure diagram and the brightness transition function, and to perform brightness compensation and color correction on the edge area to obtain edge fusion data.

[0019] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0020] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0021] In summary, the technical solution provided by the present invention constructs a complete feature representation system by combining the feature extraction methods of the spatial domain and the frequency domain, effectively solving the edge detection problem of the traditional method in similar brightness scenes; adopts a multi-core context feature guidance module to realize the fusion of feature information of local details and global structures, and significantly enhances the feature expression ability of edge areas; introduces learnable position encoders and brightness encoders, and reduces the information loss in the display screen splicing process through a dynamic encoding mechanism; designs a multi-scale detail fusion module, which integrates local spatial position information and global structural semantic information, and accurately captures the display characteristics of different areas; constructs an edge topology structure through an adaptive clustering algorithm and a minimum spanning tree method, and realizes intelligent grouping and connection of edge areas; a dynamic frame rate matching mechanism based on a PID controller ensures that the processing delay is controlled within a preset threshold to meet the real-time display requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the steps of an edge fusion method of an XR immersive LED display screen in one embodiment of the present invention;

[0023] Figure 2 is a block diagram of the edge fusion system of an XR immersive LED display screen in one embodiment of the present invention;

[0024] Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0025] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0027] Reference Figure 1 , this embodiment provides an edge fusion method for an XR immersive LED display screen, comprising the following steps:

[0028] S1, performing spatial domain feature extraction and frequency domain feature extraction on the image data of the XR immersive LED display screen to obtain an RGB three-channel spatial domain feature matrix and a frequency domain edge feature matrix;

[0029] Among them, the image data of the XR immersive LED display screen is separated into three RGB channels to obtain separate R channel, G channel and B channel feature matrices. The feature matrix of each channel contains the brightness and color information of the channel in the spatial domain, which can respectively reflect the brightness and hue changes of different color channels in the image. The brightness and contrast of the RGB three-channel feature matrix are calculated to obtain two key spatial domain feature matrices, namely the spatial domain brightness feature matrix and the spatial domain contrast feature matrix. The spatial domain brightness feature matrix is ​​used to represent the brightness distribution of the image and can reflect the overall brightness and darkness of the image; while the spatial domain contrast feature matrix measures the brightness difference between image regions and can highlight the detail information and edge regions in the image. The spatial domain brightness feature matrix and the spatial domain contrast feature matrix are combined to obtain the RGB three-channel spatial domain feature matrix. The image data is fast Fourier transformed to convert the image data into the frequency domain to obtain the basic feature data in the frequency domain. The feature extraction in the frequency domain can reveal the distribution of frequency components in the image, especially the high-frequency information of the image. The basic feature data in the frequency domain are input into three convolutional layers and convolution operations are performed respectively. The first convolution layer uses a 3×3 convolution kernel, the second convolution layer uses a 5×5 convolution kernel, and the third convolution layer uses a 7×7 convolution kernel. Convolution kernels of different sizes can capture frequency domain features of different scales and extract edge and texture information of the image. Through this series of convolution operations, three groups of frequency domain convolution features are obtained, namely the first, second and third groups of frequency domain convolution features, which respectively reflect different levels and scales of frequency domain information. The three groups of frequency domain convolution features are subjected to feature dimensionality reduction and matrix transformation. Feature dimensionality reduction reduces redundant information, extracts the most representative features, and improves computational efficiency and processing accuracy. Matrix transformation makes the feature matrix more suitable for subsequent feature fusion and processing through appropriate transformations. Through this step, the frequency domain edge feature matrix is ​​obtained, which contains the important features of the image in the frequency domain, especially the frequency distribution of the edge area.

[0030] S2, based on the spatial domain feature matrix and the frequency domain edge feature matrix, uses multi-scale convolution kernels and global context attention mechanism to process features and obtain a multi-core context feature map;

[0031] Specifically, multi-scale convolution kernels are used to perform parallel convolution processing on the spatial domain feature matrix. The local detail features in the image are captured by convolution kernels of different scales, and the convolution kernels of different sizes process different levels of detail information respectively to obtain a multi-level local detail feature matrix group. The local detail feature matrix group is subjected to feature cascade operation to effectively fuse the features of each level to obtain a richer multi-scale feature representation matrix. This matrix comprehensively describes the multi-level feature information of the image in the spatial domain. At the same time, the frequency domain edge feature matrix is ​​processed by global average pooling and maximum pooling dual paths to capture the context information of the image from a global perspective. Global average pooling helps to extract the overall statistical information of the image, while maximum pooling emphasizes the key areas in the image by selecting the strongest response in the feature map. The combination of the two effectively captures the global context features in the image and obtains a dual-path global context feature vector. The dual-path global context feature vector is processed by the first fully connected layer and the ReLU activation function to obtain the intermediate layer feature vector. The introduction of the ReLU activation function helps to improve the nonlinear expression ability of the network, so that the model can better learn complex feature relationships. The feature vectors of the intermediate layer are processed by the second fully connected layer and the Sigmoid activation function to obtain the channel attention weight matrix. The Sigmoid activation function limits the output weights to the range of [0, 1] to ensure that the attention to each channel can be effectively controlled. The channel attention weight matrix can weight the features of different channels, thereby highlighting important features and suppressing irrelevant parts. Based on the obtained channel attention weight matrix, the multi-scale feature representation matrix is ​​weighted to obtain the attention enhancement feature matrix. The channel attention mechanism is used to strengthen the expression of important features in the image, so that the edge and detail information are further highlighted, thereby effectively improving the image fusion effect. In order to optimize the transmission of features, the attention enhancement feature matrix and the spatial domain feature matrix are added element by element by residual connection operation. The residual connection can retain the original feature information, while combining the new features processed by the attention mechanism, avoiding information loss and accelerating the convergence of the network. The fused feature matrix is ​​standardized by the batch normalization layer, and the distribution of features is optimized by the Softmax activation function to ensure that the values ​​of the feature map have good statistical properties and distribution characteristics, and the final multi-core context feature map is obtained. The batch normalization layer effectively solves the gradient vanishing problem in training and improves the stability and efficiency of training, while the Softmax activation function helps adjust the response strength of the feature map so that each part of the image can obtain a reasonable weight when fused.

[0032] S3, inputting the multi-core context feature map into the position encoding module and the brightness encoding module for encoding processing to obtain a position-brightness comprehensive encoding feature matrix;

[0033] It should be noted that the multi-core context feature map is coordinate mapped by the sine-cosine position encoding function in the position encoding module. The spatial position information of the image is converted into a coding format that can be used for deep learning model processing. The sine-cosine position encoding function can generate a set of codes with periodic characteristics, ensuring that the model can understand the relative relationship between different pixel positions in the image, thereby capturing spatial structure information. Through this mapping, a two-dimensional position encoding matrix is ​​obtained, which contains the spatial information of each pixel position. The two-dimensional position encoding matrix is ​​input into the multi-head self-attention mechanism for feature decomposition, and the relationship between different positions in the image is captured at the same time to obtain the position attention feature, so that the network can pay attention to the position relationship of different regions in the image globally. In order to improve the expression ability of the network, the position attention feature is nonlinearly transformed to obtain the position space feature vector. In the brightness encoding module, the brightness distribution of the multi-core context feature map is calculated to extract the brightness feature of the image. Brightness distribution is an important part of the image display effect, especially when edge fusion occurs, the brightness transition will directly affect the visual effect. By calculating the brightness distribution, the initial brightness feature obtained contains the brightness change information in the image. The initial brightness feature is input into the 3×3 depth-separable convolution layer for local feature processing. Depthwise separable convolution is an efficient convolution method that can process the spatial information and channel information of the convolution kernel separately and capture local brightness features more finely. Through this processing, the local brightness features obtained enhance the detailed performance of brightness information. The local brightness features are weighted by the channel attention mechanism. The channel attention mechanism assigns different weights to the features of different channels, so that important brightness information is strengthened and unimportant parts are suppressed. Through this process, the brightness attention feature is obtained, which can effectively emphasize the brightness information in the image that is important for the display effect. The brightness attention feature is multiplied by the position space feature vector. The position space feature and the brightness feature are combined to extract more expressive features through interaction. The feature fusion matrix obtained is normalized by Layer Normalization and residual connection. Layer Normalization can normalize the features during training, improve the stability of training, and reduce the problem of gradient disappearance. The residual connection can retain the original feature information and combine it with the newly generated features to avoid information loss and accelerate the convergence of the network, and finally obtain the position-brightness comprehensive encoding feature matrix.

[0034] S4, through the feature pyramid structure, the position-brightness comprehensive coding feature matrix is ​​decomposed at multiple scales, and processed by depth-separable convolution and channel attention to obtain a multi-scale detail feature map;

[0035] Specifically, the position-brightness integrated coding feature matrix is ​​subjected to maximum pooling operations of different step sizes, wherein the maximum pooling operations of step sizes of 2, 4, and 8 are respectively used to obtain a three-layer feature pyramid structure of 1 / 2, 1 / 4, and 1 / 8 scales. By gradually reducing the resolution of the features to extract information of different scales, the model can capture the global and local features in the image at different scales. The spatial dimension features of each layer of the three-layer feature pyramid structure are extracted to obtain the corresponding depth feature group. Each depth feature group is input into the point-by-point convolution layer for channel dimension feature fusion. Point-by-point convolution is a special convolution operation that uses a 1×1 convolution kernel to process the features, which can effectively realize the information interaction between channels and obtain the initial separation feature group. The 1 / 4 scale and 1 / 8 scale feature maps in the initial separation feature group are upsampled by bilinear interpolation to make them reach the same scale as the 1 / 2 scale feature map, and obtain a unified scale feature group. Bilinear interpolation is an upsampling method that restores low-resolution feature maps to higher resolutions by linear interpolation, thereby establishing a connection between features of different scales. By unifying all feature maps to 1 / 2 scale, it is convenient to unify and fuse the features in the future. The channel dimension modeling is performed on the unified scale feature group to obtain the channel attention weight vector. Through channel dimension modeling, the importance of features of different channels is evaluated to highlight those features that are more critical in visual effects. Based on the channel attention weight vector, the channel dimension weighted calculation is performed on the unified scale feature group to obtain the weight-enhanced feature group. By assigning different weights to each channel, the more important features in the context are enhanced, while the unimportant features are suppressed, so that the overall feature expression is more concentrated and effective. The weight-enhanced feature group is subjected to feature connection operation and feature dimensionality reduction to obtain a multi-scale fused feature map. The feature connection operation preserves the information of multi-scale features by splicing different feature groups; the feature dimensionality reduction reduces redundant features and retains the most representative features to reduce computational complexity and improve computational efficiency. The multi-scale fused feature map is upsampled to the original size so that it can be processed at the same scale as the position-brightness comprehensive coding feature matrix. The multi-scale detail feature map is obtained by residual connection of the multi-scale fused feature map with the position-brightness comprehensive coding feature matrix.

[0036] S5, based on the brightness distribution and gradient information in the multi-scale detail feature map, an adaptive clustering algorithm is used to cluster pixels in the edge area of ​​the XR immersive LED display screen to obtain an edge topology structure map and a brightness transition function;

[0037] Among them, the brightness information of the image is counted based on the multi-scale detail feature map. The distribution of each brightness value in the image is counted by the brightness histogram method to obtain the overall brightness distribution characteristics. The local area in the image is scanned step by step through the regional sliding window, and the variance of the brightness in each sliding window is calculated to extract the change information of the local brightness, and the edge area brightness feature vector is obtained, which contains the local difference of brightness in space and can effectively reflect the brightness change characteristics of the edge area of ​​the image. At the same time, the shape and direction characteristics of the edge are captured by gradient information. The gradient value of each pixel in the horizontal and vertical directions is calculated by using the horizontal and vertical Sobel operators to perform gradient operations on the multi-scale detail feature map. By calculating the amplitude and direction of the gradient, the edge gradient feature vector is obtained to describe the change of the edge in different directions. The introduction of gradient features enables edge detection to combine brightness changes and gradient changes, so as to locate the edge area more accurately. The edge area brightness feature vector and the edge gradient feature vector are feature concatenated to obtain the edge comprehensive feature matrix. The edge comprehensive feature matrix is ​​subjected to DBSCAN density clustering to obtain the initial clustering result. DBSCAN is a density-based clustering algorithm that can effectively process noise points and find clusters of arbitrary shapes. It is suitable for edge clustering in images. Through the DBSCAN algorithm, the pixels in the edge area are clustered according to feature similarity, so that the edge area is divided into multiple clusters with similar characteristics. The clusters preliminarily reflect the basic structure of the edge area of ​​the display screen. According to the initial clustering results, the center point coordinates, average brightness value and brightness standard deviation of each cluster are calculated to obtain a clustering statistical parameter group. These parameters are used to describe the position and brightness characteristics of each cluster, especially the mean and variance of the brightness reflect the uniformity and variation of the brightness within each cluster. Based on the clustering statistical parameter group, a weighted undirected graph is constructed, and the edge connection calculation is performed through the Kruskal minimum spanning tree algorithm to obtain an edge topology structure diagram. The Kruskal algorithm is a minimum spanning tree algorithm that can find the minimum spanning tree connecting all nodes in a weighted undirected graph, realize the effective connection of each cluster in the edge area, and ensure the coherence and integrity of the edge. The brightness mean of adjacent clusters in the edge topology diagram is calculated by nonlinear difference to achieve brightness transition between clusters. Nonlinear difference makes the brightness transition smoother and avoids sudden brightness changes between edge areas. At the same time, in order to optimize the brightness smoothing effect, the brightness difference is processed in combination with the Gaussian smoothing function, so that the brightness change in the transition area is more natural and continuous, and the parameters of the brightness transition function are obtained. These parameters describe the change law of the brightness of the edge area from one cluster to another. In order to ensure that the brightness transition function can be stable and consistent under different circumstances, the brightness transition function parameters are normalized to obtain the final brightness transition function.

[0038] S6, based on the edge topology diagram and the brightness transition function, dynamically matches the display frame rate through the PID controller, and performs brightness compensation and color correction on the edge area to obtain edge fusion data.

[0039] Specifically, the display frame rate of the edge area is sampled and analyzed to obtain the frame rate distribution information under the current display state. By combining with the brightness transition function, the ideal target frame rate is calculated, and the target frame rate is intended to ensure the smoothness of the display and the effect of edge fusion. By comparing the actual frame rate with the target frame rate, a frame rate difference sequence is obtained to describe the deviation between the current frame rate and the ideal frame rate. The frame rate difference sequence is input into a PID controller for operation, and the proportional coefficient of the PID controller is P, the integral coefficient is I, and the differential coefficient is D. The PID controller is a feedback control algorithm that realizes rapid and accurate adjustment of system deviations through the combined action of proportion, integration and differentiation. Through the operation of the PID controller, a frame rate correction control amount is obtained, which is used to adjust the display frame rate in real time to make it closer to the target frame rate, ensuring the smoothness of the display and the seamless fusion of the edge. Based on the edge topology structure diagram, multiple parallel processing channels are constructed, each channel corresponds to a clustered display point group, forming a multi-channel processing structure, realizing parallel processing of multiple display points in the edge area, thereby improving processing efficiency and accuracy. Based on the frame rate correction control amount, the control parameters of the processing speed of each channel are calculated to obtain the processing speed matrix, which is used to control the processing speed of each channel to achieve the effect of dynamic matching with the frame rate, thereby ensuring the synchronization between each channel and the consistency of the display effect. On the basis of controlling the frame rate and processing speed, the brightness difference compensation is performed on each display point in the edge area according to the brightness transition function. By calculating the difference between the brightness of the display point and the ideal brightness and compensating the difference, the transition between the brightness of each display point and the surrounding display points is ensured to be smooth. At the same time, local smoothing is performed in combination with the brightness gradient of the surrounding display points to obtain the brightness correction matrix to reduce the brightness mutation in the edge area, make the edge transition more natural, and enhance the overall visual effect. The display data of the edge area is processed by dual paths through the parallel brightness adjustment channel and color correction channel. In the brightness adjustment channel, the brightness correction matrix is ​​applied to adjust the brightness of the display point to ensure the consistency of the brightness; while in the color correction channel, adjustments are made to the problem of color inconsistency to eliminate the color difference that may exist in the edge area. Through the parallel processing of these two channels, the edge processing data is obtained. Based on the processing speed matrix, the edge processing data is dynamically matched and cached to obtain synchronous fusion data. The purpose of dynamic frame rate matching is to ensure that the frame rate can be adjusted in real time as the content changes during the display process, thereby avoiding visual freezes and flickers caused by inconsistent frame rates. Through cache processing, the data is properly buffered when the frame rate changes to balance the processing speed and display speed to ensure that the final display effect is smooth and stable. According to the connection relationship of the edge topology diagram, the synchronous fusion data is regionally reorganized, and the outputs of each parallel processing channel are recombined according to the topological relationship of the edge area to obtain complete edge fusion data.

[0040] In one example, spatial domain feature extraction and frequency domain feature extraction are performed on image data of an XR immersive LED display screen to obtain an RGB three-channel spatial domain feature matrix and a frequency domain edge feature matrix, including:

[0041] The image data of the XR immersive LED display screen is separated into three RGB channels to obtain an R channel feature matrix, a G channel feature matrix, and a B channel feature matrix;

[0042] Perform brightness calculation and contrast calculation on the R channel feature matrix, the G channel feature matrix and the B channel feature matrix to obtain a spatial domain brightness feature matrix and a spatial domain contrast feature matrix;

[0043] The spatial domain brightness feature matrix and the spatial domain contrast feature matrix are combined to obtain the RGB three-channel spatial domain feature matrix;

[0044] Perform fast Fourier transform on the image data to obtain frequency domain basic feature data, and perform convolution operation on the frequency domain basic feature data through the first convolution layer, the second convolution layer and the third convolution layer respectively to obtain the first group of frequency domain convolution features, the second group of frequency domain convolution features and the third group of frequency domain convolution features, wherein the convolution kernel size of the first convolution layer is 3×3, the convolution kernel size of the second convolution layer is 5×5, and the convolution kernel size of the third convolution layer is 7×7;

[0045] The first group of frequency domain convolution features, the second group of frequency domain convolution features and the third group of frequency domain convolution features are subjected to feature dimension reduction and matrix conversion to obtain a frequency domain edge feature matrix.

[0046] In this example, the image data of the XR immersive LED display is separated into three RGB channels to extract the information of the red, green and blue channels respectively. Assume that the input image data is a matrix ,in and Represents the position coordinates of the pixel, Represents the color channel, with values ​​of R, G, and B. Separate the image matrix according to the color channel to obtain the R channel feature matrix , G channel feature matrix and the B channel feature matrix . These three matrices represent the intensity distribution of red, green, and blue in the image. Perform brightness calculation and contrast calculation on the feature matrix of each color channel to obtain the brightness and contrast feature matrix of the spatial domain. Brightness is used to describe the overall brightness of the image. For an image with three channels of R, G, and B, the brightness characteristics of the spatial domain are obtained by calculating the brightness of each channel. Suppose the brightness calculation formula is:

[0047] ;

[0048] in, Represents pixels The coefficients 0.299, 0.587 and 0.114 are the contribution coefficients of the R, G and B channels to the brightness respectively. These coefficients are obtained based on the weighted sensitivity of the human eye to different colors. Through this formula, the spatial domain brightness feature matrix is ​​obtained , where each element represents the brightness value of the corresponding pixel position in the image. Calculate the contrast feature matrix. Contrast is used to measure the brightness difference between different areas in the image, reflecting the clarity and detail information of the image. The calculation of contrast involves the brightness change of the local area. Suppose the contrast calculation formula is:

[0049] ;

[0050] in, Represents a pixel The local window of is the average brightness of all pixels in the window, is the total number of pixels in the window. Through the above formula, we get the spatial domain contrast feature matrix C, which describes the contrast value of each pixel in the image and can reveal the intensity of image details. and the contrast feature matrix Perform feature combination to obtain the RGB three-channel spatial domain feature matrix that integrates brightness and contrast information Assuming that the feature combination adopts the element-by-element connection method, the feature combination formula is:

[0051] ;

[0052] in Represents the element-by-element concatenation operation of the matrix. Through this combination, a feature matrix containing brightness and contrast information is obtained. The image data is subjected to frequency domain feature extraction. The image data is subjected to fast Fourier transform to convert the image from the spatial domain to the frequency domain. Suppose the input image matrix is , then the frequency domain basic feature data after fast Fourier transform It is expressed as:

[0053] ;

[0054] in, is the coordinate in the frequency domain, and are the width and height of the image, Represents an imaginary unit. Through fast Fourier transform, the spatial information of the original image is converted into frequency components, which can describe the periodic changes in the image. In particular, the high-frequency part often corresponds to the edge and detail information. The convolution kernel size of the first convolution layer is , and its convolution operation is expressed as:

[0055] ;

[0056] in, The size is The convolution kernel of , * indicates the convolution operation, is the first set of frequency domain convolution features. Similarly, the second convolution layer uses a size of The third convolution layer uses a convolution kernel of size The convolution kernel of , respectively, obtains the second set of frequency domain convolution features And the third set of frequency domain convolution features Convolution kernels of different sizes capture different scales of features in the frequency domain. Smaller convolution kernels are more suitable for extracting detailed features, while larger convolution kernels capture more global frequency information. The three sets of frequency domain convolution features are subjected to feature dimensionality reduction and matrix transformation to reduce redundant information and extract the most representative frequency domain features. Let the feature dimensionality reduction operation be the function , then the frequency domain features after dimension reduction are expressed as:

[0057] ;

[0058] By reducing the dimension of features, the computational complexity is effectively reduced while retaining the main frequency domain information. Matrix transformation is performed on these frequency domain features after dimension reduction. Let the matrix transformation operation be , then the transformed feature matrices are:

[0059] ;

[0060] Through matrix conversion, the frequency domain features are adjusted to a suitable representation form so as to be fused with other features. All the converted frequency domain feature matrices are integrated to obtain the frequency domain edge feature matrix.

[0061] In one example, based on the spatial domain feature matrix and the frequency domain edge feature matrix, a multi-scale convolution kernel and a global context attention mechanism are used for feature processing to obtain a multi-core context feature map, including:

[0062] A multi-scale convolution kernel is used to perform parallel convolution processing on the spatial domain feature matrix to obtain a local detail feature matrix group, and a feature cascade operation is performed on the local detail feature matrix group to obtain a multi-scale feature representation matrix;

[0063] Perform global average pooling and maximum pooling dual-path processing on the frequency domain edge feature matrix to obtain a dual-path global context feature vector;

[0064] The dual-path global context feature vector is processed through the first fully connected layer and the ReLU activation function to obtain the intermediate layer feature vector;

[0065] The feature vector of the middle layer is processed through the second fully connected layer and the Sigmoid activation function to obtain the channel attention weight matrix;

[0066] The multi-scale feature representation matrix is ​​weighted based on the channel attention weight matrix to obtain the attention enhanced feature matrix;

[0067] The attention-enhanced feature matrix is ​​added to the spatial domain feature matrix by element-by-element residual connection to obtain a fused feature matrix, which is then normalized by a batch normalization layer and a Softmax activation function to obtain a multi-core context feature map.

[0068] In this example, multi-scale convolution kernels are used to perform parallel convolution processing on the spatial domain feature matrix to extract spatial features from different scales. Suppose the input spatial domain feature matrix is , by using convolution kernels of different sizes in parallel to perform convolution operations, feature information of different scales can be obtained. Assume that the convolution kernel sizes used are , and , then the corresponding convolution calculation is expressed as:

[0069] ;

[0070] in, They are , , The convolution kernel of , * indicates the convolution operation, are local detail feature matrices of different scales. Through convolution kernels of different sizes, different levels of image information can be extracted. Small-sized convolution kernels pay more attention to details, while large-sized convolution kernels pay more attention to the overall structure and texture. Perform feature concatenation operations, concatenate these matrices in the feature dimension, and obtain a representation matrix that integrates multi-scale features, denoted as The feature cascade is represented as:

[0071] ;

[0072] in The channel-by-channel concatenation operation of the representation matrix is ​​performed to obtain a feature representation containing multi-level information by cascading features of different scales. A dual-path process of global average pooling and maximum pooling is performed to extract global context information. Global average pooling and maximum pooling are expressed as:

[0073] ;

[0074] in, and are the width and height of the frequency domain feature matrix respectively, represents the global average pooling feature vector, Represents the global maximum pooling feature vector. Through these operations, the overall statistical information of the image and the most important feature information are extracted respectively, and a dual-path global context feature vector containing more global context is obtained. , The dual-path global context feature vector is input into the first fully connected layer and nonlinearly processed by the ReLU activation function to improve the expressiveness of the model. Assume that the weight matrix of the first fully connected layer is , then the intermediate layer feature vector is expressed as:

[0075] ;

[0076] in, is the middle layer feature vector, and are the weight and bias parameters of the first fully connected layer, Represents the rectified linear unit activation function, which is used to increase the nonlinearity of the model so as to learn more complex feature relationships. After the second fully connected layer, the intermediate layer feature vector is processed by the Sigmoid activation function to obtain the channel attention weight matrix. Assume that the weight matrix of the second fully connected layer is , then the channel attention weight matrix is ​​expressed as:

[0077] ;

[0078] in, is the channel attention weight matrix, and are the weight and bias parameters of the second fully connected layer, respectively. The Sigmoid activation function is used to limit the output value between [0,1], thereby assigning a weight to each channel. Based on the channel attention weight matrix, the multi-scale feature representation matrix Perform weighted processing to obtain the attention enhancement feature matrix. The formula for weighted processing is expressed as:

[0079] ;

[0080] in, is the attention-enhanced feature matrix, represents the channel index, Represents pixel coordinates. By weighting the features of each channel, important features are enhanced while irrelevant features are suppressed, so that the model pays more attention to the key information in the image. With the original spatial domain feature matrix Perform element-by-element residual connection operations to obtain the fused feature matrix:

[0081] ;

[0082] The role of residual connection is to retain the original feature information, while combining the features of attention enhancement to make the feature expression richer, which helps to maintain the integrity of information and accelerate the convergence of the model. Standardization is performed to ensure that the features of each channel have a consistent scale during the training process and improve the training stability of the model. The formula for batch normalization is:

[0083] ;

[0084] in, and Channel The mean and variance of A small constant to prevent the denominator from being zero, is the feature matrix after batch normalization. The normalized fusion feature matrix is ​​processed by the Softmax activation function to obtain the final multi-core context feature map. The function of the Softmax activation function is to convert each feature value into a probability distribution in order to better represent the importance of the feature. The formula of Softmax is:

[0085] ;

[0086] in, Represents a multi-core context feature map.

[0087] In one example, the multi-core context feature map is input into the position encoding module and the brightness encoding module for encoding processing to obtain a position-brightness comprehensive encoding feature matrix, including:

[0088] The multi-core context feature map is coordinate mapped through the sine-cosine position encoding function in the position encoding module to obtain a two-dimensional position encoding matrix;

[0089] The two-dimensional position encoding matrix is ​​decomposed into multi-head self-attention features to obtain the position attention features, and the position attention features are nonlinearly transformed to obtain the position space feature vector;

[0090] The multi-core context feature map is input into the brightness encoding module to calculate the brightness distribution to obtain the initial brightness feature, and the initial brightness feature is input into the 3×3 depth-separable convolution layer for local feature processing to obtain the local brightness feature;

[0091] The local brightness features are weighted by the channel attention mechanism to obtain the brightness attention features, and the brightness attention features are multiplied by the position space feature vector to obtain the feature fusion matrix.

[0092] The feature fusion matrix is ​​normalized through Layer Normalization and residual connection to obtain the position-brightness comprehensive coding feature matrix.

[0093] In this example, for the input multi-core context feature map ,in and Represent the height and width of the feature map respectively, Indicates the number of channels and encodes the position of the feature map. Position encoding is performed through sine and cosine functions to provide the model with the position information of each feature. For position encoding, the following sine and cosine functions are used for calculation:

[0094] ;

[0095] ;

[0096] in, and Respectively indicate the location Channel and Similarly, and Indicates location In this way, each position in the two-dimensional plane is encoded to generate a two-dimensional position encoding matrix , which provides unique encoding information for each position, enabling the model to distinguish features at different spatial positions. The two-dimensional position encoding matrix is ​​input into the multi-head self-attention mechanism for feature decomposition. The multi-head self-attention mechanism captures the relationship between different positions by dividing the input features into multiple heads, each of which independently calculates the attention weight and combines the features. Assume that the two-dimensional position encoding matrix Divided into heads, and the feature of each head is expressed as in For each head, the formula for calculating the attention feature is:

[0097] ;

[0098] in, , and are the linear transformation matrices of the query and value, is the dimension of the key. By calculating, we get the position attention feature , which contains the relationship information between each position and other positions. The attention features of all heads are concatenated and transformed linearly to obtain the final position attention features:

[0099] ;

[0100] in, is the output linear transformation matrix. The position attention feature is nonlinearly transformed and the ReLU activation function is used to enhance the nonlinear representation ability of the model to obtain the position space feature vector:

[0101] ;

[0102] At the same time, the multi-core context feature map Input to the brightness encoding module to calculate the brightness distribution and obtain the initial brightness feature The main purpose of the brightness encoding module is to capture the brightness variation characteristics of the image, thereby helping to achieve more accurate brightness adjustment during edge fusion. , and input it into a 3×3 depthwise separable convolution layer for local feature processing. Depthwise separable convolution is an efficient convolution method that performs spatial convolution (depthwise convolution) on each channel and then integrates information between channels through pointwise convolution. Suppose the convolution kernel of depthwise convolution is , the convolution kernel of point-by-point convolution is , then the local brightness feature is expressed as:

[0103] ;

[0104] Through this convolution method, local brightness features containing local brightness information are obtained , which can effectively capture the detailed changes in brightness. The channel attention mechanism is used to assign weights to local brightness features to obtain brightness attention features. The channel attention mechanism highlights key brightness features by calculating the importance weight of each channel. The local brightness features are globally pooled to obtain the statistical features of each channel. , and then calculate the attention weight of each channel through two fully connected layers and Sigmoid activation function:

[0105] ;

[0106] in, are the weight matrices of the two fully connected layers, , is the bias vector, is the channel attention weight vector. The brightness attention feature is obtained by multiplying the weight vector with the local brightness feature channel by channel:

[0107] ;

[0108] in, is the channel index, is the spatial position. Attention characteristics of brightness and the position space eigenvector Perform tensor multiplication to achieve feature fusion and obtain the feature fusion matrix :

[0109] ;

[0110] in, Indicates the element-by-element multiplication operation. In this way, the brightness feature and the position feature are combined so that the fused feature contains both the brightness change information and the contextual association of the spatial position. In order to improve the stability and expression ability of the feature, the feature fusion matrix Perform Layer Normalization and residual connection. The formula for Layer Normalization is:

[0111] ;

[0112] in, and Channel The mean and variance of is a small constant used to prevent division by zero. Layer Normalization makes the features in each channel have the same scale, thereby improving the training stability and generalization ability of the model. Add element by element to get the position-brightness comprehensive coding feature matrix:

[0113] ;

[0114] The introduction of residual connections can preserve both the original features and the fused features, which helps prevent the gradient vanishing problem and accelerates the convergence of the network.

[0115] In one example, the position-brightness integrated coding feature matrix is ​​decomposed at multiple scales through a feature pyramid structure, and processed by depthwise separable convolution and channel attention to obtain a multi-scale detail feature map, including:

[0116] The position-brightness integrated coding feature matrix is ​​subjected to maximum pooling operations with step sizes of 2, 4, and 8, respectively, to obtain a three-layer feature pyramid structure with scales of 1 / 2, 1 / 4, and 1 / 8;

[0117] The spatial dimension features of each layer of the three-layer feature pyramid structure are extracted to obtain a deep feature group, and the deep feature group is input into the point-by-point convolution layer for channel dimension feature fusion to obtain the initial separation feature group;

[0118] The feature maps of 1 / 4 and 1 / 8 scales in the initial separation feature group are bilinearly interpolated and upsampled to 1 / 2 scale to obtain a unified scale feature group, and the channel dimension of the unified scale feature group is modeled to obtain the channel attention weight vector;

[0119] Perform a weighted calculation of the channel dimension on the channel attention weight vector and the uniform scale feature group to obtain a weight-enhanced feature group, and perform feature connection operations and feature dimensionality reduction based on the weight-enhanced feature group to obtain a multi-scale fusion feature map;

[0120] The multi-scale fusion feature map is upsampled to the original size and residually connected with the position-brightness integrated coding feature matrix to obtain a multi-scale detail feature map.

[0121] In this example, the position-brightness integrated coding feature matrix is ​​decomposed at multiple scales to extract the feature information of the image at different scales. Suppose the position-brightness integrated coding feature matrix is ,in , are the height and width of the feature map, respectively. The matrix is ​​subjected to maximum pooling operations with different step sizes, that is, pooling operations with step sizes of 2, 4, and 8 are used to obtain feature pyramid structures of different scales, including and , the corresponding feature scales are 1 / 2, 1 / 4 and 1 / 8 of the original features. The formula for maximum pooling is:

[0122] ;

[0123] in 2, 4 and 8 respectively, Represents the position index of the feature map after pooling, is the channel index. Through multi-scale pooling operations, the feature pyramid structure is effectively constructed to capture contextual information of different scales, so that the model has a good understanding of both global and local features. The spatial dimension feature extraction is performed on each layer of the three-layer feature pyramid structure to obtain a deep feature group. The spatial dimension feature extraction is achieved by deep convolution. Suppose each layer of feature map The deep feature group obtained after the deep convolution operation is For depthwise convolution, the size of the convolution kernel is usually , to capture the local detail information in the feature map. Its calculation formula is:

[0124] ;

[0125] in, Denotes the deep convolution kernel, and * denotes the convolution operation. By performing deep convolution on the features of each scale, the spatial feature information at that scale is extracted. Input the point-by-point convolution layer respectively, perform channel dimension feature fusion, and obtain the initial separation feature group Point-wise convolution is a The convolution operation realizes feature fusion through cross-channel convolution. The formula of point-by-point convolution is expressed as:

[0126] ;

[0127] in, is the point-by-point convolution kernel, is the spatial position index, is the output channel index. Through point-by-point convolution, the features of each channel can be weighted and summed to achieve information fusion between channels and obtain a more compact feature representation. The 1 / 4 and 1 / 8 scale feature maps in the initial separated feature group are upsampled by bilinear interpolation and restored to 1 / 2 scale to obtain a unified scale feature group. Bilinear interpolation is an upsampling method that interpolates values ​​by using linear weighting of neighboring pixels. The formula is:

[0128] ;

[0129] in, is the interpolation weight, and Indicates the position rounding operation, is the ratio between the original scale and the target scale. By restoring all feature maps to a scale of 1 / 2, features of different scales can be aligned at the same scale. Perform channel dimension modeling to obtain the channel attention weight vector. Perform global average pooling on the uniform scale feature group to obtain the statistical features of each channel. , and then calculate the weight of each channel through two fully connected layers and ReLU activation function, and then normalize it through Sigmoid activation function. The formula is:

[0130] ;

[0131] in, are the weight matrices of the fully connected layers, , is the bias vector, is the channel attention weight vector. In this way, each channel is modeled to highlight important features and suppress irrelevant features. The channel attention weight vector and the uniform scale feature group are weighted in the channel dimension to obtain the weighted enhanced feature group :

[0132] ;

[0133] Through this weighted operation, the channel features that are important for the display effect are enhanced. Then, based on the weighted enhanced feature group, feature connection operations and feature dimensionality reduction are performed to obtain a multi-scale fusion feature map. The feature concatenation operation is to concatenate features from different sources in the channel dimension to retain information from different scales, while the feature dimensionality reduction is done through a The convolution of compresses the number of channels to reduce the amount of calculation. The formula is:

[0134] ;

[0135] in, is the feature dimension reduction operation, Represents the concatenation of features. Next, the multi-scale fused feature map is upsampled to the original size to be added to the original position-brightness integrated coding feature matrix. The upsampling is achieved through bilinear interpolation. After restoring the feature map to the original resolution, the fused features are combined with the original features using residual connections to obtain the final multi-scale detail feature map. :

[0136] ;

[0137] The residual connection ensures the retention of the original feature information, while fusing the processed multi-scale information to make the final feature expression richer, thereby providing a more natural brightness and color transition in the edge area of ​​the display.

[0138] In one example, based on the brightness distribution and gradient information in the multi-scale detail feature map, an adaptive clustering algorithm is used to cluster pixels in the edge area of ​​the XR immersive LED display to obtain an edge topology structure map and a brightness transition function, including:

[0139] Based on the multi-scale detail feature map, the brightness histogram is counted, and the local brightness variance is calculated through the regional sliding window to obtain the brightness feature vector of the edge area;

[0140] The multi-scale detail feature map is subjected to gradient operation through the horizontal and vertical Sobel operators, and the gradient amplitude and direction are calculated to obtain the edge gradient feature vector;

[0141] The edge area brightness feature vector and the edge gradient feature vector are concatenated to obtain an edge comprehensive feature matrix, and the edge comprehensive feature matrix is ​​clustered by DBSCAN density to obtain an initial clustering result;

[0142] According to the initial clustering results, the center point coordinates, average brightness value and brightness standard deviation of each cluster are calculated to obtain the clustering statistical parameter group, and a weighted undirected graph is constructed based on the clustering statistical parameter group. The edge connection is calculated by the Kruskal minimum spanning tree algorithm to obtain the edge topology structure diagram;

[0143] The nonlinear difference calculation is performed on the brightness mean of adjacent clusters in the edge topology diagram, and the transition processing is performed in combination with the Gaussian smoothing function to obtain the brightness transition function parameters. At the same time, the brightness transition function parameters are normalized to obtain the brightness transition function.

[0144] In this example, the brightness histogram statistics and local brightness variance calculation are performed based on the multi-scale detail feature map to extract the brightness characteristics of the edge area. Suppose the multi-scale detail feature map is ,in and Represent the height and width of the feature map respectively, is the number of channels. The brightness information is counted by extracting the brightness information in the feature map and constructing a brightness histogram. Let the brightness histogram be ,in Indicates the brightness level, Indicates the number of pixels at this brightness level. By counting the pixel distribution at each brightness level, the global information of the image brightness is obtained. In order to analyze the local brightness features, a regional sliding window is applied to the feature map to calculate the local brightness variance. Assume that the sliding window size is , then the local brightness variance of each position in the window is expressed as:

[0145] ;

[0146] in, Represents pixels The local brightness variance at Represents the brightness value within the window, is the average brightness in the window. By calculating the local variance, the brightness change of each pixel is obtained, so as to extract the brightness characteristics of the edge area with significant changes. These local brightness variances are used as features to form the edge area brightness feature vector to describe the brightness change of the image. The gradient information of the edge is extracted by applying the Sobel operator. The Sobel operator is an edge detection operator that performs convolution operations on the image in the horizontal and vertical directions to calculate the gradient value. Assume that the horizontal Sobel operator and the vertical Sobel operator are respectively and , then the gradient of the multi-scale detail feature map is expressed as:

[0147] ;

[0148] Among them, * represents the convolution operation, and Pixels By calculating the gradient magnitude and direction, the gradient magnitude and direction It is expressed as:

[0149] ;

[0150] in, Represents pixels The gradient strength at Indicates the gradient direction. Through gradient operation, the edge gradient feature vector is extracted to describe the strength and direction of the edge in the image. The edge area brightness feature vector and the edge gradient feature vector are concatenated to obtain a more comprehensive feature representation, namely the edge comprehensive feature matrix The formula for feature concatenation is:

[0151] ;

[0152] in, Represents the brightness feature vector and the gradient eigenvector Splice on the feature dimension. Apply the DBSCAN density clustering algorithm to the edge comprehensive feature matrix to obtain the initial clustering result. DBSCAN is a density-based clustering method that can identify clusters of any shape by analyzing the density of points in space and can handle noise points. is the neighborhood radius, MinPts is the minimum number of samples for each cluster, and the clustering condition of DBSCAN is expressed as: if the neighborhood of a point contains at least MinPts points, then the point is a core point, and then expands to form a cluster. After obtaining the initial clustering results, calculate the statistical information of each cluster, including the coordinates of the center point, the average brightness value, and the brightness standard deviation. Suppose the pixel set contained in a cluster is , then the center point coordinates of the cluster , average brightness value and brightness standard deviation It is expressed as:

[0153] ;

[0154] ;

[0155] in, Represents clusters The number of pixels in the image. Through these statistical information, a clustering statistical parameter group is obtained to describe the spatial position and brightness characteristics of each cluster. Based on the clustering statistical parameter group, a weighted undirected graph is constructed. Each node in the graph represents a cluster, and the edge weights between nodes represent the distance between clusters. Suppose two clusters and The center point coordinates are and , then the edge weight between the two It is expressed as:

[0156] ;

[0157] Through these edge weights, a weighted undirected graph is constructed. Then the Kruskal minimum spanning tree algorithm is used to process the weighted undirected graph to obtain the edge topology graph. The Kruskal algorithm is a greedy algorithm that aims to find a minimum spanning tree so that the total edge weight connecting all nodes is minimized, thereby achieving the optimal connection between clusters. After obtaining the edge topology graph, the brightness mean of adjacent clusters is calculated nonlinearly to achieve a smooth transition of brightness. Assume that the brightness mean of two adjacent clusters is and , then the nonlinear difference of brightness is expressed as:

[0158] ;

[0159] This formula uses cubic Bezier interpolation to achieve a smooth transition of brightness and avoid abrupt brightness changes. At the same time, in order to optimize the smoothing effect of brightness, the brightness difference is processed in combination with the Gaussian smoothing function. The formula of the Gaussian smoothing function is:

[0160] ;

[0161] Through Gaussian smoothing, the brightness change is made more natural, and the brightness transition function parameters are obtained. In order to ensure the consistency and stability of the brightness transition function in different situations, the brightness transition function parameters are normalized. The normalized brightness transition function is set to , then the normalized formula is:

[0162] ;

[0163] in, and are the minimum and maximum values ​​of the brightness transition function, respectively. Through this normalization process, the value of the brightness transition function is ensured to be within a certain range, thereby improving the visual consistency and quality of the display screen during edge fusion.

[0164] In one example, based on the edge topology diagram and the brightness transition function, the display frame rate is dynamically matched through a PID controller, and brightness compensation and color correction are performed on the edge area to obtain edge fusion data, including:

[0165] Sampling and analyzing the display frame rate of the edge area, and calculating the target frame rate according to the brightness transition function to obtain a frame rate difference sequence;

[0166] The frame rate difference sequence is input into a PID controller with a proportional coefficient of P, an integral coefficient of I, and a differential coefficient of D for operation to obtain a frame rate correction control amount;

[0167] Constructing parallel processing channels according to the edge topology structure diagram, each channel corresponds to a clustered display point group, obtaining a multi-channel processing structure, and calculating control parameters of the processing speed of each channel in the multi-channel processing structure based on the frame rate correction control amount, obtaining a processing speed matrix;

[0168] The brightness difference of each display point in the edge area is compensated according to the brightness transition function, and local smoothing is performed in combination with the brightness gradient of the surrounding display points to obtain a brightness correction matrix;

[0169] The display data of the edge area is processed in dual paths through a parallel brightness adjustment channel and a color correction channel to obtain edge processing data;

[0170] The edge processing data is dynamically matched with the frame rate and cached according to the processing speed matrix to obtain synchronous fusion data, and the synchronous fusion data is regionally reorganized according to the connection relationship of the edge topology structure diagram to obtain edge fusion data.

[0171] In this example, the display frame rate of the edge area is sampled and analyzed to ensure that the edge fusion effect of the display screen is stable under different brightness and color conditions. Suppose the sequence obtained by display frame rate sampling is ,in Represents a time series. By measuring the change trend of the display frame rate, the stability and response speed of the current display state can be determined. According to the brightness transition function , calculate the ideal target frame rate , the target frame rate is designed to ensure smooth transition of brightness and coherence of edge fusion during display. Frame rate difference sequence Calculated by the difference between the actual frame rate and the target frame rate, the formula is:

[0172] ;

[0173] The frame rate difference sequence describes the deviation between the current display frame rate and the ideal frame rate. In order to effectively adjust the frame rate difference, the frame rate difference sequence The input is calculated into a PID controller, which consists of three parts: proportional (P), integral (I) and differential (D), which effectively controls the response speed and stability of the system. The control formula of the PID controller is:

[0174] ;

[0175] in, Indicates the frame rate error, , and are the proportional, integral and differential coefficients respectively, is the frame rate correction control quantity. The proportional part is responsible for the response of the current error, the integral part is responsible for the cumulative compensation of the historical error, and the differential part smoothes the control quantity by predicting the change trend of the error. Through the adjustment of the PID controller, a control signal for correcting the frame rate is obtained, so that the display frame rate is closer to the target frame rate, improving the stability and visual effect of the display. After obtaining the frame rate correction control quantity, a parallel processing channel is constructed according to the edge topology diagram, and each channel corresponds to a cluster display point group. Assume that the number of clusters in the edge topology diagram is , then the number of parallel processing channels is , each channel is responsible for a display point group within a cluster. Through multi-channel processing, a multi-channel processing structure is constructed. The advantage of this structure is that it can realize parallel processing of different edge areas, thereby improving the display efficiency and fusion effect. Based on the frame rate correction control amount , calculate the control parameters of the processing speed of each channel in the multi-channel processing structure to obtain the processing speed matrix . Assume that the initial processing speed of each channel is , after introducing the frame rate correction, the processing speed is expressed as:

[0176] ;

[0177] in, is the corrected processing speed, is the proportional gain factor, which is used to adjust the effect of the control amount on the processing speed. In this way, the speed coordination between different channels is achieved, making the display of the entire edge area more consistent, avoiding visual jumps or tearing caused by inconsistent processing speeds in different areas. According to the brightness transition function, the brightness difference of each display point in the edge area is compensated. For each display point, its brightness value is corrected according to the brightness transition function to ensure a natural brightness transition with the adjacent area. Suppose the display point The initial brightness is , then the brightness after brightness compensation is expressed as:

[0178] ;

[0179] in, is the brightness difference, calculated according to the brightness transition function. Combined with the brightness gradient of the surrounding display points, the brightness correction matrix is ​​obtained by local smoothing to reduce the abruptness of the brightness change. . Local smoothing is achieved using the weighted average method, the formula is:

[0180] ;

[0181] in, Represents pixels Neighborhood of To smooth the weights, a local weighted average is used to ensure a smoother brightness transition in the edge area. After brightness compensation and local smoothing, the display data in the edge area is processed in two paths through a parallel brightness adjustment channel and a color correction channel to obtain edge processing data. The brightness adjustment channel makes fine adjustments to the brightness of the display point, while the color correction channel eliminates color deviation by adjusting the ratio of the RGB channels. Suppose the RGB value of a display point is , the RGB value after brightness adjustment is expressed as:

[0182] ;

[0183] in, is the brightness adjustment factor, through the brightness correction matrix At the same time, the color correction channel is determined by introducing the color adjustment coefficient To make a correction:

[0184] ;

[0185] Through the processing of these two channels, the brightness and color of the edge area are ensured to meet the visual requirements, and the edge processing data with brightness and color adjustment is obtained. According to the processing speed matrix, the edge processing data is dynamically frame-rate matched and cached to obtain synchronous fusion data. Dynamic frame rate matching adjusts the frame rate of each channel so that the display content is synchronized between different areas to avoid inconsistency. Suppose the frame rate matching factor of each channel is , then the frame rate after matching is:

[0186] ;

[0187] The purpose of cache processing is to delay the display data appropriately during the frame rate matching process to balance the display speed between different channels. Ensure that the final display effect is smooth and stable. According to the connection relationship of the edge topology diagram, the synchronous fusion data is regionally reorganized to obtain the final edge fusion data. According to the connection relationship between the clusters in the edge topology diagram, the display data of different channels are recombined to form a complete edge area display image. Through reorganization, the brightness and color transition between edge areas are ensured to be coherent, and edge fusion is finally achieved.

[0188] Reference Figure 2 , this embodiment provides an edge fusion system for an XR immersive LED display screen, including:

[0189] Feature extraction module 1, used to perform spatial domain feature extraction and frequency domain feature extraction on the image data of the XR immersive LED display screen, and obtain an RGB three-channel spatial domain feature matrix and a frequency domain edge feature matrix;

[0190] Feature processing module 2, used to perform feature processing based on the spatial domain feature matrix and the frequency domain edge feature matrix using a multi-scale convolution kernel and a global context attention mechanism to obtain a multi-core context feature map;

[0191] Coding processing module 3, used for inputting the multi-core context feature map into the position coding module and the brightness coding module for coding processing to obtain a position-brightness comprehensive coding feature matrix;

[0192] The multi-scale decomposition module 4 is used to perform multi-scale decomposition on the position-brightness integrated coding feature matrix through a feature pyramid structure, and obtain a multi-scale detail feature map through depth-separable convolution and channel attention processing;

[0193] The pixel clustering module 5 is used to cluster pixels in the edge area of ​​the XR immersive LED display screen according to the brightness distribution and gradient information in the multi-scale detail feature map and adopt an adaptive clustering algorithm to obtain an edge topology structure map and a brightness transition function;

[0194] The dynamic matching module 6 is used to dynamically match the display frame rate through the PID controller based on the edge topology structure diagram and the brightness transition function, and to perform brightness compensation and color correction on the edge area to obtain edge fusion data.

[0195] In this embodiment, for the specific implementation of each unit in the above system embodiment, please refer to the above method embodiment, which will not be repeated here.

[0196] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0197] Those skilled in the art will understand that Figure 3The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0198] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0199] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0200] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0201] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An edge fusion method for an XR immersive LED display, characterized in that: The following steps are involved: The image data of the XR immersive LED display screen is subjected to spatial domain feature extraction and frequency domain feature extraction to obtain an RGB three-channel spatial domain feature matrix and a frequency domain edge feature matrix; specifically comprising: performing RGB three-channel separation on the image data of the XR immersive LED display screen to obtain an R channel feature matrix, a G channel feature matrix and a B channel feature matrix; performing brightness calculation and contrast calculation on the R channel feature matrix, the G channel feature matrix and the B channel feature matrix to obtain a spatial domain brightness feature matrix and a spatial domain contrast feature matrix; performing feature combination on the spatial domain brightness feature matrix and the spatial domain contrast feature matrix to obtain an RGB three-channel feature matrix. performing fast Fourier transform on the image data to obtain frequency domain basic feature data, and performing convolution operation on the frequency domain basic feature data through a first convolution layer, a second convolution layer, and a third convolution layer, respectively, to obtain a first group of frequency domain convolution features, a second group of frequency domain convolution features, and a third group of frequency domain convolution features, wherein the convolution kernel size of the first convolution layer is 3×3, the convolution kernel size of the second convolution layer is 5×5, and the convolution kernel size of the third convolution layer is 7×7; performing feature dimension reduction and matrix conversion on the first group of frequency domain convolution features, the second group of frequency domain convolution features, and the third group of frequency domain convolution features to obtain a frequency domain edge feature matrix; Based on the spatial domain feature matrix and the frequency domain edge feature matrix, a multi-scale convolution kernel and a global context attention mechanism are used to perform feature processing to obtain a multi-core context feature map; The multi-core context feature map is input into the position coding module and the brightness coding module for coding processing to obtain a position-brightness comprehensive coding feature matrix; specifically including: coordinate mapping the multi-core context feature map through the sine-cosine position coding function in the position coding module to obtain a two-dimensional position coding matrix; multi-head self-attention feature decomposition of the two-dimensional position coding matrix to obtain position attention features, and nonlinear transformation of the position attention features to obtain position space feature vectors; the multi-core context feature map is input into the brightness coding module to calculate the brightness distribution to obtain initial brightness features, and the initial brightness features are input into a 3×3 depth-separable convolutional layer for local feature processing to obtain local brightness features; weights are assigned to the local brightness features through the channel attention mechanism to obtain brightness attention features, and tensor multiplication is performed on the brightness attention features and the position space feature vectors to obtain a feature fusion matrix; feature normalization processing is performed on the feature fusion matrix through Layer Normalization and residual connection to obtain a position-brightness comprehensive coding feature matrix; The position-brightness integrated coding feature matrix is ​​decomposed at multiple scales through a feature pyramid structure, and processed by depthwise separable convolution and channel attention to obtain a multi-scale detail feature map; According to the brightness distribution and gradient information in the multi-scale detail feature map, an adaptive clustering algorithm is used to perform pixel clustering on the edge area of ​​the XR immersive LED display screen to obtain an edge topology structure map and a brightness transition function; Based on the edge topology structure diagram and the brightness transition function, the display frame rate is dynamically matched through a PID controller, and brightness compensation and color correction are performed on the edge area to obtain edge fusion data.

2. The edge fusion method of the XR immersive LED display according to claim 1, characterized in that: The method of performing feature processing based on the spatial domain feature matrix and the frequency domain edge feature matrix using a multi-scale convolution kernel and a global context attention mechanism to obtain a multi-core context feature map includes: Using a multi-scale convolution kernel to perform parallel convolution processing on the spatial domain feature matrix to obtain a local detail feature matrix group, and performing a feature cascade operation on the local detail feature matrix group to obtain a multi-scale feature representation matrix; Performing global average pooling and maximum pooling dual-path processing on the frequency domain edge feature matrix to obtain a dual-path global context feature vector; Processing the dual-path global context feature vector through a first fully connected layer and a ReLU activation function to obtain an intermediate layer feature vector; The intermediate layer feature vector is processed by a second fully connected layer and a Sigmoid activation function to obtain a channel attention weight matrix; Performing weighted processing on the multi-scale feature representation matrix based on the channel attention weight matrix to obtain an attention enhanced feature matrix; The attention enhancement feature matrix and the spatial domain feature matrix are subjected to a residual connection operation of element-by-element addition to obtain a fused feature matrix, and the fused feature matrix is ​​standardized through a batch normalization layer and a Softmax activation function to obtain a multi-core context feature map.

3. The edge fusion method of the XR immersive LED display according to claim 1, characterized in that: The position-brightness integrated coding feature matrix is ​​decomposed at multiple scales by a feature pyramid structure, and processed by depth-separable convolution and channel attention to obtain a multi-scale detail feature map, including: Performing maximum pooling operations with step sizes of 2, 4, and 8 on the position-brightness comprehensive coding feature matrix respectively to obtain a three-layer feature pyramid structure with scales of 1 / 2, 1 / 4, and 1 / 8; Extracting spatial dimension features from each layer of features in the three-layer feature pyramid structure to obtain a deep feature group, and inputting the deep feature group into a point-by-point convolution layer to perform channel dimension feature fusion to obtain an initial separation feature group; Perform bilinear interpolation upsampling on the 1 / 4 and 1 / 8 scale feature maps in the initial separation feature group to 1 / 2 scale to obtain a unified scale feature group, and perform channel dimension modeling on the unified scale feature group to obtain a channel attention weight vector; Performing a weighted calculation of the channel dimension on the channel attention weight vector and the uniform-scale feature group to obtain a weight-enhanced feature group, and performing a feature connection operation and feature dimensionality reduction based on the weight-enhanced feature group to obtain a multi-scale fusion feature map; The multi-scale fusion feature map is upsampled to the original size, and is residually connected with the position-brightness integrated coding feature matrix to obtain a multi-scale detail feature map.

4. The edge fusion method of the XR immersive LED display according to claim 3, characterized in that: According to the brightness distribution and gradient information in the multi-scale detail feature map, an adaptive clustering algorithm is used to perform pixel clustering on the edge area of ​​the XR immersive LED display screen to obtain an edge topology structure map and a brightness transition function, including: Performing brightness histogram statistics based on the multi-scale detail feature map, and calculating local brightness variance through a regional sliding window to obtain a brightness feature vector of an edge region; Performing gradient operations on the multi-scale detail feature map by using horizontal and vertical Sobel operators, and calculating the gradient amplitude and direction to obtain an edge gradient feature vector; Performing feature concatenation on the edge region brightness feature vector and the edge gradient feature vector to obtain an edge comprehensive feature matrix, and performing DBSCAN density clustering on the edge comprehensive feature matrix to obtain an initial clustering result; The center point coordinates, average brightness value and brightness standard deviation of each cluster are calculated according to the initial clustering result to obtain a clustering statistical parameter group, and a weighted undirected graph is constructed based on the clustering statistical parameter group, and edge connection calculation is performed through the Kruskal minimum spanning tree algorithm to obtain an edge topology structure diagram; The brightness mean values ​​of adjacent clusters in the edge topology structure diagram are subjected to nonlinear difference calculation, and transition processing is performed in combination with a Gaussian smoothing function to obtain brightness transition function parameters. Meanwhile, the brightness transition function parameters are normalized to obtain a brightness transition function.

5. The edge fusion method of the XR immersive LED display according to claim 4, characterized in that: The method of dynamically matching the display frame rate based on the edge topology structure diagram and the brightness transition function through a PID controller, and performing brightness compensation and color correction on the edge area to obtain edge fusion data includes: Sampling and analyzing the display frame rate of the edge area, and calculating the target frame rate according to the brightness transition function to obtain a frame rate difference sequence; The frame rate difference sequence is input into a PID controller with a proportional coefficient of P, an integral coefficient of I, and a differential coefficient of D for operation to obtain a frame rate correction control amount; Constructing parallel processing channels according to the edge topology structure diagram, each channel corresponds to a clustered display point group, obtaining a multi-channel processing structure, and calculating control parameters for the processing speed of each channel in the multi-channel processing structure based on the frame rate correction control amount, obtaining a processing speed matrix; Performing brightness difference compensation on each display point in the edge area according to the brightness transition function, and performing local smoothing processing in combination with the brightness gradient of surrounding display points to obtain a brightness correction matrix; Performing dual-path processing on the display data of the edge area through a parallel brightness adjustment channel and a color correction channel to obtain edge processing data; Dynamic frame rate matching and cache processing are performed on the edge processing data according to the processing speed matrix to obtain synchronous fusion data, and regional reorganization is performed on the synchronous fusion data according to the connection relationship of the edge topology structure diagram to obtain edge fusion data.

6. An edge fusion system for an XR immersive LED display, characterized in that: For implementing the steps of the method according to any one of claims 1 to 5, the system comprises: A feature extraction module is used to perform spatial domain feature extraction and frequency domain feature extraction on the image data of the XR immersive LED display screen to obtain an RGB three-channel spatial domain feature matrix and a frequency domain edge feature matrix; A feature processing module, used to perform feature processing based on the spatial domain feature matrix and the frequency domain edge feature matrix using a multi-scale convolution kernel and a global context attention mechanism to obtain a multi-core context feature map; A coding processing module, used for inputting the multi-core context feature map into a position coding module and a brightness coding module for coding processing to obtain a position-brightness comprehensive coding feature matrix; A multi-scale decomposition module, used to perform multi-scale decomposition on the position-brightness integrated coding feature matrix through a feature pyramid structure, and obtain a multi-scale detail feature map through depth-separable convolution and channel attention processing; A pixel clustering module, for performing pixel clustering on the edge area of ​​the XR immersive LED display screen according to the brightness distribution and gradient information in the multi-scale detail feature map and using an adaptive clustering algorithm to obtain an edge topology structure map and a brightness transition function; The dynamic matching module is used to dynamically match the display frame rate through a PID controller based on the edge topology structure diagram and the brightness transition function, and to perform brightness compensation and color correction on the edge area to obtain edge fusion data.

7. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images