Image display method, system, device and medium based on multi-layer transparent display screen

Abstract

This invention relates to the field of image processing technology, specifically providing an image display method, system, device, and medium based on a multi-layer transparent display screen. The method includes: obtaining the number of display screen layers of the display device, and setting the number of image segmentation layers based on the number of display screen layers and image content, wherein the number of segmentation layers does not exceed the number of display screen layers; assigning one or more labels to each pixel of the image using a deep learning model, wherein the labels indicate the corresponding display screen layer index, and the labels and display screen layer indices have a mapping relationship; grouping the image into multiple sub-images according to the labels, each sub-image corresponding to a display screen layer index; performing transparent background completion processing on the sub-images to optimize the display effect; and outputting the processed sub-images to the corresponding display screen layers according to the display screen layer indexes corresponding to the sub-images. This invention achieves pixel-level intelligent layering through a deep learning model, significantly improving the quality of multi-layer transparent displays.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to an image display method, system, device and medium based on a multi-layer transparent display screen. Background Technology

[0002] Traditional transparent displays, when displayed in a single layer, often obscure background objects with their image content, affecting the perceived transparency. To improve transparency and image quality, multi-layer transparent display technology has emerged. Existing technologies typically segment image content and distribute it across different display layers for overlay. However, these methods have significant limitations: First, image segmentation strategies are often simplistic (e.g., fixed-position segmentation), failing to intelligently and adaptively layer the image content based on its complex semantic information (e.g., object depth, edges, and transparency). This results in unreasonable content allocation between layers, impacting the final fusion effect and transparency. Second, improper background processing of segmented sub-images can easily produce visual artifacts or interference during transparent display, affecting background clarity. Current technologies lack an efficient, intelligent method that optimizes transparent background display, making it difficult to balance high-quality image presentation with excellent background transparency. Summary of the Invention

[0003] In view of the above-mentioned shortcomings of the prior art, the present invention provides an image display method, system, device and medium based on a multi-layer transparent display screen to solve the above-mentioned technical problems.

[0004] In a first aspect, the present invention provides an image display method based on a multi-layer transparent display screen, comprising:

[0005] The number of display screen layers of the display device is obtained, and the number of image segmentation layers is set according to the number of display screen layers and the image content, wherein the number of segmentation layers does not exceed the number of display screen layers;

[0006] A deep learning model is used to assign one or more labels to each pixel of an image. The labels indicate the corresponding display layer index, and there is a mapping relationship between the labels and the display layer index.

[0007] The image is grouped into multiple sub-images according to the label, and each sub-image corresponds to a display layer index;

[0008] Transparent background padding is applied to the sub-images to optimize the display effect;

[0009] Based on the display layer index corresponding to the sub-image, the processed sub-image is output to the corresponding display layer.

[0010] In an optional implementation, obtaining the number of display screen layers of the display device and setting the number of image segmentation layers based on the number of display screen layers and the image content includes:

[0011] Set the display device's screen layer count to the maximum available layer count;

[0012] Extract the optical properties of the image, including transparency, depth of field, and contrast;

[0013] The actual number of segmentation layers is determined based on the optical properties and preset rules, wherein the actual number of segmentation layers does not exceed the maximum number of usable layers.

[0014] In an optional implementation, the actual number of segmentation layers is determined based on the optical properties and preset rules, wherein the actual number of segmentation layers does not exceed the maximum number of available layers, including:

[0015] Pre-set the threshold range of optical properties and set the corresponding number of segmentation layers for each threshold range;

[0016] The theoretical number of segmentation layers for the image is determined based on the threshold range to which the optical properties of the image belong and the number of segmentation layers corresponding to that threshold range.

[0017] If the theoretical number of segmentation layers exceeds the maximum number of usable layers, then the maximum number of usable layers is output as the actual number of segmentation layers of the image;

[0018] If the theoretical number of segmentation layers does not exceed the maximum number of available layers, then the theoretical number of segmentation layers is output as the actual number of segmentation layers of the image.

[0019] In an optional implementation, the deep learning model employs a fully convolutional network as its basic framework, including:

[0020] An encoder is used to extract multi-level image features through convolutional layers, where deep features capture semantic information and shallow features preserve spatial details.

[0021] The decoder is used to progressively upsample feature maps to the original image size through transposed convolutional layers and fuse shallow features of the corresponding resolution in the encoder through skip connections.

[0022] The output layer uses 1x1 convolutions to adjust the number of channels in the feature map output by the decoder to the number of display layers C, generating a probability tensor of size [H, W, C], where each channel corresponds to the predicted probability of a display layer index; the activation function of the output layer is Softmax, which makes the sum of the probabilities of each pixel in all channels equal to 1;

[0023] The Conditional Random Field (CRF) optimization module operates on the probability tensor of the output layer, refining the segmentation boundary by minimizing an energy function; the energy function includes:

[0024] Univariate potential energy: provided by the output layer probability tensor;

[0025] Binary potential energy: defined by the Gaussian kernel, including the spatial distance kernel and the color similarity kernel.

[0026] In an optional implementation, a deep learning model is used to assign one or more labels to each pixel of the image, including:

[0027] The image, the number of segmentation layers, and the number of display screen layers are input into the deep learning model, and the deep learning model outputs the probability of a pixel in each channel;

[0028] A threshold is set, and channels with a probability greater than the threshold are considered valid layer indices;

[0029] The corresponding label is assigned to the pixel based on the effective layer index.

[0030] In an optional implementation, it further includes:

[0031] If the number of pixels corresponding to any label is less than the set pixel number threshold, then the pixels corresponding to that label will be moved to a label in an adjacent layer.

[0032] In an optional implementation, the sub-image is subjected to transparent background completion processing, including:

[0033] Identify the areas in the sub-image that need repair, and set the pixel values ​​within those areas to pre-specified specific values;

[0034] Calculate the similarity between the out-of-domain pixels adjacent to the edge of the region and the edge of the region, and use the out-of-domain pixels with high similarity as diffusion sources;

[0035] Based on similarity, information about the diffusion source is gradually transferred to the area that needs to be repaired to fill the missing parts of the area;

[0036] The edges of the filled region are updated, and diffusion sources are re-identified based on the updated edges to repair the region until the region is completely filled.

[0037] Smooth the area after it has been filled.

[0038] In a second aspect, the present invention provides an image display system based on a multi-layer transparent display screen, comprising:

[0039] The layer number determination module is used to obtain the number of display screen layers of the display device, and set the number of image segmentation layers according to the number of display screen layers and the image content, wherein the number of segmentation layers does not exceed the number of display screen layers;

[0040] The pixel prediction module is used to assign one or more labels to each pixel of an image using a deep learning model. The labels indicate the corresponding display layer index, and the labels and display layer indexes have a mapping relationship.

[0041] The image grouping module is used to group an image into multiple sub-images according to the label, and each sub-image corresponds to a display layer index;

[0042] The image completion module is used to perform transparent background completion processing on sub-images to optimize the display effect;

[0043] The image output module is used to output the processed sub-image to the corresponding display layer according to the display layer index of the sub-image.

[0044] Thirdly, a device is provided, comprising:

[0045] Memory, used to store image display programs based on multi-layer transparent displays;

[0046] A processor is configured to implement the steps of the image display method based on a multi-layer transparent display screen as provided in the first aspect when executing the image display program based on the multi-layer transparent display screen.

[0047] Fourthly, a computer-readable medium is provided, on which an image display program based on a multi-layer transparent display screen is stored, wherein when the image display program based on the multi-layer transparent display screen is executed by a processor, the image display program based on the multi-layer transparent display screen implements the steps of the image display method based on the multi-layer transparent display screen provided in the first aspect.

[0048] The beneficial effects of this invention are as follows: The image display method, system, device, and medium based on a multi-layer transparent display screen provided by this invention, based on semantic pixel-level tag allocation (such as object depth and edge features), precisely controls the image content of each layer, making inter-layer fusion more natural, improving image sharpness, and effectively eliminating edge misalignment and ghosting caused by traditional segmentation; by performing transparent background completion processing on sub-images, it reduces pixel interference in non-content areas, significantly reducing background distortion while maintaining light transmittance; by dynamically adjusting the number of segmentation layers, it is compatible with scenarios requiring simultaneous presentation of multiple information, such as commercial windows and AR navigation, significantly improving the user experience. This invention achieves pixel-level intelligent layering through a deep learning model, achieving a breakthrough improvement in the quality of multi-layer transparent displays.

[0049] Furthermore, the design principle of this invention is reliable, the structure is simple, and it has a very wide range of application prospects. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1This is a schematic flowchart of a method according to an embodiment of the present invention.

[0052] Figure 2 This is a schematic block diagram of a system according to an embodiment of the present invention.

[0053] Figure 3 This is a schematic diagram of the structure of a device provided in an embodiment of the present invention. Detailed Implementation

[0054] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0056] The image display method based on a multi-layer transparent display screen provided in this embodiment of the invention is executed by a computer device, and correspondingly, the image display system based on a multi-layer transparent display screen runs in the computer device.

[0057] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The executing entity can be an image display system based on a multi-layer transparent display screen. Depending on different requirements, the order of the steps in this flowchart can be changed, and some can be omitted.

[0058] like Figure 1 As shown, the method includes:

[0059] S1. Obtain the number of display screen layers of the display device, and set the number of image segmentation layers according to the number of display screen layers and the image content, wherein the number of segmentation layers does not exceed the number of display screen layers;

[0060] S2. Use a deep learning model to assign one or more labels to each pixel of the image, wherein the labels indicate the corresponding display layer index and the labels have a mapping relationship with the display layer index;

[0061] S3. Group the image into multiple sub-images according to the label, with each sub-image corresponding to a display layer index;

[0062] S4. Perform transparent background completion processing on the sub-image to optimize the display effect;

[0063] S5. Output the processed sub-image to the corresponding display layer according to the display layer index of the sub-image.

[0064] In one embodiment of the present invention, based on step S1, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0065] S101. Obtain the number of transparent LED display layers of the display device and set the number of display layers of the display device to the maximum available number of layers.

[0066] Display devices typically have a dedicated hardware information reading module that communicates with the display control chip via the system bus. Taking I²C or SPI communication protocols as an example, the hardware information reading module sends a specific query command to the control chip. Upon receiving the command, the control chip extracts the current layer number information of the transparent LED display from its internally stored device configuration information and sends this information back to the reading module. Due to the physical structure and hardware limitations of the display, there is a theoretical maximum number of layers, which is also stored in the configuration register of the control chip. After obtaining the current layer number information, the reading module further compares it with the maximum layer number information in the configuration register, determines the larger value as the maximum available layer number, and sets the display layer number to this maximum available layer number through the system control interface, thus providing hardware-level support for subsequent image segmentation processing.

[0067] From a system architecture perspective, this process involves the collaborative work of the hardware and software layers. The display control chip at the hardware layer is responsible for storing and providing information related to the number of display layers, while the device management program at the software layer interacts with the hardware through a driver interface. At the operating system level, the device management program, based on a device driver model, constructs a complete device information acquisition and configuration process. When the system starts or a display device is connected, the device management program first identifies the transparent LED display device through a device enumeration mechanism, then calls a specific function in the driver program to trigger the hardware information reading module to perform the layer acquisition operation. After obtaining the maximum number of available layers, the device management program writes the set value into the configuration register of the display control chip through device control commands, completing the process of setting the number of display layers.

[0068] S102. Extract the optical properties of the image, including transparency, depth of field, and contrast.

[0069] For transparency extraction, a pixel-value-based analysis method is employed. In the RGB color space, each pixel is represented by three components: red (R), green (G), and blue (B). In the RGBA color space, which supports transparency information, an additional transparency channel (A) is added. The algorithm iterates through each pixel of the image, reading the value of its A channel. This value typically ranges from 0 to 255, where 0 represents complete transparency and 255 represents complete opacity. For images that do not directly support the RGBA format, color space conversion is required first. Taking a common JPEG image as an example, it can be converted to RGBA format using an image processing library (such as OpenCV) before transparency information is extracted. To improve the accuracy of transparency extraction, a neighborhood mean filtering algorithm can be used to smooth the extracted transparency data, reducing transparency fluctuations caused by image noise.

[0070] Depth extraction involves the depth information of an image. In computer vision-based methods, commonly used depth calculation techniques include stereo vision and monocular vision. Stereo vision uses two or more cameras to capture the same scene from different angles, utilizing the parallax principle to calculate the depth information of each point in the image. Specifically, it first extracts feature points from images acquired by multiple cameras (using algorithms such as SIFT and SURF), then establishes correspondences between different images using feature point matching algorithms (such as nearest neighbor matching based on Euclidean distance), and finally calculates the depth value of each point based on triangulation principles. Monocular vision is more complex and usually requires combining prior scene knowledge or machine learning models to estimate depth. For example, deep learning-based monocular depth estimation models are trained on a large amount of image data with depth labels to learn the mapping relationship between image features and depth, thus enabling depth calculation of input images. In practical applications, the appropriate depth calculation method can be selected based on the specific application scenario and hardware conditions.

[0071] Contrast extraction employs a statistical method. First, the image is divided into several appropriately sized sub-regions (e.g., 8×8 pixel blocks). For each sub-region, the maximum pixel value I is calculated. max and minimum value I min Then, according to the contrast calculation formula Calculate the contrast value of the sub-region. To obtain the contrast of the entire image, a weighted average method can be used. Different weights are assigned to the sub-regions based on their position and importance in the image, and the weighted average of the contrast values ​​of all sub-regions is calculated as the image contrast. Furthermore, image processing techniques such as histogram equalization can be combined to enhance the contrast information of the image and improve the accuracy of contrast extraction.

[0072] S103. Determine the actual number of segmentation layers based on the optical properties and preset rules, wherein the actual number of segmentation layers does not exceed the maximum number of usable layers.

[0073] A threshold range for optical attributes is preset, and a corresponding number of segmentation layers is set for each threshold range. Based on the threshold range to which the optical attributes of the image belong and the number of segmentation layers corresponding to that threshold range, a theoretical number of segmentation layers is determined for the image. If the theoretical number of segmentation layers exceeds the maximum available number of layers, the maximum available number of layers is output as the actual number of segmentation layers of the image. If the theoretical number of segmentation layers does not exceed the maximum available number of layers, the theoretical number of segmentation layers is output as the actual number of segmentation layers of the image.

[0074] For transparency, based on actual display requirements and hardware performance, its threshold range is divided into multiple intervals, such as [0,63], [64,127], [128,191], and [192,255], with a corresponding number of segmentation layers assigned to each interval, for example, 2, 3, 4, and 5 layers respectively. When determining the theoretical number of segmentation layers for an image, the threshold range to which the image transparency belongs is first determined, and then the theoretical number of segmentation layers is determined based on the segmentation layer number corresponding to that range. A similar method is used for depth of field and contrast to divide the threshold range and set the corresponding number of segmentation layers. In practical applications, the setting of the threshold range and the corresponding number of segmentation layers needs to be optimized through extensive experiments and data analysis to achieve the best display effect and hardware resource utilization efficiency.

[0075] After obtaining the theoretical number of partitioning layers, it needs to be compared with the maximum available number of layers. The specific comparison and output logic is implemented through programming. Taking Python as an example, the following code snippet can be used:

[0076] if theoretical_layers>max_available_layers:

[0077] actual_layers = max_available_layers,

[0078] else:

[0079] actual_layers = theoretical_layers.

[0080] This process ensures that the number of image segmentation layers does not exceed the hardware capacity of the display device, guaranteeing display quality while avoiding hardware resource constraints and performance degradation caused by over-segmentation. In this way, a dynamic match is achieved between the number of image segmentation layers and the hardware performance of the display device, improving the overall display quality and operating efficiency of the transparent LED display screen.

[0081] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0082] The fully convolutional network (FCN) architecture used in deep learning models is based on the encoder-decoder paradigm, achieving end-to-end pixel-level classification. Unlike traditional convolutional neural networks, FCN organically combines feature extraction and spatial localization capabilities by removing fully connected layers, making it particularly suitable for multi-layer image segmentation tasks on transparent LED displays.

[0083] The encoder uses a pre-trained ResNet-50 as its backbone network, progressively reducing the feature map resolution through five stages of convolutional operations. Each stage contains multiple residual blocks; the low-resolution feature maps generated in deeper stages capture high-level semantic information, while the high-resolution feature maps generated in shallower stages preserve rich spatial details. To enhance feature representation capabilities, batch normalization and ReLU activation functions are applied after each convolutional layer.

[0084] The decoder uses cascaded transposed convolutional layers to progressively restore the feature map size. Each transposed convolutional layer magnifies the feature map by a factor of 2 and reduces the number of channels. Through a skip connection mechanism, feature maps of the same resolution in the encoder are fused into the corresponding layer of the decoder, effectively solving the problem of detail loss caused by simple upsampling, and enabling the model to utilize both deep semantic information and shallow spatial details.

[0085] The output layer uses a 1×1 convolutional kernel to map the multi-channel feature map output by the decoder to a number of channels equal to the number of display layers. This linear transformation projects the high-dimensional features into the label space. To ensure that the prediction of each pixel in each layer is statistically significant, a Softmax activation function is used for normalization, so that the predicted value of each pixel in all channels conforms to the probability distribution characteristics, providing a theoretical basis for subsequent optimization.

[0086] Conditional Random Fields (CRFs) are used as a post-processing module to refine the FCN output by leveraging local image context information and spatial continuity. This study employs a fully connected CRF model, whose energy function includes both univariate and binary potentials. The univariate potential directly uses the probability values ​​from the FCN output, while the binary potential captures inter-pixel dependencies through a Gaussian kernel function, considering both spatial distance and color similarity.

[0087] The mean field approximation algorithm is used to perform efficient inference on the CRF model. By iteratively updating the label distribution of each pixel, it can usually converge in 5-10 iterations, which significantly improves the accuracy and spatial consistency of the segmentation boundary.

[0088] The process of assigning one or more labels to each pixel of an image using the deep learning model described above includes:

[0089] S201. Input the image, the number of segmentation layers, and the number of display screen layers into the deep learning model, and the deep learning model outputs the probability of a pixel in each channel;

[0090] S202. Set a threshold, and all channels with a probability greater than the threshold are considered as valid layer indices;

[0091] S203. Assign a corresponding label to the pixel based on the effective layer index.

[0092] The pixel label assignment process converts the probability tensor into a discrete label set. The multi-label assignment algorithm proposed in this invention is based on a probability threshold mechanism. When the predicted probability of a certain channel exceeds a preset threshold, the display layer corresponding to that channel is considered a valid label for that pixel. This design allows a pixel to belong to multiple display layers simultaneously, making it particularly suitable for handling semi-transparent areas in transparent LED display scenarios.

[0093] Furthermore, to avoid isolated small region labels caused by noise or model errors, this invention designs a label correction mechanism. By analyzing the number of pixels in each label region, when the number of pixels corresponding to a certain label is less than a preset threshold, the region is merged into its largest neighbor region. This mechanism effectively improves the spatial consistency of the segmentation results through region connectivity analysis.

[0094] In one embodiment of the present invention, based on step S3, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0095] To efficiently perform image grouping operations, a reasonable data structure design is required for image data and label information. Image data is stored using a three-dimensional array, with dimensions corresponding to the image's height H, width W, and number of color channels (e.g., 3 for RGB images, 4 for RGBA images), denoted as image[H][W][Ccolor]. Pixel label information is also stored as a two-dimensional array, with dimensions corresponding to the image's height and width, denoted as labels[H][W], where each element's value represents the display layer index of the corresponding pixel. Furthermore, to store the grouped sub-images, a dynamic array sub_images is constructed, with a length equal to the number of display layers C, and each element corresponding to a sub-image. Sub-images also use a three-dimensional array structure, i.e., sub_images[c][H][W][Ccolor], where c represents the display layer index, with a value ranging from 0 to C−1.

[0096] The specific grouping process includes:

[0097] Initialization phase: Before performing the grouping operation, the dynamic array sub_images that stores the sub-images needs to be initialized. Traverse sub_images, allocate memory space for each element with the same height, width and number of color channels as the original image, and initialize the pixel values ​​in the sub-image array to the default background values ​​(such as black, RGB value [0,0,0]).

[0098] Traversal and Grouping Phase: A double loop is used to traverse each pixel of the image. The outer loop controls the height direction of the image, and the inner loop controls the width direction. In each loop, the label value l of the pixel at coordinates (i,j) is obtained through labels[i][j]. This label value corresponds to the display layer index. Based on the obtained label value l, the pixel at coordinates (i,j) in the original image is copied to the corresponding position in sub_images[l][i][j], completing the transfer of the pixel from the original image to the corresponding sub-image. In this way, pixels with the same label (i.e., belonging to the same display layer) are aggregated into the corresponding sub-image.

[0099] Post-processing stage: After grouping, post-processing operations can be performed on the generated sub-images according to actual needs. For example, for some sub-images with excessively sparse pixel distribution, preprocessing techniques such as image scaling and cropping can be used to remove invalid blank areas, reduce data volume, and improve the efficiency of subsequent display processing. Simultaneously, color correction and contrast adjustment can be performed on each sub-image to optimize the display effect of each layer.

[0100] In one embodiment of the present invention, based on step S4, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0101] S401. Repair Area Determination and Initialization

[0102] When determining the areas in a sub-image that need repair, various detection strategies can be employed. Threshold-based detection methods identify repair areas by setting pixel value standards. For grayscale images, pixels with grayscale values ​​below a specific value (e.g., 30) are marked as pixels to be repaired; for color images, thresholds are set for the red, green, and blue color channels respectively, and if the value of any channel is below the corresponding threshold, the pixel is included in the candidate repair area.

[0103] Morphologically based detection methods first use image processing operations such as opening and closing operations to remove noise and minor interference from the image. Then, through connected component analysis, regions whose area, shape, and other features meet preset conditions (such as area greater than a certain minimum value and perimeter less than a certain maximum value) are selected as the repair regions.

[0104] Once the repair area is determined, the pixel values ​​within that area need to be set to specific values. In practice, this specific value is usually chosen as a marker value that facilitates subsequent processing, such as 0 in grayscale images and black in color images (i.e., the red, green, and blue channel values ​​are all 0). This clearly marks the repair area and provides a clear starting boundary for subsequent image repair operations.

[0105] S402. Similarity Calculation and Selection of Diffusion Sources

[0106] When calculating the similarity between the out-of-domain pixels adjacent to the edge of the repaired region and the region edge, a metric model that comprehensively considers multiple factors is adopted. This model evaluates the similarity between pixels from three dimensions: spatial distance, color features, and texture features.

[0107] In terms of spatial distance, the straight-line distance between out-of-domain pixels and region edge pixels is calculated; the closer the distance, the closer the spatial locations. Color feature similarity is calculated by comparing the differences in the values ​​of the red, green, and blue color channels, assigning different weights based on the importance of each channel (e.g., even weighting); the smaller the difference, the more similar the colors. Texture feature similarity is evaluated by extracting texture parameters such as contrast and correlation using the gray-level co-occurrence matrix, and then comparing these parameters between out-of-domain pixels and region edge pixels; the closer the parameters, the more similar the textures.

[0108] Finally, the evaluation results from the three dimensions are combined according to certain weights to obtain a comprehensive similarity metric. The weights of each dimension need to be optimized and adjusted through extensive experiments based on the actual repair results.

[0109] Based on the calculated similarity metric, out-of-domain pixels with high similarity are selected as diffusion sources. In practice, a similarity threshold is set; only out-of-domain pixels with similarity metrics exceeding this threshold are included in the diffusion source set. To ensure the diversity and effectiveness of the diffusion sources, all out-of-domain pixels are sorted from highest to lowest similarity value, and the top few pixels are selected as the final diffusion sources based on the size of the repaired region and the image complexity.

[0110] S403. Information Dissemination and Area Restoration

[0111] Information about the diffusion source is transferred to the area requiring repair using a priority-based diffusion method. A priority queue is constructed to store diffusion sources, with pixels in the queue ordered from highest to lowest similarity value. Each time, the diffusion source pixel with the highest similarity value is retrieved from the queue, and its pixel value is transferred to its adjacent unrepaired pixels within the repair area.

[0112] During the pixel value transfer process, a weighted average method is used to update the target pixel value to avoid significant differences between the repair area and the surrounding area. The ratio of the diffusion source pixel value to the original target pixel value is adjusted according to the fusion coefficient, which ranges from 0 to 1. The specific value can be flexibly adjusted according to the actual repair effect.

[0113] After each round of information diffusion, the edges of the filled region need to be updated. Morphological dilation is used to expand the boundaries of the filled region, and then logical operations (such as comparing to identify differences) are performed between the dilated region and the original repaired region to obtain an updated set of edge pixels. Based on the updated edges, the similarity between adjacent out-of-domain pixels and the new edges is recalculated, and diffusion sources are selected again according to the previous diffusion source selection strategy for the next round of region repair. This process of information diffusion, edge updating, and diffusion source selection is repeated until all pixels within the repaired region are reasonably filled, achieving visual consistency between the repaired region and its surrounding areas.

[0114] S404. Smoothing of Repair Area

[0115] The filled area is then smoothed using a bilateral filtering algorithm. This algorithm considers both the spatial distance and color differences between pixels when processing the image. Pixels that are close together and have small color differences are given higher weights during the filtering process. This effectively eliminates transition artifacts between the repaired area and its surrounding areas while preserving the edge details of the image.

[0116] In the implementation of bilateral filtering, spatial weights and color weights are defined to measure the relationship between pixels. Spatial weights are calculated based on the distance between pixels, with closer pixels receiving higher weights; color weights are calculated based on the difference in pixel color, with similar colors receiving higher weights. Finally, the filtered pixel value is obtained by weighted averaging of the neighboring pixels. The calculation parameters for spatial and color weights need to be finely adjusted based on image characteristics and desired restoration effect.

[0117] Through the above complete technical process, efficient and high-quality restoration of the sub-image restoration area is achieved, thereby improving the image display effect of the transparent LED display screen.

[0118] In one embodiment of the present invention, based on step S5, a possible embodiment will be given below, and its specific implementation will be described in a non-limiting manner.

[0119] A layered output mechanism is constructed based on the index mapping relationship between sub-images and display layers. The system establishes the correspondence between sub-images and target display layers by parsing the layer index tags in the sub-image filenames or metadata (e.g., 'c' in sub_image_c.png represents the layer index). An asynchronous output queue management mechanism is employed, storing the processed sub-images in a buffer queue according to their layer index order. Each sub-image data is then transferred to the corresponding display layer's video memory address space via a hardware driver interface. During the output process, data format verification and resolution matching are performed to ensure that the sub-image size matches the physical resolution of the display layer. Gamma correction and color space conversion (e.g., from RGB to the display's specific color space) are used to optimize the display effect, ultimately achieving accurate mapping output between sub-images and display layers.

[0120] In some embodiments, the image display system based on a multi-layer transparent display screen may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the image display system based on a multi-layer transparent display screen may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 1 (Description) Image display functionality based on a multi-layer transparent display screen.

[0121] In this embodiment, the image display system based on a multi-layer transparent display screen can be divided into multiple functional modules according to the functions it performs, such as... Figure 2 As shown. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and is stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0122] The layer number determination module is used to obtain the number of display screen layers of the display device, and set the number of image segmentation layers according to the number of display screen layers and the image content, wherein the number of segmentation layers does not exceed the number of display screen layers;

[0123] The pixel prediction module is used to assign one or more labels to each pixel of an image using a deep learning model. The labels indicate the corresponding display layer index, and the labels and display layer indexes have a mapping relationship.

[0124] The image grouping module is used to group an image into multiple sub-images according to the label, and each sub-image corresponds to a display layer index;

[0125] The image completion module is used to perform transparent background completion processing on sub-images to optimize the display effect;

[0126] The image output module is used to output the processed sub-image to the corresponding display layer according to the display layer index of the sub-image.

[0127] Figure 3 The image display method based on a multi-layer transparent display screen provided in the embodiments of this application can be applied to devices. Those skilled in the art will understand that the device structure involved in the embodiments of this invention does not constitute a limitation on the device. A device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the device includes, but is not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0128] The device 300 may include a processor 310, a memory 320, and a communication unit 330. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0129] The memory 320 can be used to store execution instructions of the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the device 300 is able to perform some or all of the steps in the above method embodiments.

[0130] The processor 310 serves as the control center of the storage device, connecting various parts of the electronic device via various interfaces and lines. It executes software programs and / or modules stored in the memory 320, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 310 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0131] The communication unit 330 is used to establish a communication channel, enabling the storage device to communicate with other devices. It can receive user data sent by other devices or send user data to other devices.

[0132] The present invention also provides a computer medium, wherein the computer medium may store a program, which, when executed, may include some or all of the steps provided in the embodiments of the present invention. The medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0133] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a medium such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other medium capable of storing program code. It includes several instructions to cause a computer device (which may be a personal computer, a server, or a second device, network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0134] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0135] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0136] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0138] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. An image display method based on a multi-layer transparent display screen, characterized in that, include: The number of display screen layers of the display device is obtained, and the number of image segmentation layers is set according to the number of display screen layers and the image content, wherein the number of segmentation layers does not exceed the number of display screen layers; A deep learning model is used to assign one or more labels to each pixel of an image. The labels indicate the corresponding display layer index, and there is a mapping relationship between the labels and the display layer index. The image is grouped into multiple sub-images according to the label, and each sub-image corresponds to a display layer index; Transparent background padding is applied to the sub-images to optimize the display effect; Based on the display layer index corresponding to the sub-image, the processed sub-image is output to the corresponding display layer; Obtaining the number of display screen layers of the display device, and setting the number of image segmentation layers based on the number of display screen layers and the image content, includes: The display screen layer of the display device is set to the maximum number of available layers; the optical properties of the image are extracted, including transparency, depth of field, and contrast. The actual number of segmentation layers is determined based on the optical properties and preset rules, wherein the actual number of segmentation layers does not exceed the maximum number of usable layers. The method for determining the actual number of segmentation layers includes: Pre-set the threshold range of optical properties and set the corresponding number of segmentation layers for each threshold range; The theoretical number of segmentation layers for the image is determined based on the threshold range to which the optical properties of the image belong and the number of segmentation layers corresponding to that threshold range. If the theoretical number of segmentation layers exceeds the maximum number of usable layers, then the maximum number of usable layers is output as the actual number of segmentation layers of the image; If the theoretical number of segmentation layers does not exceed the maximum number of usable layers, then the theoretical number of segmentation layers is output as the actual number of segmentation layers of the image; Using a deep learning model to assign one or more labels to each pixel of an image, including: The image, the number of segmentation layers, and the number of display screen layers are input into the deep learning model, and the deep learning model outputs the probability of a pixel in each channel; A threshold is set, and channels with a probability greater than the threshold are considered valid layer indices; The corresponding label is assigned to the pixel based on the effective layer index; If the number of pixels corresponding to any label is less than the set pixel number threshold, then the pixels corresponding to that label will be moved to a label in an adjacent layer.

2. The method according to claim 1, characterized in that, The deep learning model uses a fully convolutional network as its basic framework, including: An encoder is used to extract multi-level image features through convolutional layers, where deep features capture semantic information and shallow features preserve spatial details. The decoder is used to progressively upsample feature maps to the original image size through transposed convolutional layers and fuse shallow features of the corresponding resolution in the encoder through skip connections. The output layer uses 1x1 convolutions to adjust the number of channels in the feature map output by the decoder to the number of display layers C, generating a probability tensor of size [H, W, C], where each channel corresponds to the predicted probability of a display layer index; the activation function of the output layer is Softmax, which makes the sum of the probabilities of each pixel in all channels equal to 1; The Conditional Random Field (CRF) optimization module operates on the probability tensor of the output layer, refining the segmentation boundary by minimizing an energy function; the energy function includes: Univariate potential energy: provided by the output layer probability tensor; Binary potential energy: defined by the Gaussian kernel, including the spatial distance kernel and the color similarity kernel.

3. The method according to claim 1, characterized in that, Perform transparent background completion processing on the sub-image, including: Identify the areas in the sub-image that need repair, and set the pixel values ​​within those areas to pre-specified specific values; Calculate the similarity between the out-of-domain pixels adjacent to the edge of the region and the edge of the region, and use the out-of-domain pixels with high similarity as diffusion sources; Based on similarity, information about the diffusion source is gradually transferred to the area that needs to be repaired to fill the missing parts of the area; The edges of the filled region are updated, and diffusion sources are re-identified based on the updated edges to repair the region until the region is completely filled. Smooth the area after it has been filled.

4. An image display system based on a multi-layer transparent display screen, characterized in that, include: The layer number determination module is used to obtain the number of display screen layers of the display device, and set the number of image segmentation layers according to the number of display screen layers and the image content, wherein the number of segmentation layers does not exceed the number of display screen layers; The pixel prediction module is used to assign one or more labels to each pixel of an image using a deep learning model. The labels indicate the corresponding display layer index, and the labels and display layer indexes have a mapping relationship. The image grouping module is used to group an image into multiple sub-images according to the label, and each sub-image corresponds to a display layer index; The image completion module is used to perform transparent background completion processing on sub-images to optimize the display effect; The image output module is used to output the processed sub-image to the corresponding display layer according to the display layer index of the sub-image; Obtaining the number of display screen layers of the display device, and setting the number of image segmentation layers based on the number of display screen layers and the image content, includes: The display screen layer of the display device is set to the maximum number of available layers; the optical properties of the image are extracted, including transparency, depth of field, and contrast. The actual number of segmentation layers is determined based on the optical properties and preset rules, wherein the actual number of segmentation layers does not exceed the maximum number of usable layers. The method for determining the actual number of segmentation layers includes: Pre-set the threshold range of optical properties and set the corresponding number of segmentation layers for each threshold range; The theoretical number of segmentation layers for the image is determined based on the threshold range to which the optical properties of the image belong and the number of segmentation layers corresponding to that threshold range. If the theoretical number of segmentation layers exceeds the maximum number of usable layers, then the maximum number of usable layers is output as the actual number of segmentation layers of the image; If the theoretical number of segmentation layers does not exceed the maximum number of usable layers, then the theoretical number of segmentation layers is output as the actual number of segmentation layers of the image; Using a deep learning model to assign one or more labels to each pixel of an image, including: The image, the number of segmentation layers, and the number of display screen layers are input into the deep learning model, and the deep learning model outputs the probability of a pixel in each channel; A threshold is set, and channels with a probability greater than the threshold are considered valid layer indices; The corresponding label is assigned to the pixel based on the effective layer index; If the number of pixels corresponding to any label is less than the set pixel number threshold, then the pixels corresponding to that label will be moved to a label in an adjacent layer.

5. An image display device based on a multi-layer transparent display screen, characterized in that, include: Memory, used to store image display programs based on multi-layer transparent displays; A processor, configured to implement the steps of the image display method based on a multi-layer transparent display screen as described in any one of claims 1-3 when executing the image display program based on the multi-layer transparent display screen.

6. A computer-readable medium storing a computer program, characterized in that, The readable medium stores an image display program based on a multi-layer transparent display screen, which, when executed by a processor, implements the steps of the image display method based on a multi-layer transparent display screen as described in any one of claims 1-3.

Images