Display screen image shadow restoration method and device, display screen and storage medium

Abstract

The invention relates to the technical field of image display, and provides a display screen image shadow restoration method and device, a display screen and a storage medium. The method comprises the following steps: performing color space conversion on a current display image of a display screen to obtain a first display image; performing global shadow restoration on the first display image through a global restoration network to obtain a second display image; performing a first round of local shadow repair on the second display image through the first local correction network to obtain a third display image; and performing a second round of local shadow repair on the third display image through the second local correction network to obtain a target display image. According to the invention, global restoration of the image shadow of the display screen and further two rounds of local shadow restoration are realized, the image display quality of the display screen is improved, and the technical problem that the local shadow of the image cannot be effectively restored in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of image display technology, and in particular to a display screen image shadow repair method, device, electronic device and storage medium. Background Art

[0002] As an important component of modern electronic devices, the display screen is directly related to the user experience. However, long-term use of the display screen is prone to image shadow problems, which damages the user's visual experience and has an adverse effect on the long-term use of the display screen.

[0003] To address the shadow problem of display screen images, existing image shadow restoration technologies mainly use Gaussian distribution modeling and deep learning methods to perform shadow restoration. Gaussian modeling repairs image shadows by estimating the brightness distribution of shadow and non-shadow areas. Deep learning methods focus on learning the mapping relationship between shadows and shadow-free images through large-scale training data to achieve illumination and texture restoration in shadow areas.

[0004] However, existing image shadow restoration technology mainly repairs the overall shadow of the image. When dealing with complex shadow textures and shadow boundary problems, especially when facing local shadows with fine textures or fuzzy boundaries, it is impossible to effectively repair the local shadows, resulting in poor actual shadow restoration effects, discontinuities in local content of the display image, or the appearance of artifacts. Summary of the invention

[0005] In view of this, the present application provides a display screen image shadow repair method, device, display screen and storage medium, which are used to solve the technical problem that the existing shadow repair technology cannot effectively repair the local shadow of the image.

[0006] In a first aspect, the present application provides a method for repairing shadows of a display screen image, which adopts the following technical solution: Acquire a current display image of the display screen, and perform color space conversion on the current display image to obtain a first display image; Performing global shadow restoration on the first display image through a global restoration network to obtain a second display image; Performing a first round of local shadow repair on the second display image by using a first local correction network to obtain a third display image; A second round of local shadow repair is performed on the third display image through a second local correction network to obtain a target display image.

[0007] Optionally, performing global shadow restoration on the first display image by using a global restoration network to obtain a second display image includes: Performing encoding processing on the first display image by the encoding module to obtain a first feature image; Performing dilated convolution processing on the first feature image through the dilated convolution module to obtain a second feature image; The second feature image is decoded by the decoding module to obtain the second display image.

[0008] Optionally, encoding the first display image by the encoding module to obtain a first feature image includes: Processing the first display image through the first convolution layer to obtain a first convolution image; Performing encoding processing on the first convolution image through a first encoding block to generate a first encoded image, and performing downsampling processing on the first encoded image through a first downsampling layer corresponding to the first encoding block to obtain a first downsampled image; encoding the first downsampled image through a second encoding block to generate a second encoded image, and downsampling the second encoded image through a second downsampled layer corresponding to the second encoding block to obtain a second downsampled image; Until the N-1th downsampled image is encoded by the Nth encoding block to generate the Nth encoded image, and the Nth encoded image is downsampled by the Nth downsampling layer corresponding to the Nth encoding block to obtain the first feature image.

[0009] Optionally, performing dilated convolution processing on the first feature image by the dilated convolution module to obtain a second feature image includes: Get the number of dilated convolution layers in the dilated convolution module and the dilation of each dilated convolution layer. Exhibition rate; Convolving the first feature image according to the number of the dilated convolutional layers and the dilation rate Dilate to obtain the second feature image.

[0010] Optionally, the decoding the second feature image by the decoding module to obtain the second display image includes: Performing upsampling processing on the second feature image through a first upsampling layer to obtain a first upsampling image, and performing decoding processing on the first upsampling image through a first decoding block corresponding to the first upsampling layer to obtain a first decoded image; Performing upsampling processing on the first decoded image through a second upsampling layer to obtain a second upsampling image, and performing decoding processing on the second upsampling image through a second decoding block corresponding to the second upsampling layer to obtain a second decoded image; Until the N-1th decoded image is upsampled by the Nth upsampling layer to obtain the Nth upsampled image, and the Nth upsampled image is decoded by the Nth decoding block corresponding to the Nth upsampling layer to obtain the Nth decoded image; The Nth decoded image is convolved by the second convolution layer to obtain the second display image.

[0011] Optionally, performing a first round of local shadow repair on the second display image by using a first local correction network to obtain a third display image includes: Acquire a first residual data set according to the first display image and the second display image; Filter out first target residual values ​​whose residual values ​​are less than a first preset residual threshold from the first residual data set to obtain a first target residual value set; Performing encoding processing on the first target residual value set by the first local encoding module to obtain a first local encoded image set; Decoding the first local coded image set by the first local decoding module to obtain a first local decoded image set; The first local decoded image set is deconvolved by the first local deconvolution layer to obtain the third display image.

[0012] Optionally, performing a second round of local shadow repair on the third display image by using a second local correction network to obtain a target display image includes: Acquire a second residual data set according to the first display image and the third display image; Filter out second target residual values ​​whose residual values ​​are less than a second preset residual threshold from the second residual data set to obtain a second target residual value set; Performing encoding processing on the second target residual value set by the second local encoding module to obtain a second local encoded image set; Decoding the second local coded image set by the second local decoding module to obtain a second local decoded image set; Performing deconvolution processing on the second local decoded image set through the second local deconvolution layer to obtain the target display image; The second preset residual threshold is smaller than the first preset residual threshold.

[0013] In a second aspect, the present application provides a display screen image shadow repair device, the device comprising: An image conversion module, used for acquiring a current display image of the display screen, and performing color space conversion on the current display image to obtain a first display image; A first restoration module, configured to perform global shadow restoration on the first display image through a global restoration network to obtain a second display image; A second restoration module, configured to perform a first round of local shadow restoration on the second display image through a first local correction network to obtain a third display image; The third restoration module is used to perform a second round of local shadow restoration on the third display image through a second local correction network to obtain a target display image.

[0014] In a third aspect, the present application provides a display screen device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the display screen image shadow repair method when executing the computer program.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium, including a computer program, which, when executed by a processor, implements the steps of the display screen image shadow repair method.

[0016] The present application obtains a first display image by performing color space conversion on an image currently displayed on a display screen, thereby facilitating processing of the first display image; performs global shadow repair on the first display image through a global restoration network to obtain a second display image, thereby achieving global overall shadow repair of the first display image; performs a first round of local shadow repair on the second display image through a first local correction network to obtain a third display image, thereby achieving the first round of local shadow repair after the global shadow repair, repairing local shadows that may still exist after the global repair, and improving the local display quality of the image; performs a second round of local shadow repair on the third display image through a second local correction network to obtain a target display image, thereby further achieving local shadow repair of the display image, improving the local shadow effect, and obtaining a higher quality target display image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a process of a display screen image shadow repair method according to an embodiment of the present application; Figure 2 This is a functional module diagram of a display screen image shadow repair device shown in an embodiment of the present application; Figure 3 A structural diagram of a display screen device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more of the listed items.

[0019] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0020] See also Figure 1 As shown, a display screen image shadow repair method shown in an embodiment of the present application includes the following steps.

[0021] S11, acquiring a current display image of a display screen, and performing color space conversion on the current display image to obtain a first display image.

[0022] Color space conversion refers to converting the current display image from RGB color space to HSV color space. Among them, RGB color space is a universal color space for display screens. Through the different intensities and combinations of the three basic colors of red, green and blue in the RGB color space, images of various colors can be generated and displayed on the display screen. The HSV color space consists of three components: hue, saturation and value. It is closer to the intuitive perception of color and is convenient for color analysis and processing. For example, the type of color can be changed by adjusting the hue, the depth of the color can be changed by adjusting the saturation, and the brightness of the color can be changed by adjusting the brightness.

[0023] When the display screen displays the same image for a long time, part of the image of the previous frame may remain, resulting in a shadow when switching to the next frame, and the displayed image is no longer completely clear. When the display screen displays the same image for a long time (such as more than 3 seconds), the current display image is obtained, and the color space of the current display image is switched. Through a mature color space switching algorithm, the current display image can be converted from the RGB color space to the HSV color space to obtain the current display image in the HSV color space, that is, the first display image.

[0024] In the above embodiment, by converting the color space of the currently displayed image into the HSV color space and adjusting the color of the currently displayed image based on the three parameters of hue, saturation and brightness of the HSV color space, the shadows on the currently displayed image can be processed more effectively, thereby improving the clarity and color expression of the displayed image.

[0025] S12: Perform global shadow restoration on the first display image through a global restoration network to obtain a second display image.

[0026] The global restoration network includes an encoding module, an extended convolution module and a decoding module. The global restoration network is used to perform overall illumination restoration on the first display image to obtain a second display image.

[0027] In an optional embodiment, performing global shadow repair on the first display image by using a global restoration network to obtain a second display image includes: Performing encoding processing on the first display image by the encoding module to obtain a first feature image; Performing dilated convolution processing on the first feature image through the dilated convolution module to obtain a second feature image; The second feature image is decoded by the decoding module to obtain the second display image.

[0028] The encoding module is responsible for encoding the first display image, extracting and abstracting the features of the first display image, reducing the resolution of the first display image in the display space, and enhancing the feature representation of the first display image in the display space to obtain a first feature image. The first feature image is a higher-level feature representation of the first display image, and the features may include abstract concepts such as the shadow, structure, texture, and color distribution of the first display image. Based on these features, the shadow information of the first display image can be more accurately obtained.

[0029] The dilated convolution module is responsible for receiving the first feature image from the encoding module and performing dilated convolution processing. Dilated convolution is a technology in convolutional neural networks. It increases the receptive space by adding intervals in the middle of the convolution kernel of the dilated convolution module, and obtains the second feature image for a wider range of image feature information in the first feature image without introducing additional parameters and computational costs. The dilated convolution module enables the global correction network to better understand the global structure and local relationship of the first feature image, thereby achieving more accurate recognition of shadow features in the first feature image.

[0030] The decoding module is responsible for receiving the second feature image from the dilated convolution process and performing decoding processing. The decoding module converts the second feature image after the encoding process and the dilated convolution process back to the spatial dimension and resolution of the first display image, restores the shadow features on the second feature image, and obtains a high-quality second display image.

[0031] By converting the first display image into an abstract first feature image through the encoding template, the structure and content in the first display image can be effectively captured, redundant information and noise can be reduced, the ability to express the features of the first display image can be improved, and the shadow features can be accurately obtained. By increasing the receptive space of each feature in the first feature image through the dilated convolution module, the global structure of the first feature image can be better understood, and the accuracy of shadow recognition and restoration of the first display image can be improved. Through the decoding module effect, the feature image after encoding and dilated convolution processing can be remapped back to the spatial dimension of the first display image, and the details and resolution of the shadow features of the second feature image can be restored to obtain a high-quality second display image.

[0032] In the above embodiment, the global restoration network can be used to fully restore and enhance the first display image, thereby obtaining a high-quality second display image.

[0033] In an optional embodiment, encoding the first display image by the encoding module to obtain the first feature image includes: Processing the first display image through the first convolution layer to obtain a first convolution image; Performing encoding processing on the first convolution image through a first encoding block to generate a first encoded image, and performing downsampling processing on the first encoded image through a first downsampling layer corresponding to the first encoding block to obtain a first downsampled image; encoding the first downsampled image through a second encoding block to generate a second encoded image, and downsampling the second encoded image through a second downsampled layer corresponding to the second encoding block to obtain a second downsampled image; Until the N-1th downsampled image is encoded by the Nth encoding block to generate the Nth encoded image, and the Nth encoded image is downsampled by the Nth downsampling layer corresponding to the Nth encoding block to obtain the first feature image.

[0034] The encoding module includes a first convolutional layer, N downsampling layers and N encoding blocks, where N is an integer greater than 1, and the downsampling layers and the encoding blocks adopt a cross structure in the encoding module, and a downsampling layer is arranged after each encoding block.

[0035] Among them, the first convolution layer is used to receive the first display image as an input image, and extract features of the first display image through multiple convolution kernels to obtain a first convolution image. Among the N coding blocks, each coding block adopts the existing batch normalization layer and activation function structure, which is responsible for further extracting and converting the features of the input image. In each coding block, the number of channels of the feature map will gradually increase to capture more abstract and advanced features. Through multiple coding blocks, the network depth and expression ability of the image can be continuously stacked to achieve deep extraction of image shadow features. Among the N downsampling layers, each downsampling layer is used to reduce the spatial resolution of the image while increasing the number of channels of the image to achieve richer shadow feature extraction.

[0036] The first convolutional layer obtains the first convolutional image from the first coding block, takes the first convolutional image as input, encodes the first convolutional image through the first coding block, obtains the first coded image with more accurate feature representation, inputs the first coded image to the first downsampling layer, downsamples the first coded image, reduces the feature size of the first coded image and retains the main information, and obtains the first downsampled image. The first downsampled image is input to the second coding block, further encodes the first downsampled image through the second coding block, obtains the second coded image, and then inputs to the second downsampling layer, performs further downsampling processing, and obtains the second downsampled image.

[0037] The above process is repeated, and new encoding blocks and downsampling layers are used gradually and deeply. Each encoding block will further extract more abstract features and reduce the size of the feature map through the corresponding downsampling layer, and finally output the first feature image obtained after being processed by N encoding blocks and the corresponding N downsampling layers.

[0038] In the above embodiment, by stacking N coding blocks and N downsampling layers, deep extraction and high-level expression of the features of the first display image are effectively achieved, so that more accurate extraction of shadow features in the first display image is achieved.

[0039] In an optional embodiment, performing dilated convolution processing on the first feature image by the dilated convolution module to obtain the second feature image includes: Get the number of dilated convolution layers in the dilated convolution module and the dilation of each dilated convolution layer. Exhibition rate; Convolving the first feature image according to the number of the dilated convolutional layers and the dilation rate Dilate to obtain the second feature image.

[0040] The dilated convolution module usually uses multiple layers of dilated convolution layers, and the dilation rate of each layer determines the size of the receptive space of the convolution kernel. The dilated convolution layer increases the effective receptive space of the convolution kernel by inserting holes in the convolution kernel without increasing the number of parameters or calculations. The dilated convolution layer and the dilation rate of each layer are determined according to the feature requirements of the first feature image. The dilation rate determines the abstraction and detail capture capabilities of the first feature image at different levels. By dilating the convolution layer and different dilation rates, the first feature image is convoluted and expanded to obtain the second feature image.

[0041] For example, suppose the dilated convolution module contains 4 dilated convolution layers, and the dilation rate of each layer is 2, 4, 8, and 16, which means that the convolution kernel of each layer will skip 2 pixels, 4 pixels, 8 pixels, and 16 pixels in space when performing convolution operations. This can significantly increase the receptive space of each convolution kernel without increasing the size or parameter amount of the convolution kernel. The first dilated convolution layer with a dilation rate of 2 will be applied to the first feature image. By skipping every 2 pixels for convolution, it can capture relatively small features. The second dilated convolution layer with a dilation rate of 4 further expands the receptive space, spanning the image with a larger stride to capture medium-sized features. The third convolution layer with a dilation rate of 8 and the convolution layer with a dilation rate of 16 have the largest receptive space and can capture very broad and abstract features in the image, thereby obtaining a second feature image containing more feature information.

[0042] In the above embodiment, through the dilated convolution module, the first feature image will gradually extract and fuse more global and complex image features in each layer of convolution operation, so as to achieve more accurate and deeper shadow feature recognition and extraction of the first feature image, facilitate the subsequent decoding module to accurately repair the shadow feature area, improve the accuracy of shadow repair of the first feature image, and improve the image shadow processing effect.

[0043] In an optional embodiment, the decoding the second feature image by the decoding module to obtain the second display image includes: Performing upsampling processing on the second feature image through a first upsampling layer to obtain a first upsampling image, and performing decoding processing on the first upsampling image through a first decoding block corresponding to the first upsampling layer to obtain a first decoded image; Performing upsampling processing on the first decoded image through a second upsampling layer to obtain a second upsampling image, and performing decoding processing on the second upsampling image through a second decoding block corresponding to the second upsampling layer to obtain a second decoded image; Until the N-1th decoded image is upsampled by the Nth upsampling layer to obtain the Nth upsampled image, and the Nth upsampled image is decoded by the Nth decoding block corresponding to the Nth upsampling layer to obtain the Nth decoded image; The Nth decoded image is convolved by the second convolution layer to obtain the second display image.

[0044] The decoding module includes N upsampling layers, N decoding blocks and a second convolutional layer. Similar to the encoding module, the upsampling layers and the decoding blocks adopt a cross structure in the decoding module, and a decoding block is set after each upsampling layer.

[0045] Among them, there are N upsampling layers, which are used to increase the spatial resolution of the feature map, thereby gradually restoring the size and details of the image and repairing the shadow features. Each upsampling layer receives the feature image decoded by the previous layer and expands it to a larger size through interpolation operations. There are N decoding blocks, which correspond to the encoding blocks, but their functions are reversed. Each decoding block receives the upsampled image of the previous layer as input, and gradually reconstructs the features of the original image through operations such as convolution and activation functions.

[0046] The second feature image is processed by the first upsampling layer to obtain the first upsampling image, and then the first decoding block processes the first upsampling image to generate the first decoded image; subsequently, in contrast to the encoding path of the encoding block, the sampling processing of the upsampling layer layer by layer and the decoding processing of the decoding block are performed until the N-1 decoded image is upsampled by the Nth upsampling layer to obtain the Nth upsampling image, and then it is decoded by the Nth decoding block to finally obtain the Nth decoded image. In the Nth decoded image, the corresponding shadow features are repaired for multiple rounds to obtain a high-quality decoded image. Finally, the Nth decoded image is input into the second convolution layer for the final convolution processing. The number of channels of the final output is adjusted or some additional feature extraction and adjustment are performed to obtain the second display image. Global shadow repair of the first display image is achieved.

[0047] In the above embodiment, the spatial resolution of the feature map is gradually expanded through the upsampling layer to restore the size and details of the image, thereby realizing the repair of shadow features in the first feature image. Through deconvolution and feature reconstruction of the decoding block, a representation close to the original image is generated from the upsampled feature map, thereby realizing effective repair of the global shadow of the first display image.

[0048] S13, performing a first round of local shadow restoration on the second display image through a first local correction network to obtain a third display image.

[0049] Among them, the first local correction network includes a first local encoding module, a first local decoding module and a first local deconvolution layer, which have the same structure as the encoding module and decoding module of the global correction network, but the first local encoding module and the first local decoding module only act on the local area of ​​the second display image, for more accurate shadow feature repair of the local area.

[0050] In the above embodiment, the shadow features of the second display image are locally repaired through the first local correction network to obtain a third display image. Compared with the second display image, the third display image has better overall visual quality, and some areas that may still have shadows or missing parts after global repair are effectively repaired, thereby obtaining a higher quality third display image, thereby improving the image display quality of the display screen and the user experience.

[0051] In an optional embodiment, performing a first round of local shadow repair on the second display image by using a first local correction network to obtain a third display image includes: Acquire a first residual data set according to the first display image and the second display image; Filter out first target residual values ​​whose residual values ​​are less than a first preset residual threshold from the first residual data set to obtain a first target residual value set; Performing encoding processing on the first target residual value set by the first local encoding module to obtain a first local encoded image set; Decoding the first local coded image set by the first local decoding module to obtain a first local decoded image set; The first local decoded image set is deconvolved by the first local deconvolution layer to obtain the third display image.

[0052] After acquiring the first display image and the second display image, the pixel difference value of each pixel in the first display image and the second display image is acquired to obtain the residual data of each pixel, and the shadow characteristics of the corresponding pixel can be reflected based on the parameter data. Based on the residual data of each pixel of the first display image and the second display image as a whole, a first residual data set is obtained. From the calculated residual data set, a first target residual value set whose residual values ​​are less than a first preset residual threshold is screened out. The first target residual value set indicates that the corresponding pixel is abnormal, and there may be an incompletely repaired shadow area, which needs to be locally repaired through a local correction network.

[0053] The first target residual value set is input into the first local encoding module, and based on the mapping relationship between the first target residual value and the corresponding pixel point, the area of ​​the corresponding pixel point is encoded, wherein the encoding process is the same as the encoding process of the global correction network, but only the pixel points corresponding to the first target residual value are processed to obtain the first local encoded image set, which can accurately reflect the local shadow features in depth. The first local encoded image is decoded by the first local decoding module, and the shadow feature part thereof is repaired and restored. Finally, the decoded first local decoded image set block is deconvolved by the first local deconvolution layer, and the repaired local details are restored and integrated into the second display image to obtain the third display image after local repair.

[0054] In the above embodiment, by further local repair of the second display image, the local shadow in the second display image can be effectively repaired, and a higher quality third display image can be generated. The precision and accuracy of the display screen in repairing the displayed image are improved, the consistency of the local repair area with the surrounding environment is maintained, and the local excessive unevenness or distortion problems that may occur in traditional image repair methods are avoided, and the local detail quality of the image is significantly improved.

[0055] S14, performing a second round of local shadow repair on the third display image through a second local correction network to obtain a target display image.

[0056] Among them, the second local correction network includes a second local encoding module, a second local decoding module and a second local deconvolution layer, which have the same structure as the first encoding module, the first decoding module and the first local deconvolution layer of the first local correction network, but the second local encoding module, the second local decoding module and the second deconvolution layer are used for re-repairing after the first local repair to prevent the third display image after the first local repair from having shadows caused by the repair, and are used for further repair of shadow features in local areas.

[0057] In the above embodiment, the second local correction network performs local shadow feature repair on the third display image to obtain the final target display image. The second local correction network is more detailed and focused on the shadow residual problem that may appear after the first local correction network repairs the local shadow, ensuring that no new problems are introduced or obvious traces are left in the first round of local shadow repair process, thereby ensuring the quality of the final target display image.

[0058] In an optional embodiment, performing a second round of local shadow repair on the third display image by using a second local correction network to obtain a target display image includes: Acquire a second residual data set according to the first display image and the third display image; Filter out second target residual values ​​whose residual values ​​are less than a second preset residual threshold from the second residual data set to obtain a second target residual value set; Performing encoding processing on the second target residual value set by the second local encoding module to obtain a second local encoded image set; Decoding the second local coded image set by the second local decoding module to obtain a second local decoded image set; Performing deconvolution processing on the second local decoded image set through the second local deconvolution layer to obtain the target display image; The second preset residual threshold is smaller than the first preset residual threshold.

[0059] The second local correction network has the same process as the first local correction network, but in the second local correction network, the second preset residual threshold is smaller than the first preset residual threshold, which is used for further local shadow feature determination. By obtaining a second target residual value set between the third display image and the first display image, a second local encoded image set with more subtle shadow features is obtained through encoding processing by a second local encoding module, and the second target residual value set is encoded by the second local encoding module to repair the shadow feature area and restore the second local encoded image set to obtain a second local decoded image set. The second local deconvolution layer deconvolves the second local decoded image set so that it can be matched and integrated with the third display image to obtain a target display image.

[0060] In the above embodiment, the second local correction network uses a second preset residual threshold value that is more sensitive than the first local correction network to focus on identifying and repairing more subtle local shadow features in the image, so that a target display image with better repair quality in the shadow area can be obtained. This ensures that the target display image can achieve higher visual quality and realism while retaining details, and realizes shadow repair of the global content and local detail content of the display image on the display screen, thereby obtaining a high-quality display image.

[0061] In summary, the present application obtains a first display image by performing color space conversion on an image currently displayed on a display screen, thereby facilitating processing of the first display image; performs global shadow repair on the first display image through a global restoration network to obtain a second display image, thereby achieving global overall shadow repair of the first display image; performs a first round of local shadow repair on the second display image through a first local correction network to obtain a third display image, thereby achieving the first round of local shadow repair after the global shadow repair, repairing local shadows that may still exist after the global repair, and improving the local display quality of the image; performs a second round of local shadow repair on the third display image through a second local correction network to obtain a target display image, thereby achieving further removal of local shadows of the image, improving the quality of the displayed image, and solving the technical problem that the existing shadow repair technology cannot effectively repair local shadows of the image.

[0062] See also Figure 2 The above is a functional module diagram of the display screen image shadow repair device provided in an embodiment of the present application.

[0063] In some embodiments, the display screen image shadow repairing device 20 may include multiple functional modules composed of computer program segments. The computer programs of the various program segments in the display screen image shadow repairing device 20 may be stored in the memory of the display screen image shadow repairing system and executed by at least one processor to perform (see Figure 1 Description) Function of the display image shadow repair method.

[0064] In this embodiment, the display screen image shadow restoration device 20 can be divided into multiple functional modules according to the functions it performs. The functional modules can include: an image conversion module 201 , a first restoration module 202 , a second restoration module 203 and a third restoration module 204 .

[0065] The image conversion module 201 is used to obtain a current display image of the display screen, and perform color space conversion on the current display image to obtain a first display image.

[0066] The first restoration module 202 is configured to perform global shadow restoration on the first display image through a global restoration network to obtain a second display image.

[0067] The second restoration module 203 is configured to perform a first round of local shadow restoration on the second display image through a first local correction network to obtain a third display image.

[0068] The third restoration module 204 is configured to perform a second round of local shadow restoration on the third display image through a second local correction network to obtain a target display image.

[0069] Through the above functional modules, global repair of display screen image shadows and further two rounds of local shadow repair are achieved, which improves the image display quality of the display screen and solves the technical problem that the existing technology cannot effectively repair local image shadows.

[0070] An embodiment of the present application also provides a display screen device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements all or part of the steps of a display screen image shadow repair method when executing the computer program.

[0071] See also Figure 3 , which is a functional module diagram of an electronic device according to an embodiment of the present application. In a preferred embodiment of the present application, the display screen device 3 includes a memory 31 , at least one processor 32 , and at least one communication bus 33 .

[0072] Those skilled in the art should understand that Figure 3 The structure of the electronic device shown does not constitute a limitation of the embodiments of the present application, and may be either a bus structure or a star structure. The printer may also include more or less other hardware or software than shown in the figure, or a different arrangement of components.

[0073] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium is, for example, a non-volatile memory, such as: magnetic media (such as hard disks, floppy disks, and tapes), optical media (such as CDROM disks and DVDs), magneto-optical media (such as optical disks), and hardware devices specially constructed to store and execute computer-executable instructions (such as read-only memory (ROM), random access memory (RAM), flash memory, etc.). Computer-executable instructions are stored on the computer-readable storage medium. The computer-readable storage medium can be executed by one or more processors or processing devices to implement the display screen image shadow repair method in the aforementioned embodiment.

[0074] In addition, it can be understood that the aforementioned embodiments are merely illustrative descriptions of the present application. Under the premise that the technical features do not conflict, the structures do not contradict, and the purpose of the invention of the present application is not violated, the technical solutions of the various embodiments can be arbitrarily combined and used in combination.

[0075] In the several embodiments provided in the present application, it should be understood that the disclosed modules and methods can be implemented in other ways. For example, the system or device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the system or unit can be electrical, mechanical or other forms.

[0076] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0077] In addition, each functional unit / module in each embodiment of the present application may be integrated into one processing unit / module, or each unit / module may exist physically separately, or two or more units / modules may be integrated into one unit / module. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of hardware plus software functional units / modules.

[0078] The above-mentioned integrated unit / module implemented in the form of a software functional unit / module can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for enabling one or more processors of a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the methods of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for repairing shadows of display screen images, It is characterized in that include: Acquire a current display image of the display screen, and perform color space conversion on the current display image to obtain a first display image; Performing global shadow restoration on the first display image through a global restoration network to obtain a second display image; Performing a first round of local shadow repair on the second display image by using a first local correction network to obtain a third display image; A second round of local shadow repair is performed on the third display image through a second local correction network to obtain a target display image.

2. The display screen image shadow repair method according to claim 1, It is characterized in that The global restoration network includes an encoding module, a dilated convolution module, and a decoding module. The global shadow restoration of the first display image is performed by the global restoration network to obtain a second display image, including: Performing encoding processing on the first display image by the encoding module to obtain a first feature image; Performing dilated convolution processing on the first feature image through the dilated convolution module to obtain a second feature image; The second feature image is decoded by the decoding module to obtain the second display image.

3. The display screen image shadow repair method according to claim 2, It is characterized in that The encoding module includes a first convolution layer, N downsampling layers, and N encoding blocks. The encoding module encodes the first display image to obtain a first feature image, including: Processing the first display image through the first convolution layer to obtain a first convolution image; Performing encoding processing on the first convolution image through a first encoding block to generate a first encoded image, and performing downsampling processing on the first encoded image through a first downsampling layer corresponding to the first encoding block to obtain a first downsampled image; encoding the first downsampled image through a second encoding block to generate a second encoded image, and downsampling the second encoded image through a second downsampled layer corresponding to the second encoding block to obtain a second downsampled image; Until the N-1th downsampled image is encoded by the Nth encoding block to generate the Nth encoded image, and the Nth encoded image is downsampled by the Nth downsampling layer corresponding to the Nth encoding block to obtain the first feature image.

4. The display screen image shadow repair method according to claim 3, It is characterized in that The step of performing dilated convolution processing on the first feature image by the dilated convolution module to obtain a second feature image includes: Get the number of dilated convolution layers in the dilated convolution module and the dilation of each dilated convolution layer. Exhibition rate; Convolving the first feature image according to the number of the dilated convolutional layers and the dilation rate Dilate to obtain the second feature image.

5. The display screen image shadow restoration method according to claim 3, wherein the decoding module comprises N upsampling layers, N decoding blocks and a second convolution layer, and the second feature image is decoded by the decoding module to obtain the second display image include: Performing upsampling processing on the second feature image through a first upsampling layer to obtain a first upsampling image, and performing decoding processing on the first upsampling image through a first decoding block corresponding to the first upsampling layer to obtain a first decoded image; Performing upsampling processing on the first decoded image through a second upsampling layer to obtain a second upsampling image, and performing decoding processing on the second upsampling image through a second decoding block corresponding to the second upsampling layer to obtain a second decoded image; Until the N-1th decoded image is upsampled by the Nth upsampling layer to obtain the Nth upsampled image, and the Nth upsampled image is decoded by the Nth decoding block corresponding to the Nth upsampling layer to obtain the Nth decoded image; The Nth decoded image is convolved by the second convolution layer to obtain the second display image.

6. The display screen image shadow repair method according to any one of claims 1 to 5, It is characterized in that The first local correction network includes the first local encoding module, the first local decoding module and the first local deconvolution layer, and performing a first round of local shadow repair on the second display image through the first local correction network to obtain a third display image includes: Acquire a first residual data set according to the first display image and the second display image; Filter out first target residual values ​​whose residual values ​​are less than a first preset residual threshold from the first residual data set to obtain a first target residual value set; Performing encoding processing on the first target residual value set by the first local encoding module to obtain a first local encoded image set; Decoding the first local coded image set by the first local decoding module to obtain a first local decoded image set; The first local decoded image set is deconvolved by the first local deconvolution layer to obtain the third display image.

7. The display screen image shadow repair method according to claim 6, It is characterized in that The second local correction network includes the second local encoding module, the second local decoding module and the second local deconvolution layer, and the second local shadow repair is performed on the third display image by the second local correction network to obtain the target display image, including: Acquire a second residual data set according to the first display image and the third display image; Filter out second target residual values ​​whose residual values ​​are less than a second preset residual threshold from the second residual data set to obtain a second target residual value set; Performing encoding processing on the second target residual value set by the second local encoding module to obtain a second local encoded image set; Decoding the second local coded image set by the second local decoding module to obtain a second local decoded image set; Performing deconvolution processing on the second local decoded image set through the second local deconvolution layer to obtain the target display image; The second preset residual threshold is smaller than the first preset residual threshold.

8. A display screen image shadow repair device, It is characterized in that The device comprises: An image conversion module, used for acquiring a current display image of the display screen, and performing color space conversion on the current display image to obtain a first display image; A first restoration module, configured to perform global shadow restoration on the first display image through a global restoration network to obtain a second display image; A second restoration module, configured to perform a first round of local shadow restoration on the second display image through a first local correction network to obtain a third display image; The third restoration module is used to perform a second round of local shadow restoration on the third display image through a second local correction network to obtain a target display image.

9. A display device, It is characterized in that The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the display screen image shadow repair method as claimed in any one of claims 1 to 7 when executing the computer program.

10. A storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method for implementing the shadow restoration of a display screen image according to any one of claims 1 to 7 are implemented.

Images