Image processing method, terminal, storage medium and computer program product

Abstract

The invention discloses an image processing method, a terminal, a storage medium and a computer program product, and the method comprises the steps that a first terminal receives a first game picture frame sent by a server side of a first game; performing deblurring processing on one or more first image areas in the first game picture frame to obtain a second game picture frame; wherein the image content of the first image area comprises set game elements and / or a first parameter of the first image area is smaller than a first set threshold value; the first parameter is used for describing the fuzzy degree of the corresponding image area; and outputting the second game picture frame.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to an image processing method, a terminal, a storage medium and a computer program product. BACKGROUND

[0002] In the related art, a game server sends a game picture frame with low resolution to a terminal in the case of poor network quality to ensure the fluency of the game. However, in this case, the game picture rendered by the terminal based on the game picture frame is relatively blurred. SUMMARY

[0003] To solve the problems in the related art, the embodiments of the present application provide an image processing method, a terminal, a storage medium and a computer program product.

[0004] The technical scheme of the embodiments of the present application is implemented as follows:

[0005] The embodiments of the present application provide an image processing method applied to a first terminal, and the method comprises the following steps.

[0006] Receiving a first game picture frame sent by a server of a first game;

[0007] Performing deblurring processing on one or more first image regions in the first game picture frame to obtain a second game picture frame; wherein the image content of the first image region comprises a set game element and / or a first parameter of the first image region is less than a first set threshold; and the first parameter is used to describe the blurring degree of the corresponding image region.

[0008] Outputting the second game picture frame.

[0009] In the above scheme, the set game element is determined based on one or more of the following: the game type of the first game and the game scene corresponding to the first game picture frame.

[0010] In the above scheme, the first parameter is determined based on the standard deviation of the gray values of a plurality of pixels in the corresponding image region.

[0011] In the above scheme, the deblurring processing on the one or more first image regions in the first game picture frame comprises:

[0012] In the case that the first game picture frame meets a set condition, the one or more first image regions are determined from all or part of the image regions in the first game picture frame, and each first image region in the one or more first image regions is deblurred.

[0013] In the above scheme, the deblurring processing on the first image region comprises:

[0014] calling the first model to process the first image region;

[0015] The first model is used for deblurring processing of an input image, and the first model is obtained by adding a first attention block in a generator of a generative adversarial network model; the first attention block is used for enhancing processing of image features related to the set game element in the input image based on a channel attention mechanism.

[0016] In the above solution, a first model parameter set used by the first model is determined from one or more model parameter sets pre-trained based on the set game element in the input first image region.

[0017] In the above solution, a convolution module in the first model is used for performing a depth separable convolution operation.

[0018] Embodiments of the present application also provide a first terminal, comprising a processor and a memory for storing a computer program capable of running on the processor,

[0019] The processor is configured to execute the computer program to perform the steps of any of the above methods.

[0020] Embodiments of the present application also provide a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above methods.

[0021] Embodiments of the present application also provide a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above methods.

[0022] In embodiments of the present application, the first terminal receives a first game screen frame sent by a service end of a first game; then, the first terminal performs deblurring processing on one or more first image regions in the first game screen frame to obtain a second game screen frame, wherein the image content of the first image region includes a set game element and / or a first parameter of the first image region is less than a first set threshold, and the first parameter is used to describe the blurring degree of the corresponding image region; then, the first terminal outputs the second game screen frame. In the above solution, the first terminal performs deblurring processing on the first image region in the first game screen frame after receiving the first game screen frame sent by the service end, that is, deblurring processing is performed on a specific image region in the first game screen frame, so that a better deblurring effect can be achieved, and the second game screen frame obtained by processing is clearer than the first game screen frame. On this basis, a clear game screen can be rendered by outputting the second game screen frame, and the blurring degree of the rendered game screen is reduced compared with related art. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 An implementation flowchart of an image processing method provided for an embodiment of the present application is shown in the figure;

[0024] Figure 2 A processing diagram of a first attention block provided for an embodiment of the present application is shown in the figure;

[0025] Figure 3 A processing diagram of a convolution operation provided for an embodiment of the present application is shown in the figure;

[0026] Figure 4 A flowchart of an image processing method provided for an embodiment of the present application is shown in the figure;

[0027] Figure 5 A flowchart of another image processing method provided for an embodiment of the present application is shown in the figure;

[0028] Figure 6 A structural diagram of an image processing device provided for an embodiment of the present application is shown in the figure;

[0029] Figure 7 A hardware composition structural diagram of a first terminal provided for an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0030] In the related art, a game server transmits game picture frames to a terminal through a network, and in the case of poor network quality, transmits game picture frames with lower resolution to the terminal to ensure the smoothness of the game. However, in this case, the game picture rendered by the terminal based on the game picture frames is relatively blurred, which affects the user's game experience.

[0031] Based on this, in an embodiment of the present application, a first terminal receives a first game picture frame sent by a server of a first game; then, the first terminal performs deblurring processing on one or more first image regions in the first game picture frame to obtain a second game picture frame, wherein the image content of the first image region includes a set game element and / or a first parameter of the first image region is less than a first set threshold, and the first parameter is used to describe the blurring degree of the corresponding image region; then, the first terminal outputs the second game picture frame. In the above scheme, the first terminal performs deblurring processing on the first image region in the first game picture frame after receiving the first game picture frame sent by the server, that is, deblurring processing is performed on the specific image region in the first game picture frame. In this way, a better deblurring effect can be achieved, so that the second game picture frame obtained by processing is clearer than the first game picture frame. On this basis, a clear game picture can be rendered by outputting the second game picture frame, which reduces the blurring degree of the rendered game picture compared with the related art.

[0032] The application will be described in further detail below with reference to the drawings and embodiments.

[0033] An image processing method is provided by the embodiments of the application, which is applied to a first terminal, and the method is described by reference to Figure 1 The method comprises the following steps.

[0034] Step 101: receiving a first game picture frame sent by a server of a first game.

[0035] In actual application, the first game can include cloud game, and the server of the first game can be deployed in the cloud. A game user can play the first game through a user interface of the first terminal.

[0036] The first game picture frame can be understood as a game picture frame of the first game, and one game picture frame is equivalent to one image for rendering a game picture.

[0037] The server of the first game can send the first game picture frame to the first terminal through a wireless network and / or a wired network during the running of the first game, and adjust the resolution of the first game picture frame to be sent based on the fluctuation of network quality before each sending. Adjusting the resolution of the game picture frame can also be understood as adjusting the quality of the game picture frame. The resolution of the game picture frame is positively correlated with the quality of the game picture frame, and the quality of the game picture frame can also be understood as the clarity of the game picture frame.

[0038] Exemplarily, in the case that the maximum resolution that the first game picture frame can reach is 1080p and the network quality is poor, the server of the first game can adjust the resolution of the first game picture frame to be sent to 480p, which is equivalent to reducing the quality of the first game picture frame to be sent, so as to reduce the data transmission amount corresponding to the first game picture frame, thereby ensuring the fluency of the first game.

[0039] In actual application, after receiving the first game picture frame, the first terminal can perform deblurring processing on the first game picture frame to improve the quality of the first game picture frame. The deblurring processing manner of the first terminal on the first game picture frame can be referred to the step 102 shown below.

[0040] Step 102: performing deblurring processing on one or more first image regions in the first game picture frame to obtain a second game picture frame.

[0041] The image content of the first image region includes a set game element and / or a first parameter of the first image region is less than a first set threshold; the first parameter is used to describe the blurring degree of the corresponding image region.

[0042] In practical applications, the first terminal can divide the first game picture frame into a plurality of image regions, and further, the first terminal can divide the first game picture frame into a plurality of image regions with a set size. Exemplarily, the set size can include 64x64, i.e., the length and width of each image region are 64.

[0043] For one or more first image regions in the first game picture frame, the first terminal can perform deblurring processing on the first image regions, and in the process of performing deblurring processing on the first image regions, the first terminal can perform deblurring processing on each of the first image regions. The first image region can be understood as a specific image region in the first game picture frame.

[0044] Here, the image content of the first image region includes a set game element, or the first parameter of the first image region is less than a first set threshold, or the image content of the first image region includes a set game element and the first parameter of the first image region is less than a first set threshold.

[0045] In practical applications, reducing the blurring degree of the set game element can greatly increase the game experience of the game user, and the set game element can also be referred to as a key game element. The set game element can include a game element that the game user pays attention to.

[0046] In an embodiment, the set game element is determined based on one or more of the following: the game type of the first game, the game scene corresponding to the first game picture frame.

[0047] Exemplarily, in the case where the game type of the first game is a battle, the game user usually pays attention to the character image in the battle, and the character can be determined as the set game element; in the case where the game type of the first game is world exploration, the game user usually pays attention to the scenery, and the element for displaying the scenery can be determined as the set game element.

[0048] Exemplarily, in the case where the game scene corresponding to the first game picture frame is performing a collection task, the target collection object corresponding to the collection task can be determined as the set game element.

[0049] In practical applications, the set game element can also be determined based on human definition, and exemplarily, the game element defined by the developer, such as a face, a specific object, etc., can be determined as the set game element.

[0050] Here, the first parameter is used to describe the blurring degree of the corresponding image region, and in practical applications, the first parameter can be understood as a blurring degree parameter of the corresponding image region. The first parameter can be negatively correlated with the blurring degree of the image region, i.e., the smaller the first parameter, the more blurred the image region.

[0051] In a case where the first parameter of the first image region is less than the first set threshold value, the first image region can be regarded as a relatively blurred image region.

[0052] In an embodiment, the first parameter is determined based on a standard deviation of the gray scale values of the plurality of pixels in the corresponding image region.

[0053] Exemplarily, the first parameter can be represented as:

[0054]

[0055] wherein s represents the first parameter, N x represents the number of pixels in the x-axis direction in the corresponding image region, N y represents the number of pixels in the y-axis direction in the corresponding image region, and f(x, y) represents the gray scale value of a pixel.

[0056] wherein, may be represented as:

[0057]

[0058] That is, may represent the average value of the gray scale values of the plurality of pixels in the corresponding image region.

[0059] In an embodiment, the deblurring processing on one or more first image regions in the first game screen frame comprises:

[0060] In a case where the first game screen frame satisfies a set condition, one or more first image regions are determined from all or part of the image regions in the first game screen frame, and the deblurring processing is performed on each of the one or more first image regions.

[0061] In actual application, in the process of determining the one or more first image regions from all or part of the image regions in the first game screen frame, the first terminal can determine whether to determine each image region as a first image region based on the first parameter of the image region and / or whether the image region includes the set game element.

[0062] In the process of determining the first image region, even if the first parameter of an image region is greater than the first set threshold value, if the image content of the image region includes the set game element, the image region can be determined as a first image region.

[0063] In actual application, the first terminal can not perform the deblurring processing on the first game picture frame in a case where the first game picture frame does not satisfy the set condition, that is, it is not necessary to determine one or more first image regions and perform the deblurring processing on each first image region. Exemplarily, the set condition can include that the first parameter of the first game picture frame is less than a second set threshold.

[0064] The deblurring algorithm in the related art usually processes the whole image when deblurring the image, and only has a good deblurring effect on specific noise, and has low applicability and precision. In the embodiment of the present application, when deblurring the first game picture frame, the specific image region in the first game picture frame is deblurred respectively, instead of processing the whole image, so that a good deblurring effect can be produced, and the second game picture frame obtained after deblurring greatly reduces the blurring degree compared with the first game picture frame.

[0065] Step 103: output the second game picture frame.

[0066] In actual application, the first terminal can output the second game picture frame to the user interface, so as to render a game picture in the user interface based on the second game picture frame. The clarity of the rendered game picture can be positively correlated with the clarity of the second game picture frame, that is, the clearer the second game picture frame is, the clearer the rendered game picture is.

[0067] In the embodiment of the present application, after the first terminal receives the first game picture frame sent by the server, the first image region in the first game picture frame is deblurred, that is, the specific image region in the first game picture frame is deblurred, so that a good deblurring effect can be produced, and the second game picture frame obtained after processing is clearer than the first game picture frame. On this basis, a clear game picture can be rendered by outputting the second game picture frame, and the blurring degree of the rendered game picture is reduced compared with the related art.

[0068] In actual application, the first terminal can output the second game picture frame in real time during the process of the first game, so as to ensure the fluency of the game picture and the real-time performance of the first game.

[0069] The manner of deblurring the first image region will be further described below.

[0070] In an embodiment, deblurring the first image region includes:

[0071] calling the first model to process the first image region;

[0072] The first model is used for deblurring the input image, and the first model is obtained by adding a first attention block in a generator of a generative adversarial network model.

[0073] In practical applications, the generative adversarial network model can include a generator and a discriminator. The generator can be understood as a generative model, and the discriminator can be understood as an adversarial model.

[0074] Here, the first model can be obtained by adding a first attention block in the generator of the generative adversarial network model, and the first attention block is processed based on the channel attention mechanism. Therefore, the first model can also be regarded as being obtained by adding the channel attention mechanism to the generator of the generative adversarial network model.

[0075] In practical applications, the generative adversarial network model used to obtain the first model can include a Deblur GAN (Deblur Generative Adversarial Network). That is, the first model can also be regarded as being obtained by adding the channel attention mechanism to the generator of the Deblur GAN.

[0076] In practical applications, referring to Figure 2 , the processing flow of the first attention block can include the following steps:

[0077] Step 1: performing convolution operation on the tensor X input into the first attention block to obtain a tensor U.

[0078] Wherein, the data size of the tensor X can be represented as H'xW'xC', the data size of the tensor U can be represented as HxWxC, C' can be understood as the channel number of the tensor X, C can be understood as the channel number of the tensor U, and the convolution operation can be represented as F tr .

[0079] In practical applications, the tensor X can be understood as a feature map of the input image of the first model.

[0080] Step 2: performing global average pooling processing on each channel of the tensor U to obtain a tensor P.

[0081] Wherein, the data size of the tensor P is 1x1xC, and the global average pooling processing can be represented as F sq .

[0082] In practical applications, the global average pooling processing can also be understood as a squeeze processing.

[0083] In practical applications, the importance of each channel in the input image of the first model can be counted by global average pooling processing on each channel.

[0084] The data of each channel in the tensor U can be understood as an image feature related to an image region in the input image of the first model, which can be related to a specific object or a specific element in the input image. An image region can also be understood as a spatial position or a spatial region.

[0085] The global average pooling processing on each channel of the tensor U can be understood as global average pooling processing on the image features related to each image region in the input image, thereby counting the importance of each image region in the input image.

[0086] The global average pooling processing can compress the feature dimension of the image feature related to an image region to 1x1, so that the image feature related to the image region is encoded in the channel dimension, thereby reflecting the activation degree of each channel in the corresponding image region of the input image.

[0087] For example, some channels in the tensor P can be sensitive to the edges of a specific image region in the input image, such as the edges of the upper left corner, while other channels in the tensor P can be sensitive to the texture of the central region.

[0088] For example, assuming that the input image of the first model represents an image containing a cat, and the size of the image is 224x224x3, on this basis, the number of channels of the tensor U can be 1024, that is, the data size of the tensor U is HxWx1024; the size of the tensor P obtained by global average pooling processing can be 1x1x1024. The value of each channel in the tensor P can correspond to a key region such as the ear or eye of the cat in the input image, and the value of each channel in the tensor P can also be called the activation value of the channel.

[0089] Step 3: Perform two-level full connection processing on the tensor P to obtain a tensor Q.

[0090] wherein the data size of the tensor Q is 1x1xC, and the two-level full connection processing can be represented as F ex .

[0091] In practical applications, the two-level full connection processing can also be understood as excitation processing.

[0092] Step 4: Normalize the values of each channel in the tensor Q, and multiply the normalized values with the data on the corresponding channel of the tensor U to obtain a tensor

[0093] In practical applications, the tensor The input data of the module after the first attention module, that is, the tensor The input data of the next level.

[0094] In actual application, the sigmoid function can be used to normalize the values of each channel in the tensor Q to the range of 0 to 1 in the process of normalizing the values of each channel.

[0095] The normalized value can be understood as an important factor (attention factor) used by the first attention module to define the features that need to be enhanced. That is, the first attention module can define which features are important features and which features are unimportant features in the processing process through the important factor, and then multiply the normalized value with the data on the corresponding channel in the tensor U, so as to provide more attention to important features and less attention to unimportant features in the processing process, that is, to enhance the processing of important features, so that the directionality of feature processing is stronger and the utilization of features is more sufficient.

[0096] In actual application, different model parameter sets can be used for the first model to control the definition of important features by the first attention module, for example, to control the first attention module to define important features as image features related to a set game element, so that the first attention module can enhance the processing of image features related to the set game element in the input image of the first model.

[0097] In an embodiment, the first model parameter set used by the first model is determined from one or more pre-trained model parameter sets based on the set game element in the input first image region.

[0098] In actual application, the first model that has not been trained can be trained in advance by one or more training sets to obtain one or more model parameter sets. Each training set can be used to train a model parameter set, and the model parameter set trained by each training set can be used to control the first attention module to define important features as image features related to a type of game element.

[0099] When the first model is called to process the first image region, the set game element in the first image region can be matched with the one or more pre-trained model parameter sets, and in the case that the set game element matches the game element corresponding to a model parameter set, the model parameter set is determined as the first model parameter set.

[0100] In actual application, the one or more sets of pre-trained model parameters can include a general set of model parameters, and the general set of model parameters can be determined as the first set of model parameters in the case that the first image region does not include the set game element.

[0101] In the embodiments of the present application, the first model for deblurring processing is obtained by adding a first attention block in the generator of the generative adversarial network model. On this basis, the first model can enhance the processing of image features related to the set game element in the first image region to be processed based on the channel attention mechanism. In this way, targeted processing can be performed to produce a better deblurring effect, and the first model can be applied to different first image regions, having strong applicability.

[0102] In actual application, the first model can include one or more convolution modules, which can be used for convolution operation.

[0103] In an embodiment, the convolution module in the first model is used for depthwise separable convolution operation.

[0104] In actual application, depthwise separable convolution can be understood as a factorized convolution operation, which can be decomposed into two convolution operations, i.e., depthwise convolution and pointwise convolution.

[0105] In related technologies, the convolution kernel in the traditional standard convolution is used in all input channels; while in the embodiments of the present application, the depthwise convolution can use different convolution kernels for each input channel, i.e., one convolution kernel corresponds to one input channel, therefore, the depthwise convolution can be understood as a depth-level convolution operation. The pointwise convolution can be understood as a standard convolution, but the size of the convolution kernel used is 1x1.

[0106] In actual application, referring to Figure 3 In the process of depthwise separable convolution operation, the depthwise convolution can be used to convolve different input channels respectively to obtain a first convolution result; and then the pointwise convolution can be used to combine the first convolution result.

[0107] In practical applications, the calculation effect of the depth separable convolution is close to that of the standard convolution, but the depth separable convolution can greatly reduce the calculation amount and the model parameter amount compared with the standard convolution.

[0108] Exemplarily, the convolution module in the first model can be represented as a lightweight convolution module (MobileNet Block), and the MobileNet Block can be used to perform a depth separable convolution operation.

[0109] Compared with the convolution module used to perform a standard convolution, the MobileNet Block can have the following advantages:

[0110] Advantage 1: The model parameter amount is greatly reduced. This is because the depthwise convolution only needs to perform convolution on each channel, instead of performing comprehensive convolution on all channels.

[0111] Advantage 2: The calculation amount is reduced. The convolution kernel size of the pointwise convolution is 1x1, so when performing the pointwise convolution, no convolution operation in the spatial dimension is needed, only linear combination in the channel dimension is needed.

[0112] Advantage 3: After reducing the calculation amount, the training and inference speed of the first model in which the MobileNet Block is located are both accelerated, thereby reducing the requirement of the first model on hardware resources in runtime, which can exemplarily include a graphics processing unit (GPU) and / or a tensor processing unit (TPU); in this way, the processing efficiency of the image can be increased.

[0113] As can be seen, in the embodiments of the present application, the convolution module used to perform a depth separable convolution operation is set for the first model, the calculation amount is reduced, the operation speed is improved, and thus the processing efficiency of the image is increased, and on this basis, the efficiency of the deblurring processing on the first image region can be increased.

[0114] The present application will be further described in detail below in combination with application embodiments.

[0115] The application embodiment of the present application provides a cloud game terminal algorithm model, which can be deployed in a terminal and used for image processing on a game picture frame received by the terminal. In practical applications, the cloud game terminal algorithm model can perform image processing based on the image processing method provided in the embodiments of the present application.

[0116] Referring to Figure 4, the image processing procedure based on the cloud game terminal algorithm model can include the following steps:

[0117] Step 1: The terminal receives a game picture frame sent by the server of the cloud game.

[0118] In actual application, the server of the cloud game can perform cloud game picture frame transmission. During the transmission process, the server can adjust the quality of the transmitted cloud game picture frame based on the fluctuation of network quality, and send the cloud game picture frame with adjusted quality to the terminal. One cloud game picture frame can also be understood as one frame of cloud game picture, and can also be understood as one image.

[0119] Step 2: The terminal performs deblurring processing on the received cloud game picture frame based on the cloud game terminal algorithm model to obtain a processing result.

[0120] In actual application, the deblurring processing manner of the cloud game terminal algorithm model on the cloud game picture frame can be understood with reference to the deblurring processing manner of the first game picture frame in the embodiments of the present application; the obtained processing result can be understood as a game picture frame after deblurring processing, that is, the processing result is equivalent to the second game picture frame in the embodiments of the present application.

[0121] In the process of deblurring processing of the game picture frame by the cloud game terminal algorithm model, in the case that the cloud game picture frame does not include key elements, the whole deblurring can be performed. This deblurring manner can be understood as a whole general scheme.

[0122] In the process of deblurring processing of the game picture frame by the cloud game terminal algorithm model, in the case that the cloud game picture frame includes key elements, the processing of image features related to the key elements can be enhanced. For example, in the case that the cloud game is a battle type game, the character image changes frequently in different game picture frames, and the background other than the character changes little. The game user often focuses on the character, and on this basis, the processing of image features related to the character can be enhanced in the deblurring processing process. Here, the key element can be understood as a game element related to the set game element or important feature in the embodiments of the present application.

[0123] Step 3: Output the processing result.

[0124] In actual application, the terminal can output the obtained processing result to the user interface to render a game picture that is clearer than the received game picture frame, so as to display the game picture to the game user.

[0125] For example, Figure 5 For example, Figure 4The image processing flow shown is supplemented with a schematic of the game picture frame involved. Among them, the image picture frame 1 is equivalent to the image picture frame received by the terminal. In the process of deblurring processing by the cloud game terminal algorithm model, the image area where the key elements in the image picture frame 1 are located can be processed, and the processing of image features related to key elements is enhanced to obtain the image picture frame 2. The image picture frame 2 is clearer than the image picture frame 1. Figure 5 The image picture frame 1' in the figure is the image picture frame 1. Here, for ease of understanding, the image area where the key elements are located is framed in the image picture frame 1'.

[0126] In the application embodiment, after the terminal receives the game picture frame sent by the cloud game server, the specific image area in the game picture frame is deblurred. In this way, a better deblurring effect can be achieved, so that the game picture frame obtained by deblurring is clearer than the received game picture frame. On this basis, a clear game picture is rendered by outputting the processing result, which reduces the blurring degree of the rendered game picture compared with related technologies.

[0127] Based on the above embodiment, the application embodiment further provides an image processing device applied to a first terminal, referring to Figure 6 The image processing device comprises:

[0128] The receiving unit 61 is configured to receive a first game picture frame sent by a service end of a first game;

[0129] The processing unit 62 is configured to perform deblurring processing on one or more first image areas in the first game picture frame to obtain a second game picture frame. The image content of the first image area includes a set game element and / or a first parameter of the first image area is less than a first set threshold. The first parameter is used to describe the blurring degree of the corresponding image area.

[0130] The output unit 63 is configured to output the second game picture frame.

[0131] In an embodiment, the set game element is determined based on one or more of the following: the game type of the first game, and the game scene corresponding to the first game picture frame.

[0132] In an embodiment, the first parameter is determined based on the standard deviation of the gray values of a plurality of pixels in the corresponding image area.

[0133] In an embodiment, the processing unit 62 performs deblurring processing on one or more first image areas in the first game picture frame, comprising:

[0134] In a case where the first game picture frame meets a set condition, one or more first image regions are determined from all or part of image regions in the first game picture frame, and each of the one or more first image regions is subjected to a deblurring process.

[0135] In an embodiment, the processing unit 62 performs the deblurring process on the first image region, including:

[0136] calling a first model to process the first image region;

[0137] The first model is used to perform the deblurring process on an input image, and the first model is obtained by adding a first attention block in a generator of a generative adversarial network model. The first attention block is used to enhance the processing of image features related to the set game element in the input image based on a channel attention mechanism.

[0138] In an embodiment, a first set of model parameters used by the first model is determined from one or more sets of pre-trained model parameters based on the set game element in the input first image region.

[0139] In an embodiment, the convolution module in the first model is used to perform a depthwise separable convolution operation.

[0140] In actual application, the receiving unit 61, the processing unit 62, and the output unit 63 can be implemented by a processor in an image processing device.

[0141] It should be noted that the image processing device provided in the above embodiments is only used as an example to illustrate the division of the above program modules. In actual application, the above processing can be completed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the image processing device and the image processing method provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0142] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of the present application, the present application further provides a first terminal, which is shown in Figure 7 The first terminal includes:

[0143] A communication interface 1 capable of information interaction with other devices;

[0144] A processor 2 connected with the communication interface 1 to realize information interaction with other devices, used to run a computer program to execute the method provided in one or more technical solutions in the above embodiments. The computer program is stored on a memory 3.

[0145] Specifically, the processor 2 is configured to receive a first game frame sent by a server of a first game;

[0146] perform deblurring processing on one or more first image regions in the first game frame to obtain a second game frame; wherein the image content of the first image region includes a set game element and / or a first parameter of the first image region is less than a first set threshold; the first parameter is used to describe the blurring degree of the corresponding image region; and

[0147] output the second game frame.

[0148] In an embodiment, the set game element is determined based on one or more of the following: the game type of the first game, the game scene corresponding to the first game frame.

[0149] In an embodiment, the first parameter is determined based on the standard deviation of the gray values of a plurality of pixels in the corresponding image region.

[0150] In an embodiment, the processor 2 performs deblurring processing on one or more first image regions in the first game frame, including:

[0151] In the case where the first game frame meets a set condition, the one or more first image regions are determined from all or part of the image regions in the first game frame, and each of the one or more first image regions is deblurred.

[0152] In an embodiment, the processor 2 performs deblurring processing on the first image region, including:

[0153] calling a first model to process the first image region;

[0154] wherein the first model is used to perform deblurring processing on the input image, and the first model is obtained by adding a first attention block in the generator of the generative adversarial network model; the first attention block is used to enhance the processing of the image features related to the set game element in the input image based on the channel attention mechanism.

[0155] In an embodiment, the first model parameter set used by the first model is determined from one or more model parameter sets pre-trained based on the set game element in the input first image region.

[0156] In an embodiment, the convolution module in the first model is used to perform depth separable convolution operation.

[0157] It should be noted that the specific processing procedure of the communication interface 1 can be understood with reference to the above method.

[0158] Of course, in actual application, each component in the first terminal is coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection communication between the components. The bus system 4 includes not only a data bus, but also a power bus, a control bus and a state signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 4 in the following description. Figure 7

[0159] The memory 3 in the embodiment of the present application is used to store various types of data to support the operation in the first terminal. Examples of the data include any computer program used for operating on the first terminal.

[0160] The method disclosed in the above embodiment of the present application can be applied to or implemented by the processor 2. The processor 2 can be an integrated circuit chip with the processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 2 or the instruction in the form of software. The processor 2 disclosed above can be a general processor, DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiments of the present application, the hardware decoding processor can be directly embodied to execute the above steps, or the hardware and software modules in the decoding processor can be combined to execute the above steps. The software module can be located in the storage medium, which is located in the memory 3. The processor 2 reads the information in the memory 3 and combines the hardware to complete the steps of the above method.

[0161] In the exemplary embodiment, the first terminal can be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general processors, controllers, MCUs, Microprocessors, or other electronic elements, which are used to execute the above method.

[0162] ​It can be understood that the memory 3 of the embodiments of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM). The magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), sync link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory described in the embodiments of the present application is intended to include, but not limited to, these and any other suitable types of memory.

[0163] In the exemplary embodiments, the embodiments of the present application also provide a storage medium, i.e., a computer storage medium, specifically a computer readable storage medium, such as the memory 3 storing the computer program executable by the processor 2 of the first terminal to complete the steps of the aforementioned image processing method.

[0164] The computer readable storage medium can be a memory of FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0165] In the exemplary embodiments, the embodiments of the present application also provide a computer program product comprising a computer program executable by the processor 2 of the first terminal to complete the steps of the aforementioned image processing method.

[0166] It should be noted that "first", "second", etc. are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0167] The term "and / or" herein is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the term "one or more" herein means any combination of at least two of any one or more of a plurality, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0168] In addition, the technical solutions described in the embodiments of the present application can be arbitrarily combined without conflict.

[0169] The above is only a preferred embodiment of the present application, and is not intended to limit the protection scope of the present application.

Claims

1. An image processing method, characterized in that, Applied to a first terminal, the method includes: Receive the first game frame sent by the server of the first game; One or more first image regions within the first game frame are deblurred to obtain a second game frame; wherein the image content of the first image region includes set game elements and / or a first parameter of the first image region is less than a first set threshold; the first parameter is used to describe the blur degree of the corresponding image region. Output the second game frame.

2. The method according to claim 1, characterized in that, The game elements are determined based on one or more of the following: the game type of the first game, and the game scene corresponding to the first game frame.

3. The method according to claim 1, characterized in that, The first parameter is determined based on the standard deviation of the gray values ​​of multiple pixels in the corresponding image region.

4. The method according to claim 1, characterized in that, The process of deblurring one or more first image regions within the first game frame includes: When the first game frame meets the set conditions, one or more first image regions are determined from all or part of the image regions within the first game frame, and each of the one or more first image regions is deblurred.

5. The method according to claim 1, characterized in that, The first image region is deblurred, including: The first model is invoked to process the first image region; The first model is used to deblur the input image, and the first model is obtained by adding a first attention block to the generator of the generative adversarial network model; the first attention block is used to enhance the processing of image features related to the set game elements in the input image based on the channel attention mechanism.

6. The method according to claim 5, characterized in that, The first model uses a first set of model parameters determined from one or more pre-trained sets of model parameters based on the set game elements in the input first image region.

7. The method according to claim 5, characterized in that, The convolution module in the first model is used to perform depthwise separable convolution operations.

8. A first terminal, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images