Image quality enhancement network training method and device

Abstract

The embodiment of the invention provides an image quality enhancement network training method and device. The method comprises the following steps: determining an image quality loss type of a video image and a loss intensity weight corresponding to the image quality loss type; and generating a corresponding image quality enhancement network according to the image quality loss type and training the image quality enhancement network until an objective function curve of an objective function weighted by the loss intensity weight converges, and outputting a training result of the converged image quality enhancement network weighted by the loss intensity weight. According to the invention, the problem that the training result output after the image quality enhancement network is trained is not associated with the image quality loss type of the input video image and the corresponding loss intensity weight is solved; therefore, the effect that the trained image quality enhancement network can process input video images of various image quality loss types more effectively is achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of communications, and in particular, to a method and device for training a network for image quality enhancement. Background Art

[0002] With the continuous development of Internet technology and the continuous improvement of display device performance, users' requirements for video image quality are also increasing day by day. In order to meet the needs of consumer groups for video image quality, image quality enhancement has gradually become the focus of attention in the current video image processing field. Usually, the displayed video image is enhanced to achieve the purpose of optimizing video content, improving picture quality and enriching video information.

[0003] At present, there are some techniques in image quality enhancement related to using traditional video image algorithm processing methods to enhance the image quality of a single loss, or cascading multiple traditional algorithms to restore multiple losses (for example, noise, blur, rain and fog effects, block effects, etc.) at the same time. However, the algorithms before and after this enhancement method affect each other, and the coupling is extremely high. Considering the diversity of loss scenarios, it takes a long time to determine the optimal parameters of each algorithm module while ensuring the accuracy of image quality enhancement. With the increasing popularity of deep learning, based on the existing rich loss data sets, more deep learning methods are used to improve the overall video image quality. However, when designing deep learning objective functions, the mean absolute error of network output and target output is directly calculated, or the combined weights of multiple objective function sub-items are given based on empirical experiments. For problem scenarios where multiple losses exist at the same time, the coupling strength of multiple loss types is not considered theoretically. Therefore, the training results output by the image quality enhancement network after the video image quality enhancement training are not related to the image quality loss type and the corresponding loss intensity weight of the input video image, and the degradation caused by multiple losses cannot be restored simultaneously and effectively. Summary of the invention

[0004] An embodiment of the present invention provides a method and device for training a picture quality enhancement network, so as to at least solve the problem in the related art that the training results output by the picture quality enhancement network after performing picture quality enhancement training on a video image are not correlated with the picture quality loss type of the input video image and the corresponding loss intensity weight.

[0005] According to one embodiment of the present invention, a method for training a picture quality enhancement network is provided, comprising: determining a picture quality loss type of a video image and a loss intensity weight corresponding to the picture quality loss type; generating a corresponding picture quality enhancement network according to the picture quality loss type and training the picture quality enhancement network until an objective function curve of an objective function weighted by the loss intensity weight converges, and outputting a converged result of the picture quality enhancement network training weighted by the loss intensity weight.

[0006] In an exemplary embodiment, the image quality enhancement training is performed on the video image according to the image quality loss type until the objective function curve of the objective function weighted by the loss intensity weight converges, and the converged image quality enhancement network training result weighted by the loss intensity weight is output, including: performing the image quality enhancement training on the video image through the image quality enhancement network to obtain the image quality enhancement network training result weighted by the loss intensity weight, wherein the image quality enhancement network includes a plurality of image quality enhancement sub-networks, and the image quality enhancement sub-networks correspond to the image quality loss type; The objective function weighted by the loss intensity weight is calculated according to the image quality loss type and the loss intensity weight corresponding to the image quality loss type, the result of the image quality enhancement sub-network training and the target video image; it is determined whether the objective function curve generated according to the objective function converges. When the objective function curve converges, the image quality enhancement training is terminated, and the converged result of the image quality enhancement network training weighted by the loss intensity weight is output. When the objective function curve does not converge, the image quality enhancement training of the video image continues to be performed through the image quality enhancement network.

[0007] According to another embodiment of the present invention, there is provided a device for training a picture quality enhancement network, comprising: a determination module for determining a picture quality loss type of a video image and a loss intensity weight corresponding to the picture quality loss type; an output module for generating a corresponding picture quality enhancement network according to the picture quality loss type and training the picture quality enhancement network until an objective function curve of an objective function weighted by the loss intensity weight converges, and outputting the converged result of the picture quality enhancement network training weighted by the loss intensity weight.

[0008] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0009] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0010] Through the present invention, since corresponding image quality enhancement networks can be generated and trained for different image quality loss types based on the determination of the image quality loss type, the objective function can be weighted based on the determination of the loss intensity weight corresponding to the image quality loss type, and then when the objective function curve generated by the weighted objective function converges, the converged result of the image quality enhancement network training weighted by the loss intensity weight is output, and the objective function of the image quality enhancement network and the output training result are both related to the image quality loss intensity weight. Therefore, the problem that the training result output after the image quality enhancement network is generated according to the image quality loss type and trained has no correlation with the image quality loss type of the input video image and the corresponding loss intensity weight can be solved, and the effect that the trained image quality enhancement network can more effectively process input video images of various image quality loss types is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a hardware structure block diagram of a computer terminal according to an image quality enhancement network training method according to an embodiment of the present invention;

[0012] Figure 2 is a flow chart of a method for training a network for image quality enhancement according to an embodiment of the present invention;

[0013] Figure 3 is a structural block diagram of a network training device for image quality enhancement according to an embodiment of the present invention;

[0014] Figure 4 is a structural block diagram of a network training device for image quality enhancement according to yet another embodiment of the present invention;

[0015] Figure 5 is a structural schematic diagram of a network training device for image quality enhancement according to another embodiment of the present invention;

[0016] Figure 6 is a structural schematic diagram of a network training device for image quality enhancement according to another embodiment of the present invention;

[0017] Figure 7 is a flow chart of a method for training a network for image quality enhancement according to yet another embodiment of the present invention;

[0018] Figure 8 is a schematic diagram of an image quality assessment module according to an embodiment of the present invention;

[0019] Fig. 9 is a schematic diagram of a picture quality enhancement network according to an embodiment of the present invention;

[0020] Fig.10 FIG. 4 is a schematic diagram of a picture quality enhancement network according to another embodiment of the present invention. DETAILED DESCRIPTION

[0021] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0023] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal for a network training method for image quality enhancement according to an embodiment of the present invention. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0024] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the image quality enhancement network training method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0025] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0026] Figure 2 is a flow chart of a method for training a network for image quality enhancement according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0027] Step S202, determining the image quality loss type of the video image and the loss intensity weight corresponding to the image quality loss type;

[0028] In step S202 of this embodiment, it includes: performing image quality evaluation on the video image based on the target video image to determine the image quality loss type of the video image and the loss intensity weight corresponding to the image quality loss type.

[0029] In an exemplary embodiment, determining the loss intensity weight corresponding to the image quality loss type of the video image includes: performing index calculation on the video image to generate a first index corresponding to the image quality loss type; judging whether the first index is less than a preset threshold value, if the first index is greater than or equal to the preset threshold value, the loss intensity weight corresponding to the image quality loss type is 0; if the first index is less than the preset threshold value, judging whether the first index is within a preset range; if the first index is not within the preset range, compressing the first index to generate a second index; if the first index is within the preset range, the second index is equal to the first index; inverting the second index to generate a third index, normalizing the third index to obtain the loss intensity weight.

[0030] By evaluating the image quality of the video image, the image quality loss type of the video image can be obtained to guide the subsequent design of the objective function, and the loss intensity weight corresponding to the obtained image quality loss type is used to weight the objective function and the results of the image quality enhancement network training.

[0031] Before step S202 of this embodiment, the method further includes: setting the image quality loss type to be evaluated; wherein the image quality loss type includes: noise, blur, rain and fog effect, block effect and color distortion.

[0032] Step S204, generates a corresponding image quality enhancement network according to the image quality loss type and trains the image quality enhancement network until the objective function curve of the objective function weighted by the loss intensity weight converges, and outputs the converged result of the image quality enhancement network training weighted by the loss intensity weight.

[0033] In step S204 of the present embodiment, the image quality enhancement training is performed on the video image according to the image quality loss type until the objective function curve of the objective function weighted by the loss intensity weight converges, and the converged image quality enhancement network training result weighted by the loss intensity weight is output, including: performing the image quality enhancement training on the video image through the image quality enhancement network to obtain the image quality enhancement network training result weighted by the loss intensity weight, wherein the image quality enhancement network includes a plurality of image quality enhancement sub-networks, and the image quality enhancement sub-networks correspond to the image quality loss type. ; Calculate the objective function weighted by the loss intensity weight according to the image quality loss type and the loss intensity weight corresponding to the image quality loss type, the result of the image quality enhancement sub-network training and the target video image; Determine whether the objective function curve generated according to the objective function converges. When the objective function curve converges, end the image quality enhancement training and output the converged result of the image quality enhancement network training weighted by the loss intensity weight. When the objective function curve does not converge, continue to perform the image quality enhancement training on the video image through the image quality enhancement network.

[0034] The image quality enhancement sub-network corresponding to the image quality loss type trains different loss types of video images respectively. Each loss type corresponds to a sub-network. The objective function is calculated based on the image quality loss type and the loss intensity weight corresponding to the image quality loss type, the training results of the image quality enhancement sub-network and the target video image. Since the entire image quality enhancement network training is constrained by the objective function, whether the curve generated by the objective function converges determines whether to end the image quality enhancement training. The training results output by the image quality enhancement network are also related to the loss intensity. Therefore, the feedback mechanism of the training is associated with the input loss type and loss intensity.

[0035] In an exemplary embodiment, the video image is trained for image quality enhancement through an image quality enhancement network, and the objective function weighted by the loss intensity weight is calculated according to the image quality loss type and the loss intensity weight corresponding to the image quality loss type, the result of the image quality enhancement sub-network training and the target video image, including: determining the objective sub-function loss according to the result of the image quality enhancement sub-network output corresponding to the image quality loss type i relative to the loss of the target video imagei ; According to the loss intensity weight a = [a1, a2, ..., an], the target sub-function loss i Weighted objective function loss = a1loss1+a2loss2+...+a n loss n , where i=1,2,...,n.

[0036] In an exemplary embodiment, obtaining the result of the image quality enhancement network training weighted by the loss intensity weight includes: calculating the result of the image quality enhancement network training weighted by the loss intensity weight out=a1out1+a2out2+...+a n out n ; Among them, out i For the output of each image quality enhancement sub-network, a i For the out i The corresponding loss intensity weight.

[0037] After step S204 of the present embodiment, the method further includes: evaluating the image quality enhancement effect of the output converged result of the image quality enhancement network training; when the image quality enhancement effect of the converged result of the image quality enhancement network training meets the preset standard, saving the parameters of the image quality enhancement network and ending the training of the image quality enhancement network; when the image quality enhancement effect of the converged result of the image quality enhancement network training does not meet the preset standard, continuing to train the image quality enhancement network.

[0038] In this embodiment, a network training device for image quality enhancement is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0039] Figure 3 is a structural block diagram of a network training device for image quality enhancement according to an embodiment of the present invention. Figure 3 As shown, the image quality enhancement network training device 300 includes: a determination module 10 and an output module 20.

[0040] A determination module 10, configured to determine a quality loss type of a video image and a loss intensity weight corresponding to the quality loss type;

[0041] The output module 20 is used to generate a corresponding image quality enhancement network according to the image quality loss type and train the image quality enhancement network until the objective function curve of the objective function weighted by the loss intensity weight converges, and output the converged result of the image quality enhancement network training weighted by the loss intensity weight.

[0042] Figure 4 is a structural block diagram of a network training device for image quality enhancement according to another embodiment of the present invention. Figure 4 As shown, the image quality enhancement network training device 400 includes Figure 3 In addition to all the modules shown, the output module 20 further includes: a first training unit 21 , a calculation unit 22 , and a second training unit 23 .

[0043] A first training unit 21 is used to perform the image quality enhancement training on the video image through an image quality enhancement network to obtain a result of the image quality enhancement network training weighted by the loss intensity weight, wherein the image quality enhancement network includes a plurality of image quality enhancement sub-networks, and the image quality enhancement sub-networks correspond to the image quality loss types;

[0044] A calculation unit 22, configured to calculate the objective function weighted by the loss intensity weight according to the image quality loss type and the loss intensity weight corresponding to the image quality loss type, the result of the image quality enhancement sub-network training and the target video image;

[0045] The second training unit 23 is used to determine whether the objective function curve generated according to the objective function converges. When the objective function curve converges, the image quality enhancement training is terminated, and the converged result of the image quality enhancement network training weighted by the loss intensity weight is output. When the objective function curve does not converge, the image quality enhancement training of the video image continues to be performed through the image quality enhancement network.

[0046] Figure 5 is a structural schematic diagram of a network training device for image quality enhancement according to another embodiment of the present invention. Figure 5 As shown, the image quality enhancement network training device 500 includes Figure 3 In addition to all the modules shown, the determination module 10 also includes: a determination unit 11, which is used to perform image quality evaluation on the video image based on the target video image to determine the image quality loss type of the video image and the loss intensity weight corresponding to the image quality loss type.

[0047] Figure 6 is a structural schematic diagram of a network training device for image quality enhancement according to another embodiment of the present invention. Figure 6 As shown, the image quality enhancement network training device 600 includes Figure 3 In addition to all the modules shown, the device also includes: an evaluation module 30, which is used to evaluate the effect of the output converged image quality enhancement network training result. When the image quality enhancement effect of the converged image quality enhancement network training result meets the preset standard, the parameters of the image quality enhancement network are saved and the image quality enhancement training of the video image is ended. When the image quality enhancement effect of the converged image quality enhancement network training result does not meet the preset standard, the image quality enhancement training of the video image continues.

[0048] To facilitate the understanding of the technical solution provided by the present invention, a detailed description will be given below in conjunction with embodiments of specific scenarios.

[0049] Figure 7 is a flow chart of a method for training a network for image quality enhancement according to another embodiment of the present invention. Figure 7 As shown, the method comprises the following steps:

[0050] S702, obtaining a loss intensity weight a;

[0051] Specifically, the input video image and the target video image are passed through the image quality assessment module, which is used to calculate the loss strength weight a=[a1, a2, ..., a n ], n represents the number of loss types. It should be noted that the loss types for image quality evaluation here are already set, including but not limited to noise, blur, rain and fog effect, block effect, color distortion and other loss types. The intensity calculation method of each loss type can be calibrated with an objective evaluation index.

[0052] For example, loss mode 1 is image blur loss. The image blur loss is used to explain the image quality assessment process, that is, the image quality assessment module needs to calculate the blur loss intensity weight a1. For example, point sharpness is used to calibrate the blur degree. The point sharpness P calculation formula is as follows:

[0053]

[0054] Among them, H and W are the length and width of the image, df is the grayscale change amplitude, and dx is the distance increment between pixels. The formula can be described as: take 8 neighboring points from each point in the image one by one, subtract it from the 8 neighboring points, first calculate the weighted sum of the 8 differences, and then add the values ​​of all points and divide by the total number of pixels. The formula can be understood as a statistic of the grayscale diffusion degree around each point in the image, that is, the more intense the diffusion degree, the larger the value, and the clearer the image.

[0055] in, Figure 8 is a schematic diagram of an image quality assessment module according to an embodiment of the present invention. Figure 8As shown, the image quality assessment module calculates the index of the input video image to generate the point sharpness index c1, compresses the point sharpness c1 to generate c1', inverts the index of c1' to generate c1", and normalizes all the indexes. The specific steps are as follows:

[0056] The index of the input video image is calculated to calculate the point sharpness index c1. When c1 is greater than the set point sharpness threshold S, it means that the input video image has a low degree of blur, has clear edges and details, and does not need blur enhancement processing, then a1=0 is directly output, and no index compression and index inversion are required; when c1 is less than the set threshold S, it means that the input video image is obviously blurred.

[0057] Index compression is performed to convert the point sharpness c1 to the range of [0, 1] to c1'. c1' represents the point sharpness index in the range of [0, 1], which is convenient for subsequent normalization of multiple loss modes. In particular, if the calculated index is originally in the range of [0, 1], it does not need to be compressed to c1'=c1. Subsequent processing pays more attention to the loss mode with a larger weight.

[0058] Invert the index, and perform the operation of c1"=1-c1' for the point sharpness c1', where c1" represents the point sharpness index in the range [0, 1] that meets the loss severity. In particular, if c1 or c1' meets the evaluation index, the larger the corresponding loss severity, the higher the corresponding loss severity, and there is no need to perform the index inversion c1"=c1'. Similarly, the objective indexes of other loss modes 2-n can be obtained. Since the calculation method of the index is not an invention point, the specific calculation is not described in detail here. Assume that the evaluation index result of mode 2 is c2",..., and the evaluation index result of mode n is c n ”.

[0059] Finally, all indicators are normalized, as shown in formula (2), so that the image quality assessment module outputs the loss intensity weight a=[a1, a2, ..., an] corresponding to each loss mode.

[0060]

[0061] Step S704: normalize the loss strength weight a=[a1, a2, ..., a n ] and the input video image are input into the image quality enhancement module based on deep learning for network training until the training converges;

[0062] Specifically, Fig. 9 is a schematic diagram of a picture quality enhancement network according to an embodiment of the present invention. Fig. 9As shown in the figure, the deep learning network corresponds to a sub-network according to each loss type, and multiple sub-networks together form a complete image quality enhancement network. During training, the objective function of the overall network is composed of weighted objective sub-functions that focus on various types of losses. Here, the weighted weight is the loss intensity weight a=[a1, a2, ..., a n ], target sub-function loss i is the loss calculated between the sub-network output of the corresponding mode i and the target video image. The loss calculation method is related to the loss type. For example, the color loss can use cosine similarity as the target sub-function. The entire network training is constrained by the objective function so that the network forward propagation result and the target output are constantly close to achieve the purpose of training. The overall objective function loss of the image quality enhancement module is shown in formula (3).

[0063] loss=a1loss1+a2loss2+...+a n loss n , a1+a2+...+a n =1 (3)

[0064] Each sub-network output outi will be multiplied by the corresponding loss strength a i , i = 1, 2, ..., n, weighted fusion into the final output out of the deep learning network, as shown in formula (4).

[0065] out=a1out1+a2out2+...+a n out n , a1+a2+...+a n =1 (4)

[0066] Step S706, the image quality enhancement network training objective function curve converges and the enhancement result is output. If the output image quality enhancement result has a good subjective effect, the network model parameters at this time are saved and the training is completed; otherwise, the training is continued until the subjective effect meets the standard.

[0067] Fig.10 is a schematic diagram of a picture quality enhancement network according to another embodiment of the present invention. Fig.10 As shown, the image quality enhancement network adopts a multi-head and multi-tail structure which is equivalent to multiple sub-network outputs.

[0068] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0069] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0070] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0071] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0072] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0073] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for training a network for image quality enhancement, characterized in that: include: Determine the image quality loss type of the video image and the loss intensity weight corresponding to the image quality loss type; Generate a corresponding image quality enhancement network according to the image quality loss type and train the image quality enhancement network until the objective function curve of the objective function weighted by the loss intensity weight converges, and output the converged result of the image quality enhancement network training weighted by the loss intensity weight.

2. The method according to claim 1, characterized in that The video image is trained for image quality enhancement according to the image quality loss type until the objective function curve of the objective function weighted by the loss intensity weight converges, and the converged image quality enhancement network training result weighted by the loss intensity weight is output, including: Performing the image quality enhancement training on the video image through the image quality enhancement network to obtain a result of the image quality enhancement network training weighted by the loss intensity weight, wherein the image quality enhancement network includes a plurality of image quality enhancement sub-networks, and the image quality enhancement sub-networks correspond to the image quality loss types; The objective function weighted by the loss intensity weight is calculated according to the image quality loss type and the loss intensity weight corresponding to the image quality loss type, the result of the image quality enhancement sub-network training and the target video image; Determine whether the objective function curve generated according to the objective function converges. When the objective function curve converges, end the image quality enhancement training and output the converged result of the image quality enhancement network training weighted by the loss intensity weight. When the objective function curve does not converge, continue to perform the image quality enhancement training on the video image through the image quality enhancement network.

3. The method according to claim 2, characterized in that The image quality enhancement training is performed on the video image through the image quality enhancement network, and the objective function weighted by the loss intensity weight is calculated according to the image quality loss type and the loss intensity weight corresponding to the image quality loss type, the result of the image quality enhancement sub-network training and the target video image, including: Determine the target sub-function loss according to the result of the image quality enhancement sub-network output corresponding to the image quality loss type i relative to the loss of the target video image i ; According to the loss intensity weight a=[a1, a2, ..., a n ] for the target sub-function loss i Weighted objective function loss = a1loss1+a2loss2+...+a n loss n , where i=1,2,...,n.

4. The method according to claim 3, characterized in that Obtaining the image quality enhancement network training result weighted by the loss intensity weight, including: calculating the image quality enhancement network training result weighted by the loss intensity weight out = a1out1+a2out2+...+a n out n ; Among them, out i For the output of each image quality enhancement sub-network, a i For the out i The corresponding loss intensity weight.

5. The method according to claim 1, characterized in that Determining the image quality loss type of the video image and the loss intensity weight corresponding to the image quality loss type includes: The image quality of the video image is evaluated based on the target video image to determine the image quality loss type of the video image and the loss intensity weight corresponding to the image quality loss type.

6. The method according to claim 5, characterized in that Determining the loss intensity weight corresponding to the image quality loss type of the video image includes: Performing index calculation on the video image to generate a first index corresponding to the image quality loss type; Determine whether the first indicator is less than a preset threshold value, if the first indicator is greater than or equal to the preset threshold value, the loss intensity weight corresponding to the image quality loss type is 0, if the first indicator is less than the preset threshold value, determine whether the first indicator is within a preset range, if the first indicator is not within the preset range, perform indicator compression on the first indicator to generate a second indicator, if the first indicator is within the preset range, the second indicator is equal to the first indicator; The second indicator is inverted to generate a third indicator, and the third indicator is normalized to obtain the loss intensity weight.

7. The method according to claim 1, characterized in that Before determining the image quality loss type of the video image and the loss intensity weight corresponding to the image quality loss type, the method further includes: Setting the image quality loss type to be evaluated; The image quality loss types include: noise, blur, rain and fog effect, block effect and color distortion.

8. The method according to claim 1, characterized in that After outputting the converged result of the image quality enhancement network training weighted by the loss intensity weight, it also includes: The image quality enhancement effect of the output converged result of the image quality enhancement network training is evaluated. When the image quality enhancement effect of the converged result of the image quality enhancement network training meets the preset standard, the parameters of the image quality enhancement network are saved and the training of the image quality enhancement network is terminated. When the image quality enhancement effect of the converged result of the image quality enhancement network training does not meet the preset standard, the image quality enhancement network continues to be trained.

9. A network training device for image quality enhancement, characterized in that: include: A determination module, used to determine the image quality loss type of the video image and the loss intensity weight corresponding to the image quality loss type; An output module is used to generate a corresponding image quality enhancement network according to the image quality loss type and train the image quality enhancement network until the objective function curve of the objective function weighted by the loss intensity weight converges, and output the converged result of the image quality enhancement network training weighted by the loss intensity weight.

10. The device according to claim 9, characterized in that The output module also includes: A first training unit is configured to perform the image quality enhancement training on the video image through an image quality enhancement network to obtain a result of the image quality enhancement network training weighted by the loss intensity weight, wherein the image quality enhancement network includes a plurality of image quality enhancement sub-networks, and the image quality enhancement sub-networks correspond to the image quality loss types; A calculation unit, configured to calculate the objective function weighted by the loss intensity weight according to the image quality loss type and the loss intensity weight corresponding to the image quality loss type, the result of the image quality enhancement sub-network training and the target video image; The second training unit is used to determine whether the objective function curve generated according to the objective function converges. When the objective function curve converges, the image quality enhancement training is terminated, and the converged result of the image quality enhancement network training weighted by the loss intensity weight is output. When the objective function curve does not converge, the image quality enhancement training of the video image continues to be performed through the image quality enhancement network.

11. The device according to claim 9, characterized in that The determination module also includes: A determination unit is used to perform image quality evaluation on the video image based on a target video image to determine the image quality loss type of the video image and the loss intensity weight corresponding to the image quality loss type.

12. The device according to claim 9, characterized in that The device also includes: An evaluation module is used to evaluate the image quality enhancement effect of the output converged image quality enhancement network training result. When the image quality enhancement effect of the converged image quality enhancement network training result meets the preset standard, the parameters of the image quality enhancement network are saved and the training of the image quality enhancement network is terminated. When the image quality enhancement effect of the converged image quality enhancement network training result does not meet the preset standard, the image quality enhancement network continues to be trained.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 8 when executed by a processor.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

Images