Image processing method and device

Abstract

The embodiment of the invention provides an image processing method, and the method comprises the steps: taking out a training image from a training set, inputting the training image into a prediction module in a target model, and obtaining a prediction image through the prediction module; inputting the predicted image into a guided JPEG compression module in the target model, and obtaining a compressed image by using the JPEG compression module; a loss function of the target model is calculated, the loss function comprises a first loss function of the training image and the prediction image and a second loss function of the training image and the compression image, and the weight of the second loss function is inversely proportional to a quantization factor adopted by a JPEG compression module; and training the target model according to the loss function, and returning to the prediction module for extracting the training image from the training set and inputting the training image into the target model and the following steps until the target model reaches the preset precision. According to the technical scheme of the embodiment of the invention, the image quality loss generated in the quantization process can be reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video coding technology, and in particular, to an image processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] After the video image is digitized, the data volume will be very large. In order to reduce the bandwidth and storage usage and speed up the processing time, the video will be compressed and encoded using video encoding technology before the video is transmitted. Among them, video encoding technology converts the original video signal into compressed video data through an encoder, thereby reducing the amount of data. In the video encoding process, many compression technologies are used, such as intra-frame compression and inter-frame compression. Among them, JPEG quantization compression, as a mature and efficient compression technology, is widely used in the video encoding process.

[0003] In the intra-frame compression process, JPEG quantization compression uses discrete cosine transform (DCT) to convert the image in the video from the spatial domain to the frequency domain. In the frequency domain, the low-frequency part contains the main information of the image, and the high-frequency part contains the detailed information of the image. Through quantization, some information in the high-frequency part is discarded to achieve the purpose of compression. However, since the quantization process is lossy, this method will cause a certain degree of image quality loss while reducing the file size, and ultimately cause a significant loss of image quality after the video is encoded compared to the original video.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides an image processing method, the method comprising: Taking out a training image from the training set and inputting it into a prediction module in the target model, using the prediction module to predict the training image to obtain a predicted image; Inputting the predicted image into a JPEG compression module derivable in the target model, and compressing the predicted image using the JPEG compression module to obtain a compressed image; Calculating a loss function of the target model, the loss function comprising a first loss function of the training image and the predicted image, and a second loss function of the training image and the compressed image, wherein a weight of the second loss function is inversely proportional to a quantization factor adopted by the JPEG compression module; The target model is trained according to the loss function, and the prediction module and subsequent steps of taking out the training images from the training set and inputting them into the target model are returned to be executed until the target model reaches a preset accuracy.

[0007] Optionally, compressing the predicted image by using the JPEG compression module to obtain a compressed image includes: Randomly select a quantization factor from a preset quantization factor value range; Obtaining a quantization table corresponding to the randomly selected quantization factor by table lookup; The predicted image is compressed according to the quantization table to obtain a compressed image.

[0008] Optionally, the first loss function is a reconstruction loss, and the second loss function is an L1 loss function.

[0009] Optionally, the first loss function is reconstruction loss and perceptual loss.

[0010] Optionally, the prediction module adopts a convolutional neural network.

[0011] Optionally, the method further comprises: Obtain each frame image and target quantization factor of the target video; Each of the frame images is input into the trained target model respectively, and the trained target model is used to obtain a compressed image corresponding to each of the frame images according to the target quantization factor.

[0012] Another aspect of an embodiment of the present application provides an image processing device, the device comprising: A prediction module is used to take out a training image from a training set and input it into the prediction module in the target model, and use the prediction module to predict the training image to obtain a predicted image; A compression module, used for inputting the predicted image into a JPEG compression module derivable in the target model, compressing the predicted image using the JPEG compression module to obtain a compressed image, wherein the JPEG compression module is used for compressing the predicted image using a quantization factor of a random size; A calculation module, used for calculating the loss function of the target model, wherein the loss function includes a first loss function of the training image and the predicted image, and a second loss function of the training image and the compressed image, wherein the weight of the second loss function is inversely proportional to the quantization factor adopted by the JPEG compression module; A training module is used to train the target model according to the loss function, and return to execute the prediction module and subsequent steps of taking training images from the training set and inputting them into the target model until the target model reaches a preset accuracy.

[0013] Another aspect of an embodiment of the present application provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein: the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.

[0014] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0015] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0016] The above technical solution adopted in the embodiment of the present application may have the following advantages: The training image is taken from the training set and input into the prediction module in the target model, and the prediction module is used to obtain the prediction image; the prediction image is input into the differentiable JPEG compression module in the target model, and the compressed image is obtained by using the JPEG compression module, and the loss function of the target model is calculated, the loss function includes a first loss function between the training image and the prediction image, and a second loss function between the training image and the compressed image, and the weight of the second loss function is inversely proportional to the quantization factor adopted by the JPEG compression module; the target module is trained according to the loss function, and the previous steps are returned to execute until the target model reaches the preset accuracy. By introducing the differentiable JPEG compression module in the target model and adopting the loss function weight inversely proportional to the quantization factor adopted by the JPEG compression module, the prediction module and the JPEG compression module can be combined to enable the target model to obtain the ability to resist quantization loss and reduce the image quality loss caused by compression. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0018] Figure 1 The following schematically shows an operating environment diagram of the image processing method according to the first embodiment of the present application; Figure 2 The flowchart of the image processing method according to the first embodiment of the present application is schematically shown; Figure 3 Schematically shows Figure 2 Flow chart of sub-steps of step S202; Figure 4 The schematic diagram shows a principle diagram of an image processing method according to the first embodiment of the present application; Figure 5 The newly added process of the image processing method according to the first embodiment of the present application is schematically shown; Figure 6 A block diagram schematically shows an image processing device according to the second embodiment of the present application; and Figure 7 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0020] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0021] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0022] First, the following terms are explained: Discrete Cosine Transform (DCT): is a mathematical transformation that converts a signal from the time domain or space domain to the frequency domain. It is closely related to the Fourier transform, but only uses real numbers and has excellent energy concentration properties for the transformation of real even signals.

[0023] JPEG quantization compression: It is a lossy compression technology, mainly used for static image compression. Its core idea is to reduce the amount of image data by quantizing discrete cosine transform (DCT) coefficients, thereby achieving the purpose of compression.

[0024] JPEG quantization factor (Quality Factor, referred to as QF): a key parameter in JPEG compression that controls image quality, and its value range is usually 1 to 100. When the QF value increases, the quality of the compressed image will improve, but the corresponding file size will also increase; on the contrary, when the QF value decreases, the compression ratio will increase and the image quality will decrease. When the QF value is set to 100, JPEG compression will become lossless compression, and the smaller the QF value, the higher the compression ratio and the greater the loss of image quality.

[0025] Reconstruction loss: In the field of machine learning and deep learning, it specifically refers to the error caused by the model when reconstructing the original data. It measures the difference between the reconstructed data and the original data.

[0026] Perceptual loss: It is a loss function commonly used in the field of deep learning, especially in image generation tasks. Different from traditional pixel-level loss functions (such as mean square error MSE), perceptual loss focuses more on the high-level semantic information of the image, that is, the content and style of the image.

[0027] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: In the related art, since the quantization process is lossy, after the video is encoded, there will be an obvious loss in image quality compared to the original video.

[0028] To this end, the embodiment of the present application provides an image processing technical solution. In this technical solution, the image quality loss caused by the quantization process can be reduced, as described below.

[0029] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0030] like Figure 1 As shown, the environment diagram includes a service platform 2, a network 4, and a client 6, wherein: The service platform 2 may be composed of a single or multiple computing devices. The multiple computing devices may include virtualized computing instances. Virtualized computing instances may include virtual machines, such as simulations of computer systems, operating systems, servers, etc. The computing device may load a virtual machine based on a virtual image and / or other data defining specific software (e.g., operating system, dedicated application, server) for simulation. As the demand for different types of processing services changes, different virtual machines may be loaded and / or terminated on one or more computing devices. A hypervisor may be implemented to manage the use of different virtual machines on the same computing device.

[0031] The service platform 2 may be configured to communicate with the client 6 or the like via a network 4. The network 4 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network 4 may include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof, and the like, or wireless links, such as cellular links, satellite links, Wi-Fi links, and the like.

[0032] The service platform 2 can provide storage, reading, writing, querying, deleting and other services, such as providing a video viewing service for the client 6. Specifically, the video can be encoded and transmitted to the client 6 for the client 6 to decode and then view.

[0033] The client 6 may be an electronic device running an operating system such as Windows, Android™ or iOS, such as a smart phone, a tablet device, a laptop computer, a virtual reality device, a gaming device, a set-top box, a vehicle-mounted terminal, or a smart TV. Based on the above operating system, various applications may be run, such as an application for watching videos. Specifically, after the service platform 2 transmits the encoded video to the client 6, the client 6 runs the application to decode the video and then watches it.

[0034] The client 6 may provide / configure a user access page for manipulating the service platform 2 or uploading an object, etc.

[0035] It should be noted that the above devices are exemplary, and the number and type of devices are adjustable in different scenarios or according to different needs.

[0036] The following describes the technical solution of the present application through multiple embodiments, taking the service platform 2 as the execution subject. It should be noted that these embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments described here. In addition, it should be noted that the execution subject of the technical solution of the present application can also be the client 6.

[0037] Embodiment 1 Figure 2 The flowchart of the image processing method according to the first embodiment of the present application is schematically shown.

[0038] like Figure 2 As shown, the image processing method may include steps S200 to S206, wherein: Step S200: taking out a training image from the training set and inputting it into a prediction module in the target model, and using the prediction module to predict the training image to obtain a predicted image.

[0039] Step S202: input the predicted image into a JPEG compression module derivable in the target model, and compress the predicted image using the JPEG compression module to obtain a compressed image.

[0040] Step S204, calculating the loss function of the target model, the loss function includes a first loss function of the training image and the predicted image, and a second loss function of the training image and the compressed image, and the weight of the second loss function is inversely proportional to the quantization factor adopted by the JPEG compression module.

[0041] Step S206, training the target model according to the loss function, and returning to execute the steps of taking out the training images from the training set and inputting them into the prediction module in the target model and subsequent steps until the target model reaches a preset accuracy.

[0042] The image processing method provided in this embodiment takes out a training image from a training set and inputs it into a prediction module in a target model, and obtains a predicted image using the prediction module; inputs the predicted image into a differentiable JPEG compression module in the target model, and obtains a compressed image using the JPEG compression module, and calculates the loss function of the target model, wherein the loss function includes a first loss function between the training image and the predicted image, and a second loss function between the training image and the compressed image, and the weight of the second loss function is inversely proportional to the quantization factor adopted by the JPEG compression module; trains the target module according to the loss function, and returns to execute the previous steps until the target model reaches a preset accuracy, and by introducing a differentiable JPEG compression module into the target model, and adopting a loss function weight inversely proportional to the quantization factor adopted by the JPEG compression module, the prediction module and the JPEG compression module can be combined to enable the target model to acquire the ability to resist quantization loss, thereby reducing the image quality loss caused by compression.

[0043] The following combination Figure 2 , each step in steps S200~S206 and other optional steps are described in detail.

[0044] Step S200 , take out the training image from the training set and input it into the prediction module in the target model, use the prediction module to predict the training image to obtain the predicted image.

[0045] The training set can be obtained from the frame images of the video, and the obtained frame images are used as training images, so that the target model obtained by training is more in line with the requirements of video compression. The training set can include a large number of training images to meet the needs of training.

[0046] It should be understood that the target model includes two parts: a prediction module and a JPEG compression module. The prediction module is used to predict the input image to obtain a predicted image; while the JPEG compression module is used to compress the predicted image to obtain a compressed image. The two parts work together to enable the target model to resist quantization loss. Among them, the prediction module can use a machine learning method to predict the input image. Specifically, the prediction module can use a deep learning algorithm for prediction.

[0047] In an exemplary embodiment, the prediction module may use a convolutional neural network. The convolutional neural network can generate a prediction image with similar features based on the input training image, providing high-quality input for the subsequent JPEG compression module. In addition, since the convolutional neural network is visually closer to the original image, the JPEG compression module can retain more important information during the quantization process, which facilitates improving the compression ratio.

[0048] Step S202 , the predicted image is input into the JPEG compression module that can be derived in the target model, and the predicted image is compressed using the JPEG compression module to obtain a compressed image.

[0049] Since general JPEG compression involves rounding or discretization operations, the general JPEG quantization process is not differentiable. In order to make it differentiable, a smooth function (such as the sigmoid function) can be used to simulate the rounding operation. In addition, JPEG compression relies on DCT to convert the image to the frequency domain and then quantize it. When implementing a differentiable JPEG compression module, differentiable DCT and inverse DCT (IDCT) layers can be used.

[0050] When using the JPEG compression module to compress the predicted image, different quantization factors can be used for compression as much as possible for different training images to improve the generalization ability of the target model.

[0051] In an optional embodiment, if Figure 3As shown, step S202 may further include steps S300 to S304: Step S300: randomly select a quantization factor from a preset quantization factor value range.

[0052] Step S302, obtaining a quantization table corresponding to the randomly selected quantization factor by table lookup.

[0053] Step S304: compress the predicted image according to the quantization table to obtain a compressed image.

[0054] The preset quantization factor value range may be a quantization factor value range commonly used in video encoding, for example, 50-100, so that the finally trained target model can better adapt to the situation in video encoding.

[0055] When determining the quantization table corresponding to the quantization factor, in addition to obtaining the corresponding quantization table by table lookup, other methods can also be used in practical applications to obtain the corresponding quantization table. For example, the quantization table can be dynamically generated according to the statistical characteristics of the quantization factor and the image content, or the quantization matrix can be predicted using a machine learning model to output an adaptive quantization table according to the input image and the quantization factor value. Compared with other methods, the table lookup method does not require complex calculations, is simpler to implement, has stable effects and performance, and can make the compressed image compatible with the conventional JPEG compression format, ensuring smooth decoding on most devices.

[0056] Specifically, when the service platform uses the JPEG compression module to compress the predicted image, it can use the JPEG compression module to randomly select a quantization factor from a preset quantization factor value range, and then obtain the quantization table corresponding to the quantization factor by table lookup, and then compress the predicted image according to the obtained quantization table to obtain a compressed image.

[0057] In this optional embodiment, by randomly selecting a quantization factor from a preset quantization factor value range, obtaining a quantization table corresponding to the randomly selected quantization factor by table lookup, and compressing the predicted image according to the quantization table to obtain a compressed image, the target model can be learned under different quantization factors, so that the final target model can adapt to actual application scenarios and improve the generalization ability of the target model; at the same time, by obtaining the quantization table by table lookup for quantization, the target model can be adapted to actual application scenarios while being simple to implement, and can also ensure that the compressed image output by the target model is compatible with most devices.

[0058] Step S204, calculate the loss function of the target model, the loss function includes a first loss function between the training image and the predicted image, and a second loss function between the training image and the compressed image, and the weight of the second loss function is inversely proportional to the quantization factor used by the JPEG compression module.

[0059] The first loss function may be a reconstruction loss or a perceptual loss, and the second loss function may be an L1 loss function, an L2 loss function, or a structural similarity loss (SSIM loss).

[0060] In an optional embodiment, the first loss function is a reconstruction loss, and the second loss function is an L1 loss function, wherein the reconstruction loss may specifically be an L1 loss function, an L2 loss function, or a structural similarity loss.

[0061] Specifically, the loss function of the target model can be expressed as: L=Construction_Loss(X,Y')+(1 / sqrt(QF))*L1_Loss(X,Y”); Among them, Construction_Loss(X, Y') represents the reconstruction loss, L1_Loss represents the L1 loss function, X is the training image, Y' is the predicted image, Y' is the compressed image, and sqrt represents the square root. In addition, in practical applications, the weight (1 / sqrt(QF)) can be replaced by 1 / QF or 1 / log(QF+C), etc.

[0062] From the loss function of the target model above, it can be seen that the larger the quantization factor QF is, the smaller the loss of the image is, the less "resistance" is needed, and the smaller the weight (1 / sqrt(QF)) is; on the contrary, the smaller the quantization factor QF is, the greater the loss of the image is, the more the target model needs to learn this quantization process to "resist" the loss, and the larger the weight (1 / sqrt(QF)) is.

[0063] Please refer to Figure 4 , which is a principle example diagram of the image processing method of the embodiment of the present application. As shown in the figure, the prediction module uses a convolutional neural network to predict the input image, and then the target model uses a derivable JPEG compression module to compress the predicted image to obtain a compressed image. The loss function of the target model includes the reconstruction loss of the training image and the predicted image and the first loss function of the training image and the compressed image.

[0064] It can be understood that the first loss function uses reconstruction loss to ensure the pixel-level similarity between the predicted image and the training image, keep the picture intact, and ensure the overall quality of the image; while the second loss function uses L1 loss function, which can better reflect the detailed information of the image and is conducive to improving compression performance. Through the combination of the first loss function and the second loss function, the reconstruction quality and compression performance can be balanced to achieve better compression effect.

[0065] In an optional embodiment, the first loss function is reconstruction loss and perceptual loss. Since reconstruction loss is used to maintain pixel-level similarity, and perceptual loss focuses on high-level features of the image, making the predicted image visually closer to human perception, by combining these two losses, the target model can optimize the overall pixel similarity while ensuring the visual quality of the predicted image at high-level features.

[0066] Step S206 , train the target model according to the loss function, and return to execute the prediction module and subsequent steps of taking training images from the training set and inputting them into the target model until the target model reaches the preset accuracy.

[0067] Specifically, the service platform updates the weight of the target model according to the loss function, and then returns to execute steps S200 to S206 until the target model reaches the preset accuracy. When the target model reaches the preset accuracy, the final trained target model is obtained.

[0068] In an optional embodiment, if Figure 5 As shown, the image processing method may further include steps S400 to S402: Step S400, obtaining each frame image and target quantization factor of the target video.

[0069] Step S402, input each frame of image into the trained target model respectively, and use the trained target model to obtain the compressed image corresponding to each frame of image according to the target quantization factor.

[0070] Specifically, the service platform can extract each frame image from the target video, obtain the target quantization factor used for this compression, input each frame image and the target quantization factor into the trained target model, and the JPEG compression module in the model quantizes the predicted image obtained from the frame image according to the given quantization factor, and outputs the compressed image corresponding to each frame image.

[0071] In this optional embodiment, by obtaining each frame image and target quantization factor of the target video, each frame image is input into the trained target model respectively, and the trained target model is used to obtain the compressed image corresponding to each frame image according to the target quantization factor. The trained target model can be effectively used to compress the target video, and because the trained target model can effectively resist quantization loss, the trained target model can be used to compress the target video, which can effectively reduce the image quality loss in the compression process.

[0072] Embodiment 2 Figure 6 The block diagram of the image processing device 500 according to the second embodiment of the present application is schematically shown. The device 500 can be divided into one or more program modules, one or more program modules are stored in a storage medium, and are executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 6 As shown, the apparatus 500 may include: a prediction module 510, a compression module 520, a calculation module 530 and a training module 540, wherein: A prediction module 510 is used to extract a training image from a training set and input it into a prediction module in a target model, and use the prediction module to predict the training image to obtain a predicted image; A compression module 520, configured to input the predicted image into a JPEG compression module derivable in the target model, and compress the predicted image using the JPEG compression module to obtain a compressed image; A calculation module 530, configured to calculate a loss function of the target model, wherein the loss function includes a first loss function of the training image and the predicted image, and a second loss function of the training image and the compressed image, wherein a weight of the second loss function is inversely proportional to a quantization factor used by the JPEG compression module; The training module 540 is used to train the target model according to the loss function, and return to execute the prediction module and subsequent steps of taking out the training images from the training set and inputting them into the target model until the target model reaches a preset accuracy.

[0073] In an optional embodiment, the compression module 520 is further configured to: Randomly select a quantization factor from a preset quantization factor value range; Obtaining a quantization table corresponding to the randomly selected quantization factor by table lookup; The predicted image is compressed according to the quantization table to obtain a compressed image.

[0074] In an optional embodiment, the first loss function is a reconstruction loss, and the second loss function is an L1 loss function.

[0075] In an optional embodiment, the first loss function is reconstruction loss and perceptual loss.

[0076] In an optional embodiment, the prediction module adopts a convolutional neural network.

[0077] In an optional embodiment, the device 500 is further used for: Obtain each frame image and target quantization factor of the target video; Each of the frame images is input into the trained target model respectively, and the trained target model is used to obtain a compressed image corresponding to each of the frame images according to the target quantization factor.

[0078] Embodiment 3 Figure 7 The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing the image processing method according to the third embodiment of the present application is schematically shown. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Figure 7 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store an operating system and various application software installed in the computer device 10000, such as program codes of image processing methods, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or are to be output.

[0079] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0080] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0081] It should be pointed out that Figure 7 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0082] In this embodiment, the image processing method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0083] Embodiment 4 An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the image processing method in the embodiment are implemented.

[0084] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device, such as the program code of the image processing method in the embodiment, etc. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.

[0085] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0086] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0087] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Taking out a training image from the training set and inputting it into a prediction module in the target model, using the prediction module to predict the training image to obtain a predicted image; Inputting the predicted image into a JPEG compression module derivable in the target model, and compressing the predicted image using the JPEG compression module to obtain a compressed image; Calculating a loss function of the target model, the loss function comprising a first loss function of the training image and the predicted image, and a second loss function of the training image and the compressed image, wherein a weight of the second loss function is inversely proportional to a quantization factor adopted by the JPEG compression module; The target model is trained according to the loss function, and the prediction module and subsequent steps of taking out the training images from the training set and inputting them into the target model are returned to be executed until the target model reaches a preset accuracy.

2. The method according to claim 1, characterized in that The method of compressing the predicted image by using the JPEG compression module to obtain a compressed image includes: Randomly select a quantization factor from a preset quantization factor value range; Obtaining a quantization table corresponding to the randomly selected quantization factor by table lookup; The predicted image is compressed according to the quantization table to obtain a compressed image.

3. The method according to claim 1, characterized in that The first loss function is a reconstruction loss, and the second loss function is an L1 loss function.

4. The method according to claim 1, characterized in that The first loss function is reconstruction loss and perceptual loss.

5. The method according to any one of claims 1 to 4, characterized in that: The prediction module adopts a convolutional neural network.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Obtain each frame image and target quantization factor of the target video; Each of the frame images is input into the trained target model respectively, and the trained target model is used to obtain a compressed image corresponding to each of the frame images according to the target quantization factor.

7. An image processing device, characterized in that: The device comprises: A prediction module is used to take out a training image from a training set and input it into the prediction module in the target model, and use the prediction module to predict the training image to obtain a predicted image; A compression module, used for inputting the predicted image into a JPEG compression module derivable in the target model, and compressing the predicted image using the JPEG compression module to obtain a compressed image; A calculation module, used for calculating the loss function of the target model, wherein the loss function includes a first loss function of the training image and the predicted image, and a second loss function of the training image and the compressed image, wherein the weight of the second loss function is inversely proportional to the quantization factor adopted by the JPEG compression module; A training module is used to train the target model according to the loss function, and return to execute the prediction module and subsequent steps of taking training images from the training set and inputting them into the target model until the target model reaches a preset accuracy.

8. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images