Image processing method, device and storage medium

Abstract

The embodiments of the present application provide an image processing method and device, and a storage medium, including: receiving a first video frame in each preset period, and using a first processing module and a second processing module to process the first video frame to obtain a first output video frame; the running time of the first processing module is less than the running time of the second processing module; the first video frame is the first video frame received in each preset period; continuing to receive a second video frame in each preset period, and using the first processing module to process the second video frame to obtain a second output video frame; outputting the first output video frame and the second output video frame in sequence.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method and device, and a storage medium. Background Art

[0002] In the process of taking photos with a mobile phone or camera, due to factors such as lens parameters or shooting environment, the portrait in the photo may be severely distorted. Currently, portrait distortion correction can be performed on a single picture. The specific portrait distortion correction can include optical distortion correction, face detection, portrait segmentation, grid optimization and interpolation calculation. The above steps are performed in sequence to correct the portrait distortion in the taken photo.

[0003] However, operations such as portrait segmentation and grid optimization in the existing process are computationally intensive and take a long time to run. If the existing portrait distortion correction process is still executed sequentially for each frame in the video, the number of frames transmitted per second will be reduced, making it difficult to meet the real-time requirements of the video. Summary of the invention

[0004] The embodiments of the present application provide an image processing method and device, and a storage medium, which can ensure that the number of frames transmitted per second of a video meets the real-time requirements of the video.

[0005] The technical solution of this application is implemented as follows:

[0006] The present application provides an image processing method, the method comprising:

[0007] In each preset period, a first video frame is received, and the first processing module and the second processing module are used to process the first video frame to obtain a first output video frame; the running time of the first processing module is less than the running time of the second processing module; the first video frame is the first video frame received in each preset period;

[0008] In each of the preset cycles, continue to receive the second video frame, and use the first processing module to process the second video frame to obtain a second output video frame;

[0009] The first output video frame and the second output video frame are output in sequence.

[0010] The present application provides an image processing device, the device comprising:

[0011] A receiving module, configured to receive the first video frame within each preset period, and continue to receive the second video frame within each preset period;

[0012] A processing module, configured to process the first video frame using a first processing module and a second processing module to obtain a first output video frame; the running time of the first processing module is less than the running time of the second processing module; the first video frame is a video frame received for the first time in each preset period; and the second video frame is processed using the first processing module to obtain a second output video frame;

[0013] An output module is used to output the first output video frame and the second output video frame in sequence.

[0014] An embodiment of the present application provides an image processing device, which includes: a processor, a memory, and a communication bus; when the processor executes a running program stored in the memory, the image processing method as described above is implemented.

[0015] An embodiment of the present application provides a storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned image processing method when executed by a processor.

[0016] The embodiment of the present application provides an image processing method and device, and a storage medium, the method comprising: receiving a first video frame in each preset period, and processing the first video frame using a first processing module and a second processing module to obtain a first output video frame; the running time of the first processing module is less than the running time of the second processing module; the first video frame is the first video frame received in each preset period; in each preset period, continue to receive the second video frame, and use the first processing module to process the second video frame to obtain a second output video frame; and output the first output video frame and the second output video frame in sequence. Using the above implementation scheme, a preset period is set for the received video frame in advance, and the first processing module is used to process each video frame, and the second processing module is used to process only the first video frame in each preset period. Since the running time of the second processing module is greater than the running time of the first processing module, the second processing module is not used to process the second video frame in the preset period, which can greatly improve the number of frames transmitted per second of the video; and the change of the video frame in the preset period is small, which can ensure that the number of frames transmitted per second of the video meets the real-time requirements of the video without affecting the viewing effect of the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A structural schematic diagram of an image processing method provided in an embodiment of the present application;

[0018] Figure 2 An exemplary data flow chart for correcting portrait distortion in video recording provided in an embodiment of the present application;

[0019] Figure 3An exemplary data flow chart for correcting portrait distortion in video preview provided in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 1 ;

[0021] Figure 5 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0022] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0023] The present application embodiment provides an image processing method, such as Figure 1 As shown, the method may include:

[0024] S101. In each preset period, a first video frame is received, and the first video frame is processed by a first processing module and a second processing module to obtain a first output video frame; the running time of the first processing module is less than the running time of the second processing module; the first video frame is the first video frame received in each preset period.

[0025] An image processing method proposed in an embodiment of the present application is suitable for a scenario in which image processing is performed on a video frame when converting an image into a video.

[0026] In the embodiment of the present application, the device for performing image processing can be any device with communication and storage functions, such as: tablet computers, mobile phones, personal computers (PCs), laptops, cameras, wearable devices and other photographic devices.

[0027] It should be noted that, in the process of converting images into videos, image processing can be performed on each video frame to meet different video requirements. In the process of performing image processing on the collected video frames, the image processing process can be divided into modules. The embodiment of the present application divides the image processing process into two types of processing modules based on the running time, one is a first processing module, and the other is a second processing module, wherein the running time of the first processing module is less than the running time of the second processing module.

[0028] It should be noted that the image processing type of the video frame may include portrait distortion correction, image enhancement, etc., which can be determined according to actual conditions and is not specifically limited in the embodiment of the present application. The following description takes portrait distortion correction as an example.

[0029] In the embodiment of the present application, the number of the first processing modules is at least one, and the number of the second processing modules is at least one. The specific number of the first processing modules and the second processing modules is determined according to the specific steps of the image processing, and the embodiment of the present application does not make any specific limitation.

[0030] In the embodiment of the present application, the first processing module and the second processing module can be used in sequence to process the first video frame to obtain the first output video frame; or the first processing module and the second processing module can be used alternately to process the first video frame to obtain the first output video frame. The specific execution order of the first processing module and the second processing module can be determined according to the specific steps of the graphics processing, and the embodiment of the present application does not make a specific limitation.

[0031] In an optional embodiment, in a portrait distortion correction scenario, the portrait distortion correction steps of optical distortion correction, face detection, portrait segmentation, grid optimization and interpolation calculation are sequentially included. According to the above-mentioned portrait distortion correction steps, it is divided into an optical distortion correction and face detection module, a portrait segmentation module, a grid optimization module and an interpolation calculation module, wherein the running time of the optical distortion correction and face detection module is 12ms, the running time of the portrait segmentation module is 15ms, the running time of the grid optimization module is 50ms, and the running time of the interpolation calculation module is 12ms. Therefore, the first processing module includes the optical distortion correction and face detection module, and the interpolation calculation module; the second processing module includes the portrait segmentation module and the grid optimization module.

[0032] In the embodiment of the present application, a video frame is received every preset receiving time period, and the specific time interval for receiving the video frame can be determined based on the number of frames transmitted per second of the video, such as receiving a video frame every 30ms by inputting a transmission algorithm.

[0033] In the embodiment of the present application, before converting the image into a video, the image processing device pre-sets a preset period, and the preset period can be set in units of the number of frames received, such as one preset period for every 3 frames of video, or one preset period for every 2 frames of video. The number of video frames received in the specific preset period is determined according to the running time of the first processing module, the running time of the second processing module, and the receiving time of two adjacent video frames. The image processing device uses the first processing module and the second processing module to process the first video frame received for the first time in each preset period to obtain a first output video frame.

[0034] It should be noted that, for the portrait distortion correction scenario, the number of frames received within the preset period must be set to satisfy the requirement that only one optimization grid generation process exists at the same time, that is, the optimization grid module generates the next optimization grid only after generating one optimization grid.

[0035] In an optional embodiment, the first processing module includes a first processing sub-module and a second processing sub-module; first, the first video frame is processed using the first processing sub-module to obtain a first processing parameter; then, the first processing parameter is processed using the second processing module to obtain an image optimization model; finally, the second processing sub-module is called to input the first processing parameter into the image optimization model to obtain a first output video frame.

[0036] In the portrait distortion correction scenario, the first processing submodule is the optical distortion correction and face detection module, the second processing submodule is the interpolation calculation module, and the second processing module is the portrait segmentation module and the optimized grid module; when the first video frame is received as input, the first video frame is input into the optical distortion correction and face detection module, and the image parameters u and face attribute parameters f after optical distortion processing are output, wherein the face attribute parameters include face frame, key points, posture, etc.; then u is input into the portrait segmentation module, and the portrait mask m is output; f and u are input into the optimized grid module to obtain the optimized grid Opt; finally, u and Opt are input into the interpolation calculation module, and Opt is used to perform interpolation calculation on u to obtain the final face corrected image and output it.

[0037] In another optional embodiment, the first processing module includes a first processing submodule and a second processing submodule. First, the first processing submodule is used to process the first video frame to obtain the first processing parameter; then the second image optimization model is searched; wherein the second image optimization model is the image optimization model finally generated when the first processing parameter is obtained; when the second image optimization model is found, the second processing submodule is called, the first processing parameter is input into the second image optimization model, and the second output video frame is obtained.

[0038] Furthermore, after using the first processing submodule to process the first video frame to obtain the first processing parameter, the second processing module is also used to process the first processing parameter to obtain the image optimization module corresponding to the first video frame. Thus, within the next preset cycle, the received video frame can be processed based on the image optimization module corresponding to the first video frame.

[0039] Further, when the second image optimization model is not found, the first processing parameters are not processed.

[0040] It should be noted that, since the first frame of video is subjected to graphics processing to obtain the first image optimization model within the first preset cycle, there is no optimized image optimization model. Therefore, in this embodiment, no subsequent optimization processing is performed on the video frames received within the first preset cycle.

[0041] In the scene of portrait distortion correction, the first processing submodule is the optical distortion correction and face detection module, the second processing submodule is the interpolation calculation module, and the second processing module is the portrait segmentation module and the optimized grid module; when the first video frame is received as input, the first video frame is input into the optical distortion correction and face detection module, and the image parameter u1 and the face attribute parameter f1 after optical distortion processing are output, wherein the face attribute parameters include face frame, key points, posture, etc.; when u1 and f1 are found, the optimized grid Opt2 finally generated by the optimization network module is optimized, and when Opt2 is found, u1 and Opt2 are input into the interpolation calculation module, and Opt2 is used to interpolate u1 to obtain the final face-corrected image and output it. When Opt2 is not found, u1 is not processed. At the same time, after outputting u1 and f1, u1 is also input into the portrait segmentation module to output the portrait mask m1; f1 and u1 are input into the optimized grid module to obtain the optimized grid Opt2; for the video frame received before obtaining Opt3, after inputting the optical distortion correction and face detection module to obtain u2 and the face attribute parameter f2, the Opt2 can be used to perform interpolation calculation on u2 to obtain the output video frame.

[0042] S102. Continue to receive the second video frame in each preset period, and use the first processing module to process the second video frame to obtain a second output video frame.

[0043] In the embodiment of the present application, the second video frame received after the first video frame in each preset period is processed by the first processing module to obtain a second output video frame.

[0044] In scenarios such as portrait distortion correction, where an optimized grid is required to perform image processing on the input video frame, the second output video frame depends not only on the processing of the first processing module, but also on the processing of the optimized grid generated by the second processing module. For example, for the portrait distortion correction scenario, the first processing submodule performs the second processing parameters on the second video frame, then obtains the image optimization model (which can be an optimized grid), and calls the second processing submodule to perform interpolation operations on the second processing parameters using the obtained optimized grid to obtain the final second output video frame.

[0045] In an optional embodiment, the second video frame is processed using the first processing module to obtain a second output video frame, including: using the first processing submodule to process the second video frame to obtain a second processing parameter; waiting to obtain a first image optimization model obtained by processing the first video frame of this round; the first video frame of this round is a video frame in the same preset period as the second video frame; when the first image optimization model is obtained, calling the second processing submodule, inputting the second processing parameter into the first image optimization model, and obtaining a second output video frame.

[0046] In the scenario of portrait distortion correction, within each preset cycle, when the second video frame is received after the first video frame, the second video frame is input into the optical distortion correction and face detection module, and u3 and f3 are output. Then, wait for obtaining the optimized grid Opt3 obtained by processing the first video frame of this round; after obtaining Opt3, u3 and Opt3 are input into the interpolation calculation module, and Opt3 is used to perform interpolation calculation on u3 to obtain the final face corrected image and output it.

[0047] It should be noted that in the process of processing the second processing parameters using the first image optimization model, since the second processing parameters in each preset cycle need to wait for the generation of the first image optimization model of this round and then perform subsequent interpolation operations in sequence, output delay problems will occur. Therefore, the process of processing the second processing parameters using the first image optimization model is more suitable for video recording.

[0048] In another optional embodiment, the first processing module is used to process the second video frame received after the first video frame within each preset period to obtain a second output video frame, including: using the first processing submodule to process the second video frame to obtain second processing parameters; searching for a third image optimization model, wherein the third image optimization model is the image optimization model finally generated when the second processing parameters are obtained; when the third image optimization model is found, calling the second processing submodule, inputting the second processing parameters into the third image optimization model, and obtaining a second output video frame.

[0049] In the portrait distortion correction scenario, when the second video frame is received as input, the second video frame is input into the optical distortion correction and face detection module, and the image parameters u3 and face attribute parameters f3 after optical distortion processing are output, where the face attribute parameters include face frame, key points, posture, etc.; when u3 and f3 are found, the optimized mesh Opt2 finally obtained by the optimization network module is optimized. When Opt2 is found, u3 and Opt2 are input into the interpolation calculation module, and Opt2 is used to interpolate u3 to obtain the final face correction image and output it. When Opt2 is not found, u3 is not processed.

[0050] It should be noted that in the process of processing the second processing parameters using the third image optimization model, since the second processing parameters in each preset period directly use the already generated second image optimization model for subsequent interpolation operations, an image optimization model will be generated in each preset period. Therefore, compared with the process of processing the second processing parameters using the first image optimization model, the video delay is smaller. Therefore, the process of processing the second processing parameters using the second image optimization model is more suitable for real-time video preview.

[0051] It should be noted that the number of the second video frames may be one or more. The specific number of the second video frames is determined according to the number of video frames received within a preset period, and the embodiment of the present application does not make any specific limitation.

[0052] S103 , output a first output video frame and a second output video frame in sequence.

[0053] After the first video frame received in each preset period is processed by the first processing module and the second processing module to obtain the first output video frame, and the second video frame received after the first video frame in each preset period is processed by the first processing module to obtain the second output video frame, the first output video frame and the second output video frame are outputted in sequence.

[0054] It can be understood that a preset period is set in advance for the received video frames, and the first processing module is used to process each video frame. The second processing module is also used to process only the first video frame in each preset period. Since the running time of the second processing module is greater than the running time of the first processing module, the second processing module is not used to process the second video frame in the preset period, which can greatly improve the number of frames transmitted per second of the video; and the change of the video frames within the preset period is small, which can ensure that the number of frames transmitted per second of the video meets the real-time requirements of the video without affecting the viewing effect of the video.

[0055] Based on the above embodiment, the portrait distortion in the video recording can be corrected. Specifically, a frame of video image is input every 30ms; every 3 frames of video image is a preset cycle. Figure 2The video portrait distortion correction data stream, frame0-frame2 is a preset cycle, frame3-frame5 is a preset cycle, frame6-frame8 is a preset cycle, frame9-frame11 is a preset cycle, the execution process of each preset cycle is the same, and the preset cycle of frame0-frame2 is taken as an example to illustrate the portrait distortion correction in the video recording process; frame0 is the first video frame, frame and frame2 are the second video frames, firstly, frame0 is received through the input transmission algorithm, frame0 is input into the optical distortion correction and face detection module, and u0 and f0 are output, and then u0 is input into the portrait segmentation module, and m0 is output; f0 and m0 are input into the optimized grid module to obtain the optimized grid Opt0; since the running time of the optical distortion correction and face detection module is 12ms, the running time of the portrait segmentation module is The time is 15ms. In the process of inputting f0 and m0 into the optimized grid module to obtain the optimized grid Opt0, frame1 is received through the input transmission algorithm, and the running time of the optimized grid module is 50ms. Frame2 will also be received. After receiving frame1 and frame2 in sequence, the optical distortion correction and face detection modules are used to process frame1 in turn to obtain u1 and f1, and frame2 is processed to obtain u2 and f2. Then wait for Opt0 to be obtained. After obtaining Opt0, the interpolation calculation module is called, u0 and Opt0 are input into the interpolation calculation module in turn, and Opt0 is used to interpolate u0 to obtain out0, u1 and Opt0 are input into the interpolation calculation module, and Opt0 is used to interpolate u1 to obtain out1, u2 and Opt0 are input into the interpolation calculation module, and Opt0 is used to interpolate u2 to obtain out2.

[0056] Based on the above embodiment, the portrait distortion in the video preview can be corrected. Specifically, a frame of video image is input every 30ms; every 2 frames of video image is a preset cycle. Figure 3The video portrait distortion correction data stream is frame0-frame1 as a preset period, frame2-frame3 as a preset period, frame4-frame5 as a preset period, frame6-frame7 as a preset period, frame0, frame2, frame4 and frame6 are the first video frame, frame1, frame3, frame5 and frame7 are the second video frame, firstly, frame0 is received, frame0 is input into the optical distortion correction and face detection module, u0 and f0 are input, and then u0 is input into the portrait segmentation module to output m0; m0 is input into the optimized grid module to obtain the optimized grid Opt0; in the process of obtaining Opt0, frame1 is received, frame1 is input into the optical distortion correction and face detection module, u1 and f1 are output, and frame2 is received after an interval of 30ms, and frame2 is input into the optical distortion correction and face detection module to output u2 and f2 (since there is no optimized grid at this time, u0, u1 and u2 are subsequently interpolated and processed); then, u2 is input into the portrait segmentation module and m2 is output; m2 is input into the optimized grid module to obtain the optimized grid Opt2; in the process of obtaining Opt2, frame3 and frame4 are received in sequence. For frame3, since Opt0 has been obtained, u3 and f3 are obtained through the optical distortion correction and face detection modules; then, u3 and Opt0 are input into the interpolation calculation module, and Opt0 is used to interpolate u3 to obtain out0, which is the output of the video It starts from frame3. For frame4, while using Opt0 to interpolate u4 output by the optical distortion correction and face detection module to obtain out1, u4 is also input into the portrait segmentation module to obtain m4, and f4 and m4 output by the optical distortion correction and face detection module are input into the optimization network module to obtain the optimized grid Opt4; and in the process of obtaining Opt4, frame5 is also received, and Opt2 is used to interpolate u6 output by the optical distortion correction and face detection module to obtain out2.

[0057] Based on the above embodiments, the present application also provides an image processing device 1. Figure 4 As shown, the image processing device 1 includes:

[0058] The receiving module 10 is used to receive the first video frame in each preset period, and continue to receive the second video frame in each preset period;

[0059] The processing module 11 is used to process the first video frame using the first processing module and the second processing module to obtain a first output video frame; the running time of the first processing module is less than the running time of the second processing module; the first video frame is the first video frame received in each preset period; the second video frame is processed using the first processing module to obtain a second output video frame;

[0060] The output module 12 is used to output the first output video frame and the second output video frame in sequence.

[0061] Optionally, the number of video frames received within the preset period is determined according to the running time of the first processing module, the running time of the second processing module and the receiving time of two adjacent video frames.

[0062] Optionally, the first processing module includes a first processing submodule and a second processing submodule;

[0063] The processing module 11 is also used to use the first processing submodule to process the first video frame to obtain a first processing parameter; use the second processing module to process the first processing parameter to obtain an image optimization model; call the second processing submodule to input the first processing parameter into the image optimization model to obtain the first output video frame.

[0064] Optionally, the device further includes: an acquisition module;

[0065] The processing module 11 is further configured to process the second video frame using the first processing submodule to obtain a second processing parameter; when the first image optimization model is acquired, the second processing submodule is called to input the second processing parameter into the first image optimization model to obtain the second output video frame;

[0066] The acquisition module is used to wait for acquisition of a first image optimization model obtained by processing the first video frame of this round; the first video frame of this round is a video frame in the same preset period as the second video frame.

[0067] Optionally, the first processing module includes a first processing submodule and a second processing submodule; the device further includes: a search module;

[0068] The processing module 11 is further configured to process the first video frame using the first processing submodule to obtain a first processing parameter; when the second image optimization model is found, the second processing submodule is called to input the first processing parameter into the second image optimization model to obtain the second output video frame;

[0069] The search module is also used to search for a second image optimization model; the second image optimization model is the image optimization model finally generated when the first processing parameters are obtained.

[0070] Optionally, the processing module 11 is also used to use the second processing module to process the first processing parameters to obtain the image optimization model corresponding to the first video frame, so as to process the received video frame based on the image optimization model corresponding to the first video frame within the next preset period.

[0071] Optionally, the processing module 11 is further configured to process the second video frame using the first processing submodule to obtain a second processing parameter; and when the third image optimization model is found, call the second processing submodule, input the second processing parameter into the third image optimization model, and obtain the second output video frame;

[0072] The search module is further used to search for a third image optimization model, where the third image optimization model is the image optimization model finally generated when the second processing parameters are obtained.

[0073] Based on the above embodiments, the embodiment of the present application also provides an image processing device, in each preset period, receiving a first video frame, and using a first processing module and a second processing module to process the first video frame to obtain a first output video frame; the running time of the first processing module is less than the running time of the second processing module; the first video frame is the first video frame received in each preset period; in each preset period, continuing to receive the second video frame, and using the first processing module to process the second video frame to obtain a second output video frame; outputting the first output video frame and the second output video frame in sequence. It can be seen that the image processing device proposed in this embodiment sets a preset period for the received video frame in advance, uses the first processing module to process each video frame, and also uses the second processing module to process only the first video frame in each preset period. Since the running time of the second processing module is greater than the running time of the first processing module, the second processing module is not used to process the second video frame in the preset period, which can greatly improve the number of frames transmitted per second of the video; and the change of the video frame in the preset period is small, which can ensure that the number of frames transmitted per second of the video meets the real-time requirements of the video without affecting the viewing effect of the video.

[0074] Figure 5 A schematic diagram of the structure of an image processing device 1 provided in an embodiment of the present application Figure 2 In practical applications, based on the same disclosure concept of the above embodiments, Figure 5 As shown, the image processing device 1 of this embodiment includes: a processor 13 , a memory 14 , a receiver 15 and a communication bus 16 .

[0075] In the process of the specific embodiment, the processing module 11, the output module 12, the acquisition module and the search module can be implemented by the processor 13 located on the image processing device 1, and the receiving module 10 can be implemented by the receiver 15 located on the image processing device 1; the processor 13 can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing image processing device (DSPD), a programmable logic image processing device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other, and this embodiment does not specifically limit it.

[0076] In the embodiment of the present application, the communication bus 16 is used to realize the connection and communication between the processor 13, the memory 14 and the receiver 15; the receiver 15 is used to receive the first video frame in each preset period, and continue to receive the second video frame in each preset period; the processor 13 implements the following image processing method when executing the running program stored in the memory 14:

[0077] The first video frame is processed by a first processing module and a second processing module to obtain a first output video frame; the running time of the first processing module is less than the running time of the second processing module; the first video frame is the first video frame received in each preset period; the second video frame is processed by the first processing module to obtain a second output video frame; the first output video frame and the second output video frame are output in sequence.

[0078] Furthermore, in the above embodiment, the number of video frames received within the preset period is determined according to the running time of the first processing module, the running time of the second processing module and the receiving time of two adjacent video frames.

[0079] Further, in the above embodiment, the first processing module includes a first processing submodule and a second processing submodule;

[0080] The processor 13 is also used to use the first processing submodule to process the first video frame to obtain a first processing parameter; use the second processing module to process the first processing parameter to obtain an image optimization model; call the second processing submodule to input the first processing parameter into the image optimization model to obtain the first output video frame.

[0081] Furthermore, in the above embodiment, the processor 13 is also used to process the second video frame using the first processing submodule to obtain a second processing parameter; wait for obtaining the first image optimization model obtained by processing the first video frame of this round; the first video frame of this round is a video frame in the same preset period as the second video frame; when the first image optimization model is obtained, call the second processing submodule, input the second processing parameter into the first image optimization model, and obtain the second output video frame.

[0082] Further, in the above embodiment, the first processing module includes a first processing submodule and a second processing submodule;

[0083] The processor 13 is also used to use the first processing submodule to process the first video frame to obtain a first processing parameter; search for a second image optimization model; the second image optimization model is the image optimization model finally generated when the first processing parameter is obtained; when the second image optimization model is found, call the second processing submodule, input the first processing parameter into the second image optimization model, and obtain the second output video frame.

[0084] Furthermore, in the above embodiment, the processor 13 is also used to process the first processing parameters using the second processing module to obtain an image optimization model corresponding to the first video frame, so as to process the received video frame based on the image optimization model corresponding to the first video frame within the next preset cycle.

[0085] Furthermore, in the above embodiment, the processor 13 is also used to process the second video frame using the first processing submodule to obtain second processing parameters; search for a third image optimization model, and the third image optimization model is the image optimization model finally generated when the second processing parameters are obtained; when the third image optimization model is found, call the second processing submodule, input the second processing parameters into the third image optimization model, and obtain the second output video frame.

[0086] An embodiment of the present application provides a storage medium having a computer program stored thereon. The computer-readable storage medium stores one or more programs. The one or more programs can be executed by one or more processors and applied to an image processing device. The computer program implements the image processing method as described above.

[0087] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0088] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for enabling an image display device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0089] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: In each preset period, a first video frame is received, and the first processing module and the second processing module are used to process the first video frame to obtain a first output video frame; the running time of the first processing module is less than the running time of the second processing module; the first video frame is the first video frame received in each preset period; In each of the preset cycles, continue to receive the second video frame, and use the first processing module to process the second video frame to obtain a second output video frame; outputting the first output video frame and the second output video frame in sequence; The number of video frames received in each preset period is determined according to the running time of the first processing module, the running time of the second processing module and the receiving time of two adjacent video frames.

2. The method according to claim 1, characterized in that The first processing module includes a first processing submodule and a second processing submodule; the first processing module and the second processing module are used to process the first video frame to obtain a first output video frame, including: Using the first processing submodule to process the first video frame to obtain a first processing parameter; Using the second processing module to process the first processing parameter to obtain an image optimization model; The second processing submodule is called, and the first processing parameter is input into the image optimization model to obtain the first output video frame.

3. The method according to claim 2, characterized in that The using the first processing module to process the second video frame to obtain a second output video frame includes: Processing the second video frame using the first processing submodule to obtain a second processing parameter; Waiting to obtain a first image optimization model obtained by processing the first video frame of this round; the first video frame of this round is a video frame in the same preset period as the second video frame; When the first image optimization model is obtained, the second processing submodule is called, and the second processing parameters are input into the first image optimization model to obtain the second output video frame.

4. The method according to claim 1, characterized in that: The first processing module includes a first processing submodule and a second processing submodule; the first processing module and the second processing module are used to process the first video frame to obtain a first output video frame, including: Using the first processing submodule to process the first video frame to obtain a first processing parameter; searching for a second image optimization model; the second image optimization model is the image optimization model finally generated when the first processing parameters are obtained; When the second image optimization model is found, the second processing submodule is called, and the first processing parameters are input into the second image optimization model to obtain the first output video frame.

5. The method according to claim 4, characterized in that After the first processing submodule is used to process the first video frame to obtain a first processing parameter, the method further includes: The first processing parameter is processed by the second processing module to obtain the image optimization model corresponding to the first video frame, so as to process the received video frame based on the image optimization model corresponding to the first video frame within the next preset cycle.

6. The method according to claim 4, characterized in that The using the first processing module to process the second video frame to obtain a second output video frame includes: Processing the second video frame using the first processing submodule to obtain a second processing parameter; searching for a third image optimization model, where the third image optimization model is the image optimization model finally generated when the second processing parameters are obtained; When the third image optimization model is found, the second processing submodule is called, and the second processing parameters are input into the third image optimization model to obtain the second output video frame.

7. An image processing device, characterized in that: The device comprises: A receiving module, configured to receive a first video frame in each preset period, and continue to receive a second video frame in each preset period; wherein the number of video frames received in each preset period is determined according to the running time of the first processing module, the running time of the second processing module, and the receiving time of two adjacent video frames; A processing module, configured to process the first video frame using the first processing module and the second processing module to obtain a first output video frame; the running time of the first processing module is less than the running time of the second processing module; the first video frame is the first video frame received in each preset period; and the second video frame is processed using the first processing module to obtain a second output video frame; An output module is used to output the first output video frame and the second output video frame in sequence.

8. An image processing device, characterized in that: The device comprises: a processor, a memory and a communication bus; when the processor executes the running program stored in the memory, the method according to any one of claims 1 to 6 is implemented.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images