Video processing method, device, electronic device and storage medium

Abstract

The embodiments of the present application provide a video processing method, device, electronic device and storage medium, which relate to the field of video shooting technology and can improve the problem of video recording interruption when the video frame rate changes in different scenarios. The video processing method includes: obtaining a video image captured by a camera; switching the video mode according to the image parameters of the current video image, and switching the video mode according to the image parameters of the current video image includes: if the overexposure ratio of the current video image is greater than the first overexposure ratio threshold, then entering the first video mode; if the overexposure ratio of the current video image is less than the second overexposure ratio threshold, then entering the second video mode; in the first video mode, the camera uses a first frame rate to capture video images, and the video images captured in any two adjacent frames have different exposure times; in the second video mode, the camera uses a third frame rate to capture video images, and the video images captured in any two adjacent frames have the same exposure time.

Description

Technical Field

[0001] The present application relates to the field of video shooting technology, and in particular to a video processing method, device, electronic device and storage medium. Background Art

[0002] With the development of technology, users have increasingly higher requirements for the video effects shot by mobile phones and other terminals. For video shooting, different frame rate requirements are based on different scenarios. However, the current frame rate switching process in different scenarios requires a restart to capture video streams with different frame rates, thus interrupting the recording. Summary of the Invention

[0003] A video processing method, device, electronic device and storage medium can improve the problem of video recording interruption when the video frame rate changes in different scenarios.

[0004] In a first aspect, a video processing method is provided, including: obtaining a video image captured by a camera; switching the video mode according to the image parameters of the current video image, wherein the switching of the video mode according to the image parameters of the current video image includes: if the overexposure ratio of the current video image is greater than a first overexposure ratio threshold, entering the first video mode; if the overexposure ratio of the current video image is less than a second overexposure ratio threshold, entering the second video mode, wherein the second overexposure ratio threshold is less than the first overexposure ratio threshold; in the first video mode, the camera captures video images using a first frame rate, and the video images captured in any two adjacent frames have different exposure times, and the video images in each two adjacent frames are fused into an image of one frame to obtain a video image of a second frame rate; in the second video mode, the camera captures video images using a third frame rate, and the video images captured in any two adjacent frames have the same exposure time.

[0005] In one possible embodiment, if the overexposure ratio of the current video image is greater than the first overexposure ratio threshold, the process of entering the first video mode includes: if the brightness value of the current video image is greater than the first brightness threshold, and the overexposure ratio of the current video image is greater than the first overexposure ratio threshold, then entering the first video mode; if the overexposure ratio of the current video image is less than the second overexposure ratio threshold, then entering the second video mode includes: if the brightness value of the current video image is greater than the first brightness threshold, and the overexposure ratio of the current video image is less than the second overexposure ratio threshold, then entering the second video mode, and the third frame rate is greater than the second frame rate; switching the video mode according to the image parameters of the current video image also includes: if the brightness value of the current video image is less than the second brightness threshold, then entering the third video mode; in the third video mode, the camera uses a fourth frame rate to capture video images, and the video images captured in any two adjacent frames have the same exposure time, and the fourth frame rate is less than the third frame rate. By switching video modes based on different image parameters of the current video, the frame rate can be dynamically adjusted according to the current video shooting scene to adapt to different scenes. For example, when the brightness value of the current video image is greater than the first brightness threshold B1 and the overexposure ratio of the current video image is greater than the first overexposure ratio threshold A1, it is a medium-high brightness HDR scene, and the first video mode is entered to implement HDR video image processing at a lower frame rate (second frame rate); when the brightness value of the current video image is greater than the first brightness threshold B1 and the overexposure ratio of the current video image is less than the second overexposure ratio threshold A2, it is a medium-high brightness non-HDR scene, and the second video mode is entered to implement ordinary video image processing at a higher frame rate (third frame rate), improving video smoothness by shortening the exposure time and increasing the frame rate; when the brightness value of the current video image is less than the second brightness threshold B2, it is a low-light scene, and the third video mode is entered to implement ordinary video image processing at a lower frame rate (fourth frame rate), increasing the exposure time and reducing the frame rate to increase the amount of light entering, thereby improving the picture effect in low-light scenes.

[0006] In one possible implementation, the third frame rate is equal to the first frame rate, and the fourth frame rate is equal to the second frame rate. In this way, regardless of the video mode, switching is performed at a multiple frame rate to improve the jitter problem caused by non-multiple frame rates.

[0007] In a possible implementation, the first overexposure ratio threshold is 25%, and the second overexposure ratio threshold is 20%; the first brightness threshold is 500 nit, and the second brightness threshold is 300 nit.

[0008] In one possible embodiment, before switching the video mode based on the image parameters of the current video image, the process further includes: if the current video image has a preset pattern, switching the video mode based on the preset pattern of the current video image; if the current video image does not have a preset pattern, entering a process of switching the video mode based on the image parameters of the current video image; the process of switching the video mode based on the preset pattern of the current video image includes: if the current video image has a preset pattern of a marquee, entering a second video mode; if the current video image has a preset pattern other than a marquee and the overexposure ratio of the current video image is greater than a first overexposure ratio threshold, entering a first video mode; if the current video image has a preset pattern other than a marquee and the overexposure ratio of the current video image is less than a second overexposure ratio threshold, entering a third video mode. In addition to actual video recording scenarios, the video recording process of an electronic device may also be used in certain testing processes. The testing process may use specific preset patterns for testing. To better match the specific testing process, the video mode may be switched based on the preset pattern, in combination with the scene and overexposure ratio. The preset marquee pattern is a cyclically displayed pattern used to test the video frame rate, such as a cyclically lit light bulb pattern.

[0009] In one possible implementation, if the current video image has no preset pattern, the process of switching the video mode according to the image parameters of the current video image is as follows: if the current video image has no preset pattern and the electronic device is in a handheld state, the process of switching the video mode according to the image parameters of the current video image is entered.

[0010] In a second aspect, a video processing device is provided, comprising: a processor and a memory, wherein the memory is used to store at least one instruction, and when the instruction is loaded and executed by the processor, the above-mentioned video processing method is implemented.

[0011] In a third aspect, an electronic device is provided, comprising: a camera; and the above-mentioned video processing device.

[0012] In a fourth aspect, a computer-readable storage medium is provided, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned video processing method.

[0013] In the video processing method, device, electronic device and storage medium in the embodiments of the present application, in a first video mode, video images of adjacent frames with different exposure times obtained by using a first frame rate are fused into one frame to obtain a video image of a second frame rate, and an HDR video image with a lower frame rate can be obtained. In a second video mode, video images with the same exposure time obtained by using a third frame rate are processed as a video stream, and a non-fused ordinary video image can be obtained. Different video modes can be switched according to the image parameters of the current video image. During the switching process, there is no need to change the settings of the camera when capturing the video image. Therefore, there is no need to restart the camera to capture the video stream, which avoids the problem of video recording interruption when the video frame rate changes in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a structural block diagram of an electronic device according to an embodiment of the present application;

[0015] Figure 2 This is a flow chart of a video processing method according to an embodiment of the present application;

[0016] Figure 3 Schematic diagram of switching conditions of different video modes in an embodiment of the present application;

[0017] Figure 4 This is a flowchart of a video processing method in the first video mode in an embodiment of the present application;

[0018] Figure 5 This is a schematic diagram of capturing a video image in the first video mode in an embodiment of the present application;

[0019] Figure 6 This is a flowchart of a video processing method in the second video mode in an embodiment of the present application;

[0020] Figure 7 This is a schematic diagram of capturing a video image in the second video mode in an embodiment of the present application;

[0021] Figure 8 This is a flowchart of a video processing method in the third video mode in an embodiment of the present application;

[0022] Figure 9 This is a schematic diagram of capturing a video image in the third video mode in an embodiment of the present application;

[0023] Figure 10 This is a flowchart of another video processing method in an embodiment of the present application;

[0024] Figure 11 This is a software structure diagram of an electronic device in an embodiment of the present application;

[0025] Figure 12 This is a schematic diagram of a user interface in movie mode according to an embodiment of the present application;

[0026] Figure 13 This is a schematic diagram of a user interface in a professional mode in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0028] Before introducing the embodiments of the present application, the electronic devices involved in the embodiments of the present application are first introduced. Figure 1 As shown, the electronic device 100 may include a processor 110, a camera 193, a display screen 194, etc. It should be understood that the structure shown in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or combine or separate certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0029] The processor 110 may include one or more processing units. For example, the processor 110 may include a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), etc. The different processing units may be independent devices or integrated into one or more processors. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. The processor 110 may also include a memory for storing instructions and data.

[0030] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0031] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0032] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0033] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0034] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0035] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0036] like Figure 2 and Figure 3 As shown, an embodiment of the present application provides a video processing method. The execution subject of the video processing method may be a processor 110, specifically an ISP or a combination of an ISP and other processors. The video processing method includes:

[0037] Step 101: Acquire a video image captured by a camera;

[0038] Step 102: Switching the video mode according to the image parameters of the current video image. Switching the video mode according to the image parameters of the current video image includes: if the overexposure ratio of the current video image is greater than a first overexposure ratio threshold A1, entering the first video mode; if the overexposure ratio of the current video image is less than a second overexposure ratio threshold A2, entering the second video mode, and the second overexposure ratio threshold A2 is less than the first overexposure ratio threshold A1;

[0039] The overexposure ratio refers to the ratio of the number of pixels in a video image that are greater than a preset pixel value to the total number of pixels. For example, in an 8-bit video image, where each pixel has a value range of 0 to 255 and the preset pixel value is 250, the overexposure ratio is the ratio of the number of pixels with a value greater than 250 to the total number of pixels. For example, if the overexposure ratio of the current video image is greater than a first overexposure ratio threshold A1, the video mode is in the first video mode. If the overexposure ratio of the video image gradually decreases to less than the first overexposure ratio threshold A1 and greater than a second overexposure ratio threshold A2, the video mode will not switch and will remain in the first video mode until the overexposure ratio of the video image decreases to less than the second overexposure ratio threshold A2, at which point the video mode will switch to the second video mode. Similarly, in the second video mode, if the overexposure ratio of the video image gradually increases to less than the first overexposure ratio threshold A1 and greater than the second overexposure ratio threshold A2, the video mode will not switch and will remain in the second video mode until the overexposure ratio of the video image increases to greater than the first overexposure ratio threshold A1, at which point the video mode will switch to the first video mode. By using such switching logic, ping-pong switching between different video modes under a single threshold can be avoided. It should be noted that the embodiment of the present application does not limit the initial video mode judgment logic. For example, an initial overexposure ratio threshold value can be set between the first overexposure ratio threshold value A1 and the second overexposure ratio threshold value A2. Before video recording, if the overexposure ratio of the video image in the preview screen is greater than the initial overexposure ratio threshold value, the first video mode is entered. If the overexposure ratio of the video image in the preview screen is not greater than the initial overexposure ratio threshold value, the second video mode is entered.

[0040] like Figure 4 and Figure 5 As shown, in the first video mode, the camera captures video images using a first frame rate, and the video images captured in any two adjacent frames have different exposure times. The video images in each of the two adjacent frames are fused into an image of one frame to obtain a video image with a second frame rate;

[0041] For example, the electronic device may include a camera 193, a global tone mapping Mapping (GTM) module 20, first deformation module 31, second deformation module 32, third deformation module 33, fourth deformation module 34, first fusion module 41, second fusion module 42, first gamma module 51 and second gamma module 52. In the first video mode, assuming that the first frame rate is 60 FPS, that is, the camera 193 uses a frame rate of 60 FPS to capture video images, and alternately captures a first exposure frame video image F1 and a second exposure frame video image F2. The exposure time of any two adjacent frames of video images is different, that is, the exposure time of F1 and F2 is different, and the exposure time of F1 is greater than the exposure time of F2. For example, 33 ms is the length of the two frames, including F1 with a longer exposure time and F2 with a shorter exposure time. The GTM module 20 performs GTM processing on the video images captured by the camera 193. The video images processed by GTM are split into two streams, one of which is a video stream and the other is a preview stream. In the video stream, the first exposure frame video image F1 is electronically stabilized by the first deformation module 31. The second exposure frame video image F2 is subjected to EIS processing by the second deformation module 32. The first exposure frame video image F1 and the second exposure frame video image F2 after the EIS processing are subjected to high dynamic range (HDR) fusion processing in the first fusion module 41. The first exposure frame video image F1 and the second exposure frame video image F2 in each two adjacent frames are fused into one frame image. In this way, after the fusion, the frame rate is reduced to half of the original, that is, it becomes a 30FPS image. The fused video image is gamma mapped by the first gamma module 51. The image of the processed video stream can be saved as a video file, that is, the 30FPS HDR video stream processing process is realized, that is, the second frame rate of the video image is 30FPS;In the preview stream, similar to the video stream processing, the first exposure frame video image F1 undergoes EIS processing via the third warping module 33, while the second exposure frame video image F2 undergoes EIS processing via the fourth warping module 34. It should be noted that since the preview stream is used only for previewing, the EIS processing performed in the preview stream can differ from that performed in the video stream. For example, a higher version of EIS processing can be performed in the video stream, while a lower version of EIS processing can be performed in the preview stream. The first exposure frame video image F1 and the second exposure frame video image F2 in the preview stream, after undergoing EIS processing, undergo HDR fusion processing in the second fusion module 42. The first exposure frame video image F1 and the second exposure frame video image F2 in each adjacent frame are fused into a single frame. This fusion reduces the frame rate to half of the original rate, to a 30 FPS image. The fused video images undergo gamma mapping via the second gamma module 52. The processed video stream images can then be previewed, thus achieving a 30 FPS HDR preview stream processing process.

[0042] like Figure 6 and Figure 7 As shown, in the second video mode, the camera captures video images using a third frame rate, and the video images captured in any two adjacent frames have the same exposure time.

[0043] For example, in the second video mode, assuming that the third frame rate is 60 FPS, that is, the camera 193 uses a frame rate of 60 FPS to capture video images, and each frame of the captured exposure frame video image F has an equal exposure time, for example, 33 ms is the length of two frames, which includes two exposure frame video images F with equal exposure times; the GTM module 20 performs GTM processing on the video image captured by the camera 193, and the video image processed by the GTM is split into two streams, one of which is a video stream and the other is a preview stream. In the video stream, the exposure frame video image F is electronically image-stabilized (Electric Image Stabilization) by the first deformation module 31. After the EIS processing, the exposure frame video image F no longer passes through the first fusion module 41, but is gamma mapped by the first gamma module 51. The image of the video stream after the processing can be saved as a video file, that is, the ordinary video stream processing process of 60FPS is realized, and a video image of 60FPS is obtained. In addition, in the preview stream, similar to the processing process of the video stream, the exposure frame video image F is EIS processed by the third deformation module 33. It should be noted that since the preview stream is only used for preview, the EIS processing performed in the preview stream can be different from the EIS processing performed in the video stream. For example, a higher version of EIS processing is performed in the video stream, and a lower version of EIS processing is performed in the preview stream. The exposure video image F in the preview stream after EIS processing bypasses the second fusion module 42, and then passes through the second gamma module 52 for gamma mapping. The image of the video stream after the processing can be previewed, that is, the preview stream processing process of 60FPS is realized. In other feasible embodiments, the third frame rate can be adjusted between the first frame rate and the second frame rate. For example, assuming that the first frame rate is 60FPS and the second frame rate is 30FPS, the third frame rate can be 30FPS, 60FPS, or between 30FPS and 60FPS.

[0044] The video processing method in the embodiment of the present application, in the first video mode, the video images of adjacent frames with different exposure times captured using the first frame rate are fused into one frame to obtain a video image of the second frame rate, and an HDR video image with a lower frame rate can be obtained. In the second video mode, the video images with the same exposure time captured using the third frame rate are processed as a video stream to obtain a non-fused ordinary video image, and different video modes can be switched according to the image parameters of the current video image. During the switching process, there is no need to change the settings of the camera when capturing the video image. Therefore, there is no need to restart the camera to capture the video stream, which avoids the problem of recording interruption when the video frame rate changes in different scenarios.

[0045] In one possible implementation, Figure 3As shown, the process of entering the first video mode if the overexposure ratio of the current video image is greater than the first overexposure ratio threshold A1 includes: if the brightness value of the current video image is greater than the first brightness threshold B1, and the overexposure ratio of the current video image is greater than the first overexposure ratio threshold A1, entering the first video mode;

[0046] If the overexposure ratio of the current video image is less than the second overexposure ratio threshold A2, the process of entering the second video mode includes: if the brightness value of the current video image is greater than the first brightness threshold B1, and the overexposure ratio of the current video image is less than the second overexposure ratio threshold A2, entering the second video mode, and the third frame rate is greater than the second frame rate;

[0047] Switching the video mode according to the image parameters of the current video image also includes:

[0048] If the brightness value of the current video image is less than the second brightness threshold B2, then enter the third video mode;

[0049] like Figure 8 and Figure 9 As shown, in the third video mode, the camera captures video images using a fourth frame rate, and the video images captured in any two adjacent frames have the same exposure time, and the fourth frame rate is less than the third frame rate.

[0050] For example, in the third video mode, assuming that the fourth frame rate is 30FPS, that is, the camera 193 uses a frame rate of 30FPS to capture video images, and each frame of the captured exposure frame video image F has an equal exposure time, for example, 33ms is the length of one frame time, that is, a video image F with a normal frame rate of 30FPS is obtained; the GTM module 20 performs GTM processing on the video image F captured by the camera 193, and the video image processed by GTM is split into two streams, one of which is a video stream and the other is a preview stream. In the video stream, the video image F is electronically anti-shake (Electric Image Stabilization) by the first deformation module 31. After the EIS processing, the video image F is processed by the first gamma module 51 for gamma mapping, and the image of the video stream after the processing can be saved as a video file, that is, the ordinary video stream processing process of 30FPS is realized, and an ordinary video image with a frame rate of 30FPS is obtained; in addition, in the preview stream, similar to the processing process of the video stream, the video image F is processed by the third deformation module 33 for EIS. It should be noted that since the preview stream is only used for preview, the EIS processing performed in the preview stream can be different from the EIS processing performed in the video stream. For example, a higher version of EIS processing is performed in the video stream, and a lower version of EIS processing is performed in the preview stream. The video image F in the preview stream after EIS processing bypasses the second fusion module 42, and then undergoes gamma mapping processing by the second gamma module 52. The image of the video stream after the processing can be previewed, that is, the preview stream processing process of 30FPS is realized.

[0051] It should also be noted that the embodiment of the present application does not limit the initial video mode judgment logic. For example, an initial brightness threshold can be set between the first brightness threshold B1 and the second brightness threshold B2. Before video recording, if the brightness of the video image in the preview screen is greater than the initial brightness threshold, the first video mode or the second video mode is entered, and the judgment is made specifically in combination with the overexposure ratio. If the brightness of the video image in the preview screen is not greater than the initial brightness threshold, the third video mode is entered.

[0052] By switching video modes based on the different image parameters of the current video, the frame rate can be dynamically adjusted according to the current video shooting scene to adapt to different scenarios. For example, if the brightness value of the current video image is greater than a first brightness threshold B1 and the overexposure ratio of the current video image is greater than a first overexposure ratio threshold A1, it is a medium-high-brightness HDR scene. Therefore, the first video mode is entered, implementing HDR video image processing at a lower frame rate (second frame rate). If the brightness value of the current video image is greater than the first brightness threshold B1 and the overexposure ratio of the current video image is less than a second overexposure ratio threshold A2, it is a medium-high-brightness non-HDR scene. Therefore, the second video mode is entered, implementing normal video image processing at a higher frame rate (third frame rate). Video smoothness is improved by shortening the exposure time and increasing the frame rate. If the brightness value of the current video image is less than the second brightness threshold B2, it is a low-brightness scene. Therefore, the third video mode is entered, implementing normal video image processing at a lower frame rate (fourth frame rate). By extending the exposure time and reducing the frame rate, the amount of light entering is increased, improving the image quality in low-brightness scenes. Switching between different video modes is smooth, avoiding frequent ping-pong switching.

[0053] In one possible implementation, the third frame rate is equal to the first frame rate, and the fourth frame rate is equal to the second frame rate. For example, the first and fourth frame rates are 60 FPS, and the second and third frame rates are 30 FPS. In this way, regardless of the video mode, switching is performed at multiple frame rates to improve jitter caused by non-multiple frame rates.

[0054] In a possible implementation, the first overexposure ratio threshold A1 is 25%, the second overexposure ratio threshold A2 is 20%; the first brightness threshold B1 is 500 nit, and the second brightness threshold B2 is 300 nit.

[0055] In one possible implementation, Figure 10 As shown, before step 102, switching the video mode according to the image parameters of the current video image, the method further includes:

[0056] Step 103: Determine whether the current video image has a preset pattern. If so, that is, if the current video image has a preset pattern, proceed to step 104: Switch the video mode according to the preset pattern of the current video image. If not, that is, if the current video image does not have a preset pattern, proceed to step 102: Switch the video mode according to the image parameters of the current video image.

[0057] Step 104, the process of switching the video mode according to the preset pattern of the current video image includes:

[0058] If the current video image has a preset marquee pattern, the second video mode is entered;

[0059] If the current video image has a preset pattern other than a marquee pattern, and the overexposure ratio of the current video image is greater than a first overexposure ratio threshold, then entering the first video mode;

[0060] If the current video image has a preset pattern other than a marquee pattern, and the overexposure ratio of the current video image is less than a second overexposure ratio threshold, the third video mode is entered.

[0061] Specifically, in addition to the actual video shooting scenes, the video recording process of the electronic device may also be used in certain testing processes. The testing process will use specific preset patterns for testing. In order to better match the specific test process, the video mode can be switched based on the preset pattern, combined with the scene and overexposure ratio. The preset pattern of the marquee is a cyclically displayed pattern used to test the video frame rate, such as a cyclically lit light bulb pattern.

[0062] In one possible embodiment, if the current video image does not have a preset pattern, the process of switching the video mode according to the image parameters of the current video image is as follows: if the current video image does not have a preset pattern and the electronic device is in a handheld state, then the process of switching the video mode according to the image parameters of the current video image is entered into step 102. The handheld state can be determined based on data from sensors such as a gyroscope in the electronic device. Since the electronic device cannot remain stable in the handheld state, if the electronic device is determined to be shaking according to relevant sensors, it is determined to be in the handheld state. In this state, the above-mentioned step 102 is entered. In the handheld state, the video can be made smoother by dynamically adjusting the frame rate in the embodiment of the present application.

[0063] In one possible embodiment, the electronic device includes a main camera, a telephoto camera, and an ultra-wide-angle camera. If the current camera is the main camera and is in a non-zoom state, that is, in a single zoom state, then step 102 is entered, that is, dynamic frame rate adjustment is performed. If the current camera is the main camera and is in a non-single zoom state, or the current camera is not the main camera, a fixed frame rate video mode is used.

[0064] The following describes an embodiment of the present application in conjunction with a software architecture. The embodiment of the present application takes the Android system with a layered architecture as an example to exemplify the software structure of the electronic device 100. Figure 11 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0065] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers: application layer, framework layer, system library layer, hardware abstraction layer (HAL), and kernel layer.

[0066] The application layer can include applications such as the camera.

[0067] The application framework layer may include the camera application programming interface (API), media recorder (MediaRecorder), and surface view (SurfaceView). Media recorder is used to record video or image data and make it accessible to applications. Surface view is used to display preview images.

[0068] The system library can include multiple functional modules, such as the camera service CameraSevice.

[0069] The hardware abstraction layer is used to provide interface support, such as including the camera process CameraPipeline for camera service calls.

[0070] The kernel layer is the layer between hardware and software. The kernel layer includes display drivers, camera drivers, etc.

[0071] In a specific video capture scenario, the application layer issues a capture request (CaptureRequest) corresponding to a recorded stream and a preview stream. It also creates a media codec instance (mediacodec) to receive the encoded recorded stream. The HAL callbacks the two streams according to the aforementioned data flow. The preview stream is sent to the display, and the recorded stream is sent to mediacodec.

[0072] The video recording processing method provided in the embodiment of the present application can be expressed as multiple functions under two shooting modes, where the two shooting modes can be: movie mode and professional mode.

[0073] Movie mode is a shooting mode related to movie themes. In this mode, the image displayed by the electronic device 100 can give the user a sensory experience of watching a movie. The electronic device 100 also provides multiple video style templates related to movie themes. Users can use these video style templates to obtain tonal-adjusted images or videos. The tones of these images or videos are similar to or the same as the tones of the movie. In the following embodiments of the present application, the movie mode can at least provide an interface for the user to trigger the LUT function and the HDR10 function. For a specific description of the LUT function and the HDR10 function, please refer to the following embodiments.

[0074] For example, assuming that the electronic device 100 is a mobile phone, in a possible implementation, as Figure 12 As shown, the electronic device can enter the movie mode in response to the user's operation. For example, the electronic device 100 can detect the user's touch operation on the camera application. In response to the operation, the electronic device 100 displays the default camera interface of the camera application. The default camera interface may include: a preview box, a shooting mode list, a gallery shortcut key, a shutter control, etc. Among them:

[0075] The preview frame can be used to display the image captured in real time by the camera 193. The electronic device 100 can refresh the display content therein in real time so that the user can preview the image currently captured by the camera 193.

[0076] One or more shooting mode options may be displayed in the shooting mode list. These one or more shooting mode options may include: a portrait mode option, a video mode option, a photo mode option, a movie mode option, and a professional option. These one or more shooting mode options may be displayed as text on the interface, such as "portrait," "video," "photo," "movie," and "professional." Without limitation, these one or more shooting mode options may also be displayed as icons or other interactive elements (IEs) on the interface.

[0077] The Gallery shortcut key can be used to open the Gallery application. The Gallery application is an image management application on electronic devices such as smartphones and tablets. It can also be called an "album." This embodiment does not limit the name of this application. The Gallery application allows users to perform various operations on images stored on electronic device 100, such as browsing, editing, deleting, and selecting.

[0078] The shutter control can be used to monitor user actions that trigger a photo. The electronic device 100 can detect user actions on the shutter control and, in response to the actions, save the image in the preview frame as a picture in the gallery application. Furthermore, the electronic device 100 can also display thumbnails of the saved images in the gallery shortcut. In other words, the user can click the shutter control to trigger a photo. The shutter control can be a button or other form of control.

[0079] The electronic device 100 can detect the user's touch operation on the movie mode option, and in response to the operation, the electronic device displays the following Figure 12 The user interface shown.

[0080] In some embodiments, the electronic device 100 may enable movie mode by default after launching the camera application. Without limitation, the electronic device 100 may also enable movie mode in other ways, such as enabling movie mode in response to a user's voice command, which is not limited in this embodiment of the present application.

[0081] The electronic device 100 can detect the user's touch operation on the movie mode option, and in response to the operation, the electronic device displays the following Figure 12 The user interface shown.

[0082] like Figure 12 The user interface shown includes function options, including HDR10, flash, LUT, and settings. These multiple function options can detect user touch operations and, in response to the user's touch operation, turn the corresponding shooting function on or off, such as HDR10, flash, LUT, and settings.

[0083] Electronic devices can enable a Lookup Table (LUT) function, which can change the display quality of preview images. Essentially, the LUT function introduces a color lookup table (LUT), which acts as a color conversion model. This color conversion model outputs adjusted color values ​​based on input color values. The color values ​​of the image captured by the camera are equivalent to the input values. Different color values, after passing through the color conversion model, each result in a corresponding output value. Ultimately, the image displayed in the preview frame is the image adjusted by the color conversion model. Electronic device 100 utilizes this LUT function to display an image composed of color values ​​adjusted by the color conversion model, achieving the effect of adjusting the image's hue. After enabling the LUT function, electronic device 100 can provide multiple video style templates, each corresponding to a color conversion model. Different video style templates can produce different display effects for preview images. Furthermore, these video style templates can be associated with movie themes. The hue adjustment effect achieved by these video style templates can be close to or identical to the hues in the movie, creating a film-like atmosphere for the user.

[0084] In addition, after the electronic device 100 turns on the LUT function, the electronic device 100 can determine a video style template from multiple video style templates based on the current preview video screen, and the determined video style template can be displayed in the interface to facilitate the user to understand the currently determined video style template. For example, multiple video style templates include "A" movie style template, "B" movie style template and "C" movie style template. The LUTs corresponding to different movie style templates can be pre-generated based on the corresponding movie color style, and the color conversion of the LUT has the style characteristics of the corresponding movie. It can be extracted from the movie style in advance to generate a LUT suitable for mobile electronic devices. Turning on the LUT function will change the color tone of the preview video screen. Figure 12 As shown in FIG, the electronic device 100 determines the movie style template "A" and displays it.

[0085] In some embodiments, the electronic device 100 can select a video style template based on a user's sliding operation. Specifically, when the electronic device 100 detects a user operation to turn on the LUT function and displays the LUT preview window, the electronic device 100 can select the first video style template located in the LUT preview window by default as the video style template selected by the electronic device 100. Thereafter, the electronic device 100 can detect the user's left and right sliding operation on the LUT preview window and move the positions of the video style templates in the LUT preview window. When the electronic device 100 no longer detects the user's sliding operation, the electronic device 100 uses the first video style template displayed in the LUT preview window as the video style template selected by the electronic device 100.

[0086] In some embodiments, in addition to using a video style template to change the display effect of a preview image, the electronic device 100 may also detect a user operation to start recording a video after adding the video style template. In response to this operation, the electronic device 100 starts recording the video, thereby obtaining a video with the display effect adjusted using the video style template. In addition, during the video recording process, the electronic device 100 may also detect a user operation to take a photo. In response to this operation, the electronic device 100 saves the preview image with the video style template added to the preview frame as a picture, thereby obtaining an image with the display effect adjusted using the video style template.

[0087] Electronic devices can turn on the HDR10 function. In HDR10 mode, HDR stands for High-Dynamic Range (HDR). Compared with ordinary images, HDR can provide more dynamic range and image details, and can better reflect the visual effects in the real environment. The 10 in HDR10 stands for 10 bits, and HDR10 can record videos with a 10-bit high dynamic range.

[0088] The electronic device 100 can detect the user's touch operation on the professional mode option and enter the professional mode. Figure 13 As shown, when the electronic device is in professional mode, the function options that may be included in the user interface include, for example: LOG option, flash option, LUT option, setting option. In addition, the user interface also includes parameter adjustment options, such as: metering M option, ISO option, shutter S option, exposure compensation EV option, focus mode AF option and white balance WB option.

[0089] In some embodiments, the electronic device 100 may enable the professional mode by default after launching the camera application. Without limitation, the electronic device 100 may also enable the professional mode in other ways, such as by activating the professional mode in response to a user's voice command, which is not limited in this embodiment of the present application.

[0090] Electronic device 100 can detect a user operation on the LOG option and, in response to the operation, enable the LOG function. The LOG function applies a logarithmic function to the exposure curve, maximizing the preservation of detail in highlights and shadows in images captured by the camera, resulting in a lower saturation in the resulting preview image. Videos recorded using the LOG function are referred to as LOG videos.

[0091] In addition to recording videos with added video style templates in Professional Mode, the electronic device 100 can also add a video style template to a video that has not been added after recording it. Alternatively, after turning on the LOG function, record a LOG video and then add a video style template to the LOG video. This allows the electronic device 100 to adjust the display effect of the screen not only before recording a video, but also after the video recording is completed, increasing the flexibility and freedom of image adjustment.

[0092] An embodiment of the present application also provides a video processing device, including: a video acquisition module, used to acquire video images captured by a camera; a switching module, used to switch the video mode according to the image parameters of the current video image, and switching the video mode according to the image parameters of the current video image includes: if the overexposure ratio of the current video image is greater than a first overexposure ratio threshold, then entering the first video mode; if the overexposure ratio of the current video image is less than a second overexposure ratio threshold, then entering the second video mode, and the second overexposure ratio threshold is less than the first overexposure ratio threshold; in the first video mode, the camera uses a first frame rate to capture video images, and the video images captured in any two adjacent frames have different exposure times, and the video images in each two adjacent frames are fused into an image of one frame to obtain a video image of the second frame rate; in the second video mode, the camera uses a third frame rate to capture video images, and the video images captured in any two adjacent frames have the same exposure time.

[0093] It should be understood that the division of the various modules of the above video processing device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a single physical entity, or physically separated. Furthermore, these modules can be implemented entirely in the form of software called through a processing element; or entirely in the form of hardware; or some modules can be implemented in the form of software called through a processing element, and some modules can be implemented in the form of hardware. For example, either the video acquisition module or the switching module can be a separate processing element, or integrated into the video processing device, such as being implemented in a chip of the video processing device. In addition, they can be stored in the memory of the video processing device in the form of a program, and called and executed by a processing element of the video processing device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed by an integrated logic circuit in the hardware of the processor element or by instructions in the form of software.

[0094] For example, the video acquisition module and the switching module may be one or more integrated circuits configured to implement the above method, such as one or more application-specific integrated circuits (ASICs), one or more digital singnal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when one of the above modules is implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call programs. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0095] An embodiment of the present application further provides a video processing device, including: a processor and a memory, the memory being used to store at least one instruction, which is loaded and executed by the processor to implement the video processing method in any of the above embodiments.

[0096] The video processing device can apply the above-mentioned video processing method, and the specific process and principle will not be repeated here.

[0097] The number of processors may be one or more, and the processor and memory may be connected via a bus or other means. The memory, as a non-transitory computer-readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the video processing device in the embodiment of the present application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions and modules stored in the memory, that is, implements the method in any of the above method embodiments. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; and necessary data, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device.

[0098] like Figure 1 As shown, an embodiment of the present application further provides an electronic device, including: a camera 193 and the above-mentioned video processing device, and the video processing device includes a processor 110.

[0099] The specific principle and working process of the video processing device are the same as those in the above embodiment and will not be described in detail here. The electronic device can be any product or component with a video shooting function, such as a mobile phone, a TV, a tablet computer, a watch, a bracelet, etc.

[0100] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer is enabled to execute the video processing method in any of the above embodiments.

[0101] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0102] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A video processing method, characterized in that: include: Get the video image captured by the camera; Switching the video mode according to the image parameters of the current video image, wherein the switching the video mode according to the image parameters of the current video image includes: If the overexposure ratio of the current video image is greater than a first overexposure ratio threshold, the first video mode is entered; if the overexposure ratio of the current video image is less than a second overexposure ratio threshold, the second video mode is entered, and the second overexposure ratio threshold is less than the first overexposure ratio threshold; In the first video mode, the camera captures video images using a first frame rate, the video images captured in any two adjacent frames have different exposure times, and the video images in each of the two adjacent frames are fused into an image of one frame to obtain a video image at a second frame rate; In the second video mode, the camera captures video images using a third frame rate, and the video images captured in any two adjacent frames have the same exposure time; The third frame rate is equal to the first frame rate.

2. The video processing method according to claim 1, wherein: If the overexposure ratio of the current video image is greater than the first overexposure ratio threshold, the process of entering the first video mode includes: If the brightness value of the current video image is greater than the first brightness threshold, and the overexposure ratio of the current video image is greater than the first overexposure ratio threshold, entering the first video mode; If the overexposure ratio of the current video image is less than the second overexposure ratio threshold, the process of entering the second video mode includes: If the brightness value of the current video image is greater than the first brightness threshold, and the overexposure ratio of the current video image is less than the second overexposure ratio threshold, then entering the second video mode, and the third frame rate is greater than the second frame rate; The switching of the video mode according to the image parameters of the current video image further includes: If the brightness value of the current video image is less than the second brightness threshold, then enter the third video mode; In the third video mode, the camera captures video images using a fourth frame rate, and video images captured in any two adjacent frames have the same exposure time, and the fourth frame rate is less than the third frame rate.

3. The video processing method according to claim 2, wherein: The fourth frame rate is equal to the second frame rate.

4. The video processing method according to claim 2, wherein: The first overexposure ratio threshold is 25%, and the second overexposure ratio threshold is 20%; The first brightness threshold is 500 nit, and the second brightness threshold is 300 nit.

5. The video processing method according to claim 2, wherein: Before switching the video mode according to the image parameters of the current video image, the method further includes: If the current video image has a preset pattern, the video mode is switched according to the preset pattern of the current video image; if the current video image does not have a preset pattern, the process of switching the video mode according to the image parameters of the current video image is entered; The process of switching the video mode according to the preset pattern of the current video image includes: If the current video image has a preset marquee pattern, then entering the second video mode; If the current video image has a preset pattern other than a marquee pattern, and the overexposure ratio of the current video image is greater than the first overexposure ratio threshold, entering the first video mode; If the current video image has a preset pattern other than horse racing, and the overexposure ratio of the current video image is less than the second overexposure ratio threshold, the third video mode is entered.

6. The video processing method according to claim 5, characterized in that: If the current video image does not have a preset pattern, the process of switching the video mode according to the image parameters of the current video image is as follows: If the current video image does not have a preset pattern and the electronic device is in a handheld state, the process of switching the video mode according to the image parameters of the current video image is entered.

7. A video processing device, characterized in that: include: A processor and a memory, wherein the memory is used to store at least one instruction, and when the instruction is loaded and executed by the processor, the video processing method according to any one of claims 1 to 6 is implemented.

8. An electronic device, characterized in that: include: Camera; The video processing device according to claim 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the video processing method according to any one of claims 1 to 6.

Images