Video processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a video processing method and device, electronic equipment and a storage medium, and belongs to the technical field of video processing. The method comprises the following steps: acquiring an original video and video parameter information of video recording; and after video recording is completed and a target condition is satisfied, processing the original video based on the video parameter information. The video processing operation can be executed on the original video based on the video parameter information under the condition that the video recording of the electronic equipment is completed and the target condition is met, so that the problem of video lagging and frame loss caused by forcibly executing video processing under the condition that the state parameter of the electronic equipment is poor can be avoided, and the video processing efficiency is improved. And the display effect of the recorded video is improved.

Description

Technical Field

[0001] The present application belongs to the field of video processing technology, and specifically relates to a video processing method, device, electronic device and storage medium. Background Art

[0002] Currently, most electronic devices are provided with an image acquisition unit, and images can be acquired through the image acquisition unit. For example, a video can be recorded using an electronic device such as a smart phone with a camera.

[0003] In related technologies, software algorithms similar to deep learning are usually used to simulate recorded videos so that mobile devices can achieve the same video recording effect as a camera. However, using software algorithm simulation will occupy the device's operating resources, increase the device's operating power consumption, and cause the device's recorded video to experience freezes and frame drops. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video processing method that can solve the problem in the prior art that using software algorithm simulation to record video will occupy the operating resources of the device, increase the operating power consumption of the device, cause the video recorded by the device to appear stuck and frame dropped, and cause the battery power to decrease rapidly.

[0005] In a first aspect, an embodiment of the present application provides a video processing method, which is applied to an electronic device, and the method includes:

[0006] Get the original video and video parameter information of the video recording;

[0007] After the video recording is completed and the target condition is met, the original video is processed based on the video parameter information.

[0008] In a second aspect, an embodiment of the present application provides a video processing device, the device comprising:

[0009] An acquisition module is used to acquire the original video and video parameter information of the video recording;

[0010] The processing module is used to process the original video based on the video parameter information after the video recording is completed and the target condition is met.

[0011] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0012] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0013] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0014] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0015] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0016] By obtaining the original video and video parameter information of the video recording; after the video recording is completed and the target conditions are met, the original video is processed based on the video parameter information. In this way, when the video recorded by the electronic device is obtained, it can be determined that the video recording of the electronic device is completed and meets the target conditions, and the video processing operation is performed on the original video based on the video parameter information. In this way, the problem of video freeze and frame loss caused by forcibly performing video processing when the state parameters of the electronic device are not good can be avoided, and the display effect of the recorded video is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of processing a video provided in the background technology of this application;

[0018] Figure 2 is a flow chart of a video processing method provided by an embodiment of the present application;

[0019] Figure 3 is a flowchart of another video processing method provided by an embodiment of the present application;

[0020] Figure 4 is a flowchart of another video processing method provided by an embodiment of the present application;

[0021] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0022] Figure 6 It is a structural diagram of a scheduling control module provided in an embodiment of the present application;

[0023] Figure 7 is a block diagram of a video processing device provided in an embodiment of the present application;

[0024] Figure 8 is a block diagram of an electronic device provided in an embodiment of the present application;

[0025] Fig. 9 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0028] Before introducing the video processing method, device, electronic device and storage medium provided by the present disclosure, the application scenarios involved in each embodiment of the present disclosure are first introduced. The present disclosure can be applied to the video recording scenario of the electronic device, and the video processing method provided by the embodiment of the present disclosure can be applied to a device with an image acquisition function. Among them, the electronic device may include a mobile phone, a camera, a video camera and a tablet computer, etc., and these electronic devices may be provided with corresponding image acquisition units, such as a camera, a lens, etc., and the corresponding image acquisition and video recording, etc. can be performed through the image acquisition unit.

[0029] At present, most electronic devices are equipped with image acquisition units, and images can be acquired through the image acquisition units, for example, electronic devices such as smart phones with cameras can be used to record videos. Among them, in the camera system of modern smart phones, in order to achieve a portrait depth of field blur effect similar to that of a SLR camera during video recording, smart phones usually use software algorithms similar to deep learning to simulate the recorded video, realize video processing effects such as intelligent recognition and background blur, to highlight the main body of the person, and thus achieve the same recording effect as a camera.

[0030] However, when a software algorithm similar to deep learning is used to simulate a recorded video in the related art, the method of processing the recorded video while recording the video is usually adopted, such as Figure 1 As shown, when the original signal data is obtained through the mobile phone sensor, such as the photosensitive element, the original signal data will be transmitted to the ISP module (Image Signal Processor). The ISP module will perform post-processing on the original signal data output by the front-end mobile phone sensor, including linear correction, noise removal, bad pixel removal, interpolation, white balance, automatic exposure control and other processing.

[0031] For example, the original signal data can first be processed by RAW (RAW Image Format) to obtain RAW data. Specifically, the light signal can be converted into an electrical signal through a photosensitive chip, and the level of the output of the photosensitive chip is recorded as RAW data. Then, the obtained RAW data can be converted into YUV (Luminance Chrominance) data. Specifically, the RAW data can be passed through a Bayer filter to obtain a generated color pixel array, and then a demosaicing algorithm can be used to convert the color pixel array into RGB (Red-Green-Blue) data. Then, a preset conversion algorithm can be used to convert each pixel point of the RGB data into YUV brightness and chromaticity information to obtain the final YUV data. Then, segmentation calculation and depth calculation can be performed on the YUV data to obtain video analysis information, which may include segmentation information and depth information.

[0032] The obtained YUV data and video analysis information can then be input into the deep learning software algorithm to simulate the recorded video and achieve video processing effects such as intelligent recognition and blurred background. When the video processing is completed, the processed video is encoded and stored.

[0033] However, in this implementation mode, if a certain frame rate is required for the recorded video, the complexity of the software algorithm used will be limited. If a better video processing effect is required, the frame rate of the recording will be limited to ensure that more time can be reserved for algorithm processing. Therefore, most of the related technologies can only achieve a recording effect of 1080P resolution 30FPS (Frames PerSecond), which will result in poor display effect of the recorded video.

[0034] Also, during the recording process, the use of algorithms requires the use of hardware such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit), which will compete for resources with the original encoding work during video recording, causing the recorded video to become stuck and have frames dropped.

[0035] In order to solve the above problems, the present disclosure provides a video processing method, device, electronic device and storage medium, which can determine the moment when the electronic device enters a scene with video processing computing power when obtaining a video recorded by the electronic device, and perform video processing operations at that moment. In this way, the problem of video freeze and frame loss caused by forcibly executing video processing when the state parameters of the electronic device are not good can be avoided, thereby improving the display effect of the recorded video.

[0036] The video processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0037] Figure 2 is a flow chart of a video processing method provided by an embodiment of the present application, such as Figure 2 As shown, the method is applied to electronic devices, which may include mobile phones, cameras, camcorders, tablet computers, etc. These electronic devices may be provided with corresponding image acquisition units, such as cameras, lenses, etc., and corresponding image acquisition and video recording may be performed through the image acquisition units.

[0038] The method may include the following steps.

[0039] In step S101, the original video and video parameter information of the video recording are obtained.

[0040] Among them, the original video can be the original signal data obtained through a mobile phone sensor, such as a photosensitive element, which has not been processed by any algorithm, and the video parameter information can be the information obtained by post-processing the original signal data output by the front-end mobile phone sensor through the ISP module.

[0041] For example, the original signal data (i.e., the original video) can first be subjected to RAW processing to obtain RAW data. Specifically, the light signal can be converted into an electrical signal through a photosensitive chip, and the level output by the photosensitive chip is recorded as RAW data. The obtained RAW data can then be converted into YUV data. Specifically, the RAW data can be passed through a Bayer filter to obtain a generated color pixel array, and then a demosaicing algorithm can be used to convert the color pixel array into RGB data. Then, a preset conversion algorithm can be used to convert each pixel point of the RGB data into YUV brightness and chromaticity information to obtain the final YUV data. Then, segmentation calculation and depth calculation can be performed on the YUV data to obtain video parameter information, which may include segmentation information and depth information.

[0042] In practical applications, the objects of video recording may be portraits, animals, natural scenery, etc. Generally speaking, when recording, the user can also select a corresponding recording mode for recording, such as time-lapse photography, slow motion, etc. Videos shot in different recording modes may have different viewing effects. In some possible implementations, different recording modes may also correspond to different software algorithms for video processing in subsequent steps. When starting recording, the user can start recording by clicking a corresponding shooting button (such as a virtual button or a physical button) in the electronic device, or the user can also use a remote method (such as a wireless signal) to control the electronic device video recording, and the embodiments of the present invention do not impose specific restrictions on this.

[0043] During the video recording process, the electronic device can record the object being filmed in real time through the mobile phone sensor, and can also store the filmed video in a storage unit in real time. After recording for a period of time, the user of the electronic device determines that the filming is completed, that is, clicks the "screen recording end shortcut key" to end the recording of the current filmed object, thereby obtaining the corresponding video.

[0044] In step S102, after the video recording is completed and the target condition is met, the original video is processed based on the video parameter information.

[0045] The target condition is that the electronic device enters a scene with video processing computing power, or that the electronic device enters a scene selected by the user by inputting a video processing operation.

[0046] In this step, the state parameters of the electronic device can be first obtained through the sensor, and then in the control module, the scene in which the electronic device is located can be determined based on the state parameters; and when the scene is the target scene, it is determined that the electronic device meets the target conditions.

[0047] Among them, the scene where the target is located can represent a scene in which the electronic device has target device performance parameters, the target device performance parameters represent the device performance parameters of the electronic device when it has video processing computing power, and the scene where the target is located can also represent the scene selected by the user by inputting a video processing operation.

[0048] For example, when it is determined that the device performance parameter of the electronic device is a target device performance parameter, it is determined that the electronic device meets the target condition, and when the electronic device meets the target condition, the original video is processed based on the video parameter information.

[0049] In some embodiments, when the original video recorded by the electronic device is obtained, video parameter information of the original video is obtained through an image signal processor module, and the original video is saved in a storage module, and the video parameter information is saved in the storage module.

[0050] And after the video recording is completed and the target conditions are met, the original video and the video parameter information can be obtained from the storage module through the control module; the original video is then decoded through the video codec module to obtain multiple single-frame images; the multiple single-frame images and the video parameter information are used as inputs of the software algorithm module to obtain multiple target images output by the software algorithm module; and then the multiple target images are re-encoded through the video codec module to obtain a target video, which is a new video after processing the original video.

[0051] Optionally, when the original video is saved in the storage module, the recorded original video can be obtained and the video codec module can be used to encode the original video according to a preset coding length to obtain an encoded video; and then the encoded video is saved in the storage module.

[0052] For example, when receiving data from the ISP module for raw video encoding, the reduced encoded GOP (Group of Pictures) length can be used for video encoding, which can reduce the video key frame interval and minimize the loss of video information during the video encoding process.

[0053] Furthermore, since the original video is encoded first when it is saved in the storage module, before the original video in the storage module is input into the software algorithm module, the video encoding and decoding module can be used to decode the original video, decompose the video into basic single-frame images, and then send the images to the software algorithm module for video processing to output the target video.

[0054] For example, the multiple single-frame images and the video parameter information can be used as inputs to a software algorithm module to obtain multiple target images output by the software algorithm module; then the multiple target images are re-encoded by the video encoding and decoding module to obtain a target video, which is a new video after processing the original video.

[0055] By adopting the above technical solution, when obtaining a video recorded by an electronic device, it is possible to determine the moment when the electronic device enters a scene with video processing computing power and perform video processing operations at that moment. In this way, it is possible to avoid the problem of video freeze and frame loss caused by forcibly executing video processing when the state parameters of the electronic device are not good, thereby improving the display effect of the recorded video.

[0056] Figure 3 is a flowchart of another video processing method provided by an embodiment of the present application. Figure 3 As shown, the method may include the following steps.

[0057] In step S201, the original video recorded by the video and the state parameters of the electronic device are obtained through sensors.

[0058] The original video may be original signal data acquired through a mobile phone sensor, such as a photosensitive element, and has not been processed by any algorithm.

[0059] In practical applications, the objects of video recording may be portraits, animals, natural scenery, etc. Generally speaking, when recording, the user can also select a corresponding recording mode for recording, such as time-lapse photography, slow motion, etc. Videos shot in different recording modes may have different viewing effects. In some possible implementations, different recording modes may also correspond to different software algorithms for video processing in subsequent steps. When starting recording, the user can start recording by clicking a corresponding shooting button (such as a virtual button or a physical button) in the electronic device, or the user can also use a remote method (such as a wireless signal) to control the electronic device video recording, and the embodiments of the present invention do not impose specific restrictions on this.

[0060] During the video recording process, the electronic device can record the object being filmed in real time through the mobile phone sensor, and can also store the filmed video in a storage unit in real time. After recording for a period of time, the user of the electronic device determines that the filming is completed, that is, clicks the "screen recording end shortcut key" to end the recording of the current filmed object, thereby obtaining the corresponding video.

[0061] In this step, the electronic device can obtain the original signal data collected by the electronic device sensor during the video recording process, and then transmit the original signal data to the ISP module to post-process the original signal data to obtain YUV data output by the ISP module, as well as segmentation information and depth information obtained by video analysis of the YUV data.

[0062] In a possible implementation, the original video may include video parameter information obtained according to the original signal data, and the video parameter information may include segmentation information and depth information, as well as YUV data obtained by converting the original signal data.

[0063] Optionally, the YUV data can be input into a video codec module to perform video encoding on the YUV data, and the encoded data can be stored in the UFS (Universal Flash Storage) of the electronic device. Similarly, the segmentation information and depth information obtained based on the original signal data can also be stored in the UFS of the electronic device.

[0064] In some embodiments, the status parameters of the electronic device may include processing parameters input by the user and device performance parameters of the device itself. The device performance parameters are used to represent the current device performance of the electronic device. Based on the device performance, it can be determined whether the electronic device is suitable for performing video processing operations.

[0065] In one possible implementation, when recording a video through the electronic device, no video processing operation may be performed on the video. Instead, after the video recording is completed, a prompt message may be output to prompt the user to input processing parameters. For example, after the video recording is completed, a display window may pop up on the display interface of the electronic device, and the display window may display a display box with the words "Do you want to process the video immediately?" and selection boxes for representing yes and no. When the user triggers the selection box indicating "yes", the video processing operation is performed immediately; and when the user triggers the selection box indicating "no", the device performance parameters of the electronic device are obtained.

[0066] In another possible implementation, the current device performance parameters of the electronic device may be obtained through a sensor unit while the electronic device is recording.

[0067] In step S202, when the control module determines that the scene where the electronic device is located is the target scene according to the state parameter, it is determined that the electronic device meets the target condition.

[0068] The state parameter may include a processing parameter input by a user and a device performance parameter of the device itself, wherein the device performance parameter is used to indicate the current device performance of the electronic device, and according to the current device performance of the electronic device, it can be determined whether the electronic device is currently suitable for performing video processing operations. The scene in which the electronic device is located is a scene in which the electronic device is under the current device performance.

[0069] In this embodiment, the processing parameters input by the user can be judged first. When the processing parameters input by the user indicate a refusal to immediately execute the video processing operation, it can be determined whether the current device performance of the electronic device is suitable for executing the video processing operation based on the device performance parameters of the electronic device. When it is determined that the current device performance of the electronic device is suitable for executing the video processing operation, it is determined that the scenario in which the electronic device is located is the scenario in which the electronic device is located when it has video processing computing power.

[0070] And, the target scene is the scene when the electronic device has video processing computing power. Specifically, the target scene can represent the scene when the electronic device has target device performance parameters, and the target device performance parameters represent the device performance parameters when the electronic device has video processing computing power.

[0071] In this step, when it is determined that the device performance parameters of the electronic device are the target device performance parameters, it is determined that the electronic device has entered the target scene, and when it is determined that the scene where the electronic device is located is the target scene, it is determined that the electronic device meets the target conditions.

[0072] In step S203, after the video recording is completed and the target condition is met, the original video is processed based on the video parameter information.

[0073] Optionally, when the original video recorded by the electronic device is obtained, video parameter information of the original video is obtained through an image signal processor module, and the original video is saved in a storage module, and the video parameter information is saved in the storage module.

[0074] And after the video recording is completed and the target conditions are met, the original video and the video parameter information can be obtained from the storage module through the control module; the original video is then decoded through the video codec module to obtain multiple single-frame images; the multiple single-frame images and the video parameter information are used as inputs of the software algorithm module to obtain multiple target images output by the software algorithm module; and then the multiple target images are re-encoded through the video codec module to obtain a target video, which is a new video after processing the original video.

[0075] In some embodiments, the user can start recording by clicking a corresponding shooting button (such as a virtual button or a physical button) in the electronic device, collect raw signal data through the sensor of the electronic device, and then transmit the raw signal data to the ISP module to post-process the raw signal data to obtain YUV data output by the ISP module, as well as segmentation information and depth information obtained by performing video analysis on the YUV data; the YUV data is input into a video codec module to perform video encoding on the YUV data, and the encoded data can be stored in the UFS (Universal Flash Storage) of the electronic device. Similarly, the segmentation information and depth information obtained based on the raw signal data can also be stored in the UFS of the electronic device.

[0076] Furthermore, when recording a video, the current device performance parameters of the electronic device are simultaneously obtained through the sensing unit, and no video processing operation is performed on the video. Instead, after the video recording is completed, a prompt message is output to prompt the user to input processing parameters. For example, after the video recording is completed, a display window may pop up on the display interface of the electronic device, and the display window may display a display box indicating "Do you want to perform video processing immediately?" and selection boxes for indicating yes and no. When the user triggers the selection box indicating "yes", the video processing operation is performed immediately; and when the user triggers the selection box indicating "no", it is determined whether the current device performance of the electronic device is suitable for performing the video processing operation based on the device performance parameters of the electronic device obtained by the sensing unit, and when it is determined that the current device performance of the electronic device is suitable for performing the video processing operation, it is determined that the scene in which the electronic device is located is the scene in which the electronic device is located when it has video processing computing power, and the moment when the electronic device enters the target scene is used as the target moment, and then the video can be processed at the target moment.

[0077] When processing the video, the stored encoded data can be first obtained from the UFS of the electronic device, and the encoded data can be decoded to obtain decoded data, as well as segmentation information and depth information obtained based on the original signal data, so as to perform video processing through a software algorithm based on the decoded data, segmentation information and depth information.

[0078] By adopting the above technical solution, the video to be processed recorded by the electronic device and the state parameters of the electronic device can be obtained; according to the state parameters, the scene in which the electronic device is located can be determined; when the scene is the target scene, the moment when the electronic device enters the target scene is determined as the target moment, wherein the target scene is the scene in which the electronic device is located when it has video processing computing power; when the target moment is reached, the video is processed. In this way, when the video recorded by the electronic device is obtained, the moment when the electronic device enters the scene with video processing computing power can be determined according to the state parameters of the electronic device, and the video processing operation can be performed at this moment. In this way, the problem of video freeze and frame loss caused by forcibly executing video processing when the state parameters of the electronic device are not good can be avoided, thereby improving the display effect of the recorded video.

[0079] Figure 4 is a flowchart of another video processing method provided by an embodiment of the present application. Figure 4 As shown, the method may include the following steps.

[0080] In step S301, an original video recorded by an electronic device and a user processing parameter of the electronic device are obtained.

[0081] In this step, the video can be recorded by the electronic device, and no video processing operation is performed on the video during the recording process. Instead, after the video recording is completed, a prompt message is output to prompt the user to input processing parameters, or after the video recording is completed, the video processing operation input by the user is detected, and the user processing parameters input by the user are determined based on the video processing operation.

[0082] For example, after the video recording is completed, a display window may pop up on the display interface of the electronic device, and the display window may display a display box asking "Do you want to process the video immediately?" and selection boxes for representing yes and no. When the user triggers the selection box indicating "yes", the user processing parameter representing the user input is to execute video processing.

[0083] Alternatively, after the video recording is completed, a video processing operation input by the user is detected, and the video processing operation may include viewing, sharing, triggering and other operations.

[0084] In step S302, the user demand level of the user for processing the video is determined according to the user processing parameter.

[0085] Optionally, when the user processing parameter includes at least one parameter of viewing, sharing, and triggering, the user demand level is determined to be the first level.

[0086] When the user demand level is the first level, it can be considered that the user needs to process the video immediately.

[0087] And, when the user processing parameter does not include any of the parameters of viewing, sharing, and triggering, the user demand level is determined to be the second level.

[0088] When the user demand level is the second level, it can be considered that the user does not need to process the video immediately.

[0089] Optionally, when any video processing operation such as viewing, sharing, or triggering is detected as input by the user, it can be determined that the user needs to use the video immediately, so the video needs to be processed immediately. Therefore, in this case, the user demand level can be determined to be the first level, that is, the user processing parameters input by the user are used to indicate the execution of video processing; if the user triggers the selection box indicating "yes", or the video processing operation input by the user is not detected, it can be determined that the user demand level is the second level, that is, the user processing parameters input by the user are used to indicate the execution of video processing according to the device performance parameters.

[0090] When it is determined that the user demand level is the first level, step S303, step S307-step S308 are executed;

[0091] When it is determined that the user demand level is the second level, step S304 to step S308 are executed;

[0092] In step S303, it is determined that the scene where the electronic device is located is the scene where the target is located.

[0093] In step S304, the device performance parameters of the electronic device are obtained.

[0094] The device performance parameter is used to represent the current device performance of the electronic device, and whether the electronic device is currently suitable for performing a video processing operation can be determined based on the current device performance of the electronic device.

[0095] The device performance parameters may include device power, device load, device memory, device temperature, and device charging status.

[0096] Considering that, when the electronic device is in a charging state, there is no need to worry about the problem of accelerated device power consumption due to the execution of algorithm simulation, which increases the device's operating power consumption and affects the user experience. Also, when the electronic device is in a charging state, the device temperature of the electronic device will increase due to charging. Therefore, there is no need to consider the impact of the device temperature.

[0097] If the device charging state indicates that the electronic device is in a charging state, executing step S305;

[0098] If the device charging state indicates that the electronic device is not in a charging state, executing step S306;

[0099] In step S305, based on the numerical relationship between the device load and the preset load threshold, and the numerical relationship between the device memory and the preset memory threshold, it is determined that the scene in which the electronic device is located is the target scene.

[0100] Optionally, when the load of the device is less than or equal to a preset load threshold, and the memory of the device is greater than or equal to a preset memory threshold, it can be determined that the scene in which the electronic device is located is the scene in which the target is located.

[0101] In step S306, the scene in which the electronic device is located is determined to be the target scene based on the numerical relationship between the device power and the preset power threshold, the numerical relationship between the device load and the preset load threshold, the numerical relationship between the device memory and the preset memory threshold, and the numerical relationship between the device temperature and the preset temperature threshold.

[0102] Optionally, the scene in which the electronic device is located can be determined to be the scene in which the target is located when the power of the device is greater than or equal to a preset power threshold, the load of the device is less than or equal to a preset load threshold, the memory of the device is greater than or equal to a preset memory threshold, and the temperature of the device is less than or equal to a preset temperature threshold.

[0103] In step S307, when the scene is the target scene, it is determined that the electronic device meets the target condition.

[0104] Among them, the scene in which the target is located is the scene in which the electronic device is located when it has video processing computing power.

[0105] In step S308, after the video recording is completed and the target condition is met, the original video is processed based on the video parameter information.

[0106] By adopting the above technical solution, when a video recorded by an electronic device is obtained, the moment when the electronic device enters a scene with video processing computing power can be determined according to the state parameters of the electronic device, and the video processing operation can be performed at that moment. In this way, the problem of video freeze and frame loss caused by forcibly executing video processing when the state parameters of the electronic device are not good can be avoided, thereby improving the display effect of the recorded video.

[0107] In some embodiments, Figure 5As shown, the electronic device may include a sensor 401, an ISP module 402, a video codec module 403, a file system 404, a software algorithm module 405, and a scheduling control module 406.

[0108] Among them, the sensor 401 is connected to the ISP module 402. The sensor 401 is used to collect raw signal data when the user records a video, and then transmit the raw signal data to the ISP module 402, so that the ISP module 402 first performs RAW processing on the raw signal data to obtain RAW data, wherein specifically, the light signal can be converted into an electrical signal through a photosensitive chip, and the level of the output of the photosensitive chip is recorded as RAW data, and then the obtained RAW data can be converted into YUV data. Specifically, the RAW data can be passed through a Bayer filter to obtain a generated color pixel array, and then the color pixel array is converted into RGB data through a de-mosaic algorithm. Then, each pixel point of the RGB data can be converted into YUV brightness and chromaticity information through a preset conversion algorithm to obtain the final YUV data, and then the YUV data can be segmented and calculated and depth calculated to obtain video analysis information, which may include segmentation information and depth information.

[0109] The ISP module 402 can transmit the obtained YUV data to the video codec module 403, and the video codec module 403 is used to encode the YUV data and store the encoded data in the UFS in the file system 404, and the ISP module 402 can also store the obtained video analysis information in the UFS in the file system 404.

[0110] After the video recording is completed, the electronic device can determine whether the electronic device has entered the target scene through the scheduling control module 406, and if it is determined that the scene where the electronic device is located is the target scene, the moment when the electronic device enters the target scene is determined as the target moment, and when the target moment is reached, the video is processed.

[0111] When processing the video, the encoded data can be first obtained from the UFS in the file system 404, and the encoded data can be transferred to the video codec module 403 for decoding to obtain decoded data, and the stored video analysis information can be obtained from the UFS in the file system 404, and the decoded data and the video analysis information can be input into the software algorithm module 405 to perform video processing through the software algorithm according to the decoded data, segmentation information and depth information.

[0112] Specifically, Figure 6 As shown below, Figure 6Taking the structural diagram of the scheduling control module 406 as an example, the solution of the present disclosure is specifically described.

[0113] After the video recording is completed, the scheduling control module 406 can obtain the detected video processing operation of the user input, and when it is determined that the user processing parameters include at least one parameter of viewing, sharing, and triggering, the user demand level is determined to be the first level; wherein, when the user demand level is the first level, it can be considered that the user needs to process the video immediately; and, when the user processing parameters do not include any of the parameters of viewing, sharing, and triggering, the user demand level is determined to be the second level, wherein, when the user demand level is the second level, it can be considered that the user does not need to process the video immediately.

[0114] In this case, the scheduling control module 406 can continue to obtain the device performance parameters, and determine whether the performance of the electronic device can support video processing based on the device performance parameters. For example, it can determine whether the scene where the electronic device is located is the target scene based on the numerical relationship between parameters such as the device power, device load, device memory, device temperature, and device charging status of the electronic device and preset thresholds. If the scene is the target scene, the moment when the electronic device enters the target scene is determined as the target moment, and then when the target moment is reached, the software algorithm module 405 is triggered to process the video.

[0115] By adopting the above technical solution, when a video recorded by an electronic device is obtained, the moment when the electronic device enters a scene with video processing computing power can be determined according to the state parameters of the electronic device, and the video processing operation can be performed at that moment. In this way, the problem of video freeze and frame loss caused by forcibly executing video processing when the state parameters of the electronic device are not good can be avoided, thereby improving the display effect of the recorded video.

[0116] The video processing method provided in the embodiment of the present application can be executed by a video processing device. In the embodiment of the present application, a video processing device executing the video processing method is taken as an example to illustrate the video processing device provided in the embodiment of the present application.

[0117] Figure 7 is a block diagram of a video processing device provided in an embodiment of the present application, such as Figure 7 As shown, the device 500 includes:

[0118] The acquisition module 501 is used to acquire the original video and video parameter information of the video recording;

[0119] The processing module 502 is used to process the original video based on the video parameter information after the video recording is completed and the target condition is met.

[0120] Optionally, the processing module includes:

[0121] An acquisition submodule, used for acquiring the state parameters of the electronic device through a sensor;

[0122] The first determination submodule is used to determine that the electronic device meets the target condition when the control module determines that the scene in which the electronic device is located is the target scene based on the state parameter, wherein the target scene is the scene in which the electronic device is located when it has video processing computing power, or the scene selected by the user by inputting a video processing operation.

[0123] Optionally, when the state parameters include user processing parameters corresponding to the video processing operation detected to be input by the user, and device performance parameters of the electronic device, the first determination submodule is used to determine the user demand level of the user for processing the video based on the user processing parameters; when it is determined that the user demand level is the first level, the scene in which the electronic device is located is determined to be the target scene; when it is determined that the user demand level is the second level, the scene in which the electronic device is located is determined to be the target scene based on the state parameters.

[0124] Optionally, the first determination submodule is used to determine that the user demand level is the first level when the user processing parameters include at least one parameter of viewing, sharing, and triggering; and to determine that the user demand level is the second level when the user processing parameters do not include any parameter of viewing, sharing, and triggering.

[0125] Optionally, when the status parameter includes a device performance parameter of the electronic device, the device performance parameter includes device power, device load, device memory, device temperature, and device charging status; the first determination submodule is used to determine that the scene in which the electronic device is located is the target scene according to the numerical relationship between the device load and a preset load threshold, and the numerical relationship between the device memory and a preset memory threshold when the device charging status represents that the electronic device is in a charging state; when the device charging status represents that the electronic device is not in a charging state, determine that the scene in which the electronic device is located is the target scene according to the numerical relationship between the device power and the preset power threshold, the numerical relationship between the device load and the preset load threshold, the numerical relationship between the device memory and the preset memory threshold, and the numerical relationship between the device temperature and the preset temperature threshold.

[0126] Optionally, the first determination submodule is used to determine that the scene in which the electronic device is located is the scene in which the target is located when the device load is less than or equal to a preset load threshold and the device memory is greater than or equal to a preset memory threshold.

[0127] Optionally, the first determination submodule is used to determine that the scene of the electronic device is the target scene when the power of the device is greater than or equal to a preset power threshold, the load of the device is less than or equal to a preset load threshold, the memory of the device is greater than or equal to a preset memory threshold, and the temperature of the device is less than or equal to a preset temperature threshold.

[0128] Optionally, the acquisition module is used to acquire video parameter information of the original video through the image signal processor module when the recorded original video is acquired, and save the original video to the storage module, and save the video parameter information to the storage module;

[0129] The processing module is used to obtain the original video and the video parameter information from the storage module through the control module after the video recording is completed and the target conditions are met; decode the original video through the video codec module to obtain multiple single-frame images; use the multiple single-frame images and the video parameter information as inputs of the software algorithm module to obtain multiple target images output by the software algorithm module; re-encode the multiple target images through the video codec module to obtain a target video, which is a video after processing the original video.

[0130] Optionally, the acquisition module is used to encode the original video according to a preset encoding length through the video encoding and decoding module to obtain an encoded video when the recorded original video is acquired; and save the encoded video to the storage module.

[0131] By adopting the above-mentioned device, when a video recorded by an electronic device is obtained, it can be determined that the video recording of the electronic device is completed and meets the target conditions, and a video processing operation can be performed on the original video based on the video parameter information. In this way, the problem of video freeze and frame loss caused by forcibly performing video processing when the status parameters of the electronic device are not good can be avoided, thereby improving the display effect of the recorded video.

[0132] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0133] The video processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0134] The video processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0135] Alternatively, if Figure 8 As shown, an embodiment of the present application further provides an electronic device 600, including a processor 601 and a memory 602, wherein the memory 602 stores programs or instructions that can be executed on the processor 601, and when the program or instructions are executed by the processor 601, the various steps of the above-mentioned video processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0136] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices described above.

[0137] Fig. 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0138] The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710.

[0139] Those skilled in the art will appreciate that the electronic device 700 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 710 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0140] The processor 710 is used to obtain the original video and video parameter information of the video recording; after the video recording is completed and the target condition is met, the original video is processed based on the video parameter information.

[0141] In the embodiment of the present application, the original video and video parameter information of the video recording are obtained; after the video recording is completed and the target condition is met, the original video is processed based on the video parameter information. In this way, when the video recorded by the electronic device is obtained, it can be determined that the video recording of the electronic device is completed and meets the target condition, and the video processing operation is performed on the original video based on the video parameter information. In this way, the problem of video freeze and frame loss caused by forcibly performing video processing when the state parameters of the electronic device are not good can be avoided, thereby improving the display effect of the recorded video.

[0142] It should be understood that in the embodiment of the present application, the input unit 704 may include a graphics processor (Graphics Processing Unit, GPU) 7041 and a microphone 7042, and the graphics processor 7041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 707 includes a touch panel 7071 and at least one of other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts: a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0143] The memory 709 can be used to store software programs and various data. The memory 709 can mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area can store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 709 can include a volatile memory or a non-volatile memory, or the memory x09 can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 709 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0144] The processor 710 may include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 710.

[0145] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned video processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0146] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0147] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned video processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0148] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0149] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned video processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0150] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0151] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0152] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A video processing method, applied to electronic equipment, characterized in that: The method comprises: Get the original video and video parameter information of the video recording; After the video recording is completed and the target condition is met, the original video is processed based on the video parameter information.

2. The method according to claim 1, characterized in that The target conditions include: Acquiring state parameters of the electronic device through a sensor; When the control module determines that the scene in which the electronic device is located is the target scene based on the state parameters, it is determined that the electronic device meets the target condition, wherein the target scene is the scene in which the electronic device is located when it has video processing computing power, or the scene selected by the user by inputting a video processing operation.

3. The method according to claim 2, characterized in that In a case where the state parameters include a user processing parameter corresponding to a video processing operation input by a user and a device performance parameter of the electronic device, determining, by a control module, according to the state parameters, that the scene in which the electronic device is located is a target scene includes: Determining, by the control module, a user demand level of a user for processing the video according to the user processing parameter; In the case where it is determined that the user demand level is the first level, determining that the scene where the electronic device is located is the scene where the target is located; When it is determined that the user demand level is the second level, the scene where the electronic device is located is determined to be the target scene according to the state parameter.

4. The method according to claim 3, characterized in that Determining the user demand level of the user for processing the video according to the user processing parameter includes: In a case where the user processing parameter includes at least one parameter of viewing, sharing, and triggering, determining that the user demand level is the first level; When the user processing parameters do not include any of the parameters of viewing, sharing, and triggering, it is determined that the user demand level is the second level.

5. The method according to claim 2 or 3, characterized in that: In the case where the status parameters include device performance parameters of the electronic device, the device performance parameters include device power, device load, device memory, device temperature, and device charging status; The state parameter determines that the scene where the electronic device is located is a target scene, including: In a case where the device charging state indicates that the electronic device is in a charging state, determining that the scene in which the electronic device is located is the target scene according to a numerical relationship between the device load and a preset load threshold, and a numerical relationship between the device memory and a preset memory threshold; When the charging state of the device indicates that the electronic device is not in a charging state, the scene in which the electronic device is located is determined to be the target scene based on the numerical relationship between the device power and a preset power threshold, the numerical relationship between the device load and a preset load threshold, the numerical relationship between the device memory and a preset memory threshold, and the numerical relationship between the device temperature and a preset temperature threshold.

6. The method according to claim 5, characterized in that The determining, based on the numerical relationship between the device load and the preset load threshold, and the numerical relationship between the device memory and the preset memory threshold, that the scene in which the electronic device is located is the target scene includes: When the device load is less than or equal to a preset load threshold, and the device memory is greater than or equal to a preset memory threshold, it is determined that the scene in which the electronic device is located is the scene in which the target is located.

7. The method according to claim 5, characterized in that Determining the scene where the electronic device is located as the target scene according to the numerical relationship between the power of the device and the preset power threshold, the numerical relationship between the load of the device and the preset load threshold, the numerical relationship between the memory of the device and the preset memory threshold, and the numerical relationship between the temperature of the device and the preset temperature threshold includes: When the power of the device is greater than or equal to a preset power threshold, the load of the device is less than or equal to a preset load threshold, the memory of the device is greater than or equal to a preset memory threshold, and the temperature of the device is less than or equal to a preset temperature threshold, the scene in which the electronic device is located is determined to be the scene in which the target is located.

8. The method according to any one of claims 1 to 7, characterized in that The obtaining of the original video and video parameter information of the video recording includes: When the recorded original video is obtained, video parameter information of the original video is obtained through an image signal processor module, and the original video is saved in a storage module, and the video parameter information is saved in the storage module; The processing of the original video based on the video parameter information includes: After the video recording is completed and the target condition is met, the original video and the video parameter information are obtained from the storage module through the control module; Decoding the original video using a video encoding and decoding module to obtain multiple single-frame images; Using the multiple single-frame images and the video parameter information as inputs to a software algorithm module to obtain multiple target images output by the software algorithm module; The multiple target images are re-encoded by the video encoding and decoding module to obtain a target video, and the target video is a video obtained by processing the original video.

9. The method according to claim 8, characterized in that The step of saving the original video to a storage module comprises: When the recorded original video is obtained, the video encoding and decoding module encodes the original video according to a preset encoding length to obtain an encoded video; The encoded video is stored in the storage module.

10. A video processing device, characterized in that: The device comprises: An acquisition module is used to acquire the original video and video parameter information of the video recording; The processing module is used to process the original video based on the video parameter information after the video recording is completed and the target condition is met.

11. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method according to any one of claims 1 to 9.

Images