Image processing method, device, storage medium and electronic device

Abstract

The present application discloses an image processing method, device, storage medium and electronic device, the method comprising: obtaining a current video code stream and determining an initial frame rate of the video code stream, generating a power consumption limit instruction, and determining a target frame rate according to the power consumption limit instruction, determining an image frame to be decoded in an image group of the video code stream according to the initial frame rate and the target frame rate, and dynamically adjusting the working state of a video decoder according to the decoding order of the image frame to be decoded in the image group to decode the image frame to be decoded. The embodiment of the present application can select an image frame to be decoded according to the target frame rate when power consumption limit is required, and decode the image frame to be decoded by dynamically adjusting the working state of the video decoder, thereby reducing the power consumption of the electronic device.

Description

Technical Field

[0001] The present application belongs to the technical field of electronic equipment, and in particular, relates to an image processing method, device, storage medium and electronic equipment. Background Art

[0002] With the rapid development of science and technology, the functions of electronic devices are becoming more and more powerful. Electronic devices such as video decoders (VideoDecoder, VDEC) can decode video images. When decoding a frame of video image, the data of multiple frames of decoded video images are usually referenced.

[0003] However, as high frame rate screens (for example, greater than or equal to 120FPS) and related applications become increasingly popular, energy consumption is also magnified in this case. As the manufacturing process continues to shrink, the heat density is getting higher and higher (the area reduction ratio is greater than the power consumption reduction ratio), which is more likely to cause equipment overheating. In related technologies, when reading data of decoded video images that need to be referenced, the power consumption of electronic devices is relatively large. Summary of the invention

[0004] The present application provides an image processing method, device, storage medium and electronic device, which can reduce the power consumption of the electronic device.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to an electronic device, wherein the electronic device includes a video decoder, and the method includes:

[0006] Obtaining a current video stream and determining an initial frame rate of the video stream;

[0007] generating a power consumption limit instruction, and determining a target frame rate according to the power consumption limit instruction;

[0008] Determining a to-be-decoded image frame in the image group of the video code stream according to the initial frame rate and the target frame rate; and

[0009] The working state of the video decoder is dynamically adjusted according to the decoding order of the image frames to be decoded in the image group to decode the image frames to be decoded.

[0010] In a second aspect, an embodiment of the present application provides an image processing device, which is applied to an electronic device, wherein the electronic device includes a video decoder, and the device includes:

[0011] An acquisition module, used to acquire a current video code stream and determine an initial frame rate of the video code stream;

[0012] A receiving module, configured to generate a power consumption limit instruction and determine a target frame rate according to the power consumption limit instruction;

[0013] A determination module, configured to determine an image frame to be decoded in an image group of the video code stream according to the initial frame rate and the target frame rate;

[0014] The decoding module is used to dynamically adjust the working state of the video decoder according to the decoding order of the image frames to be decoded in the image group, so as to decode the image frames to be decoded.

[0015] In a third aspect, an embodiment of the present application provides a storage medium on which a computer program is stored. When the computer program is run on a computer, the computer executes the above-mentioned image processing method.

[0016] In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a video decoder, wherein the memory stores a plurality of instructions, and the processor loads the instructions in the memory to perform the following steps:

[0017] Obtaining a current video stream and determining an initial frame rate of the video stream;

[0018] generating a power consumption limit instruction, and determining a target frame rate according to the power consumption limit instruction;

[0019] Determining a to-be-decoded image frame in the image group of the video code stream according to the initial frame rate and the target frame rate; and

[0020] The working state of the video decoder is dynamically adjusted according to the decoding order of the image frames to be decoded in the image group to decode the image frames to be decoded.

[0021] The image processing method provided in the embodiment of the present application can obtain the current video code stream and determine the initial frame rate of the video code stream, generate a power consumption limit instruction, and determine the target frame rate according to the power consumption limit instruction, determine the image frame to be decoded in the image group of the video code stream according to the initial frame rate and the target frame rate, and dynamically adjust the working state of the video decoder according to the decoding order of the image frame to be decoded in the image group to decode the image frame to be decoded. The embodiment of the present application can select the image frame to be decoded according to the target frame rate when power consumption limit is required, and decode the image frame to be decoded by dynamically adjusting the working state of the video decoder, thereby reducing the power consumption of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application.

[0024] Figure 2 A schematic diagram of the structure of a video decoding system provided in an embodiment of the present application.

[0025] Figure 3 A schematic diagram of data storage in a video decoder provided in an embodiment of the present application.

[0026] Figure 4 A schematic diagram of a scene showing the reference relationship between image frames in an image group provided in an embodiment of the present application.

[0027] Figure 5 A schematic diagram of a decoding scenario of an image group without power consumption restriction provided in an embodiment of the present application.

[0028] Figure 6 A schematic diagram of a flow chart of adjusting a decoder state for decoding provided in an embodiment of the present application.

[0029] Figure 7 A schematic diagram of a first decoding scenario of an image group under power consumption limitation provided in an embodiment of the present application.

[0030] Figure 8 A schematic diagram of a second decoding scenario of an image group provided in an embodiment of the present application under power consumption limitation.

[0031] Fig. 9 Another flowchart of adjusting the decoder state for decoding provided in an embodiment of the present application.

[0032] Fig.10 A schematic diagram of a third decoding scenario of an image group under power consumption limitation provided in an embodiment of the present application.

[0033] Fig.11 A schematic diagram of a fourth decoding scenario of an image group under power consumption limitation provided in an embodiment of the present application.

[0034] Fig.12 Another flowchart of the image processing method provided in the embodiment of the present application.

[0035] Fig.13 A schematic diagram of a decoding scenario of a video decoder provided in an embodiment of the present application.

[0036] Fig.14 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application.

[0037] Fig.15 Another structural schematic diagram of the image processing device provided in an embodiment of the present application.

[0038] Fig.16 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0039] Fig.17 Another schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] Please refer to the drawings, where the same component symbols represent the same components, and the principles of the present application are illustrated by implementing them in an appropriate computing environment. The following description is based on the illustrated specific embodiments of the present application, which should not be considered as limiting other specific embodiments of the present application that are not described in detail herein.

[0041] In the following description, the specific embodiments of the present application will be described with reference to the steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be mentioned several times as being performed by a computer, and the computer execution referred to herein includes the operation of a computer processing unit by an electronic signal representing data in a structured form. This operation converts the data or maintains it at a location in the memory system of the computer, which can be reconfigured or otherwise change the operation of the computer in a manner familiar to testers in the field. The data structure maintained by the data is a physical location in the memory, which has specific characteristics defined by the data format. However, the principles of the present application are described in the above text, which does not represent a limitation, and testers in the field will understand that the various steps and operations described below can also be implemented in hardware.

[0042] The terms "first", "second", and "third" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the listed steps or modules, but some embodiments also include steps or modules that are not listed, or some embodiments also include other steps or modules inherent to these processes, methods, products, or devices.

[0043] See also Figure 1 , Figure 1 A schematic diagram of a process flow of an image processing method provided in an embodiment of the present application. The image processing method provided in an embodiment of the present application is applied to an electronic device, and the specific process may be as follows:

[0044] Step 101, obtain the current video stream and determine the initial frame rate of the video stream.

[0045] There are two organizations that develop video coding and decoding technologies internationally. One is the International Telecommunication Union (ITU-T), which develops standards such as H.261, H.263, and H.263+, and the other is the International Organization for Standardization (ISO), which develops standards such as MPEG-1, MPEG-2, and MPEG-4. H.264 is a new digital video coding standard jointly developed by the Joint Video Team (JVT) formed by the two organizations. In the embodiment of the present application, when decoding a video, the bit stream of the video is first obtained.

[0046] After obtaining the video bitstream, the frame rate of the video bitstream, namely FPS (Frames PerSecond), is further determined. The frame rate refers to the number of frames transmitted per second, which generally refers to the number of frames in an animation or video. FPS is a measure of the amount of information used to save and display dynamic videos. The more frames per second, the smoother the displayed action will be. Movies are played at a speed of 24 frames per second, which means that 24 still images are continuously projected on the screen within one second. However, with the upgrading of technology and playback equipment, videos with 30FPS, 60FPS and even 120FPS have gradually appeared, so there are higher requirements for video decoding.

[0047] See also Figure 2 , Figure 2 It is a structural diagram of a video decoding system in the related art. In the video decoding system, the central processing unit (CPU), embedded neural network processor (NPU), image signal processor (ISP), video decoder (VDEC) and display processor (DISP) read and write data from DRAM through the bus and dynamic random access memory controller (DRAMC). The central processing unit, video decoder and display processor share bandwidth in a time-sharing manner. The priority of the central processing unit and display processor is higher than that of the video encoder. It should be noted that the display processor can be set in the video decoding system according to specific needs, or the display processor may not be set. The video decoder needs to perform motion compensation (MC) when decoding, which will occupy a large bandwidth.

[0048] Electronic devices such as video decoders attach great importance to cost. In order to achieve the lowest cost and highest production yield, DRAM is usually used as the main storage space for frame buffering. Figure 3 , Figure 3 The figure is a schematic diagram of data storage in a video decoder in the related art. Bitstreams, image frames to be buffered, and temporary data are all stored in the DRAM in the video decoder. However, the bandwidth provided by DRAM is small. Temporary data can be a temporal motion vector (TMV) and other data.

[0049] Although the video decoder can adopt the internal cache strategy of the video decoder for various video bitstreams, for example, the video bitstream can be the first video and audio lossy compression standard organized by the Moving Picture Experts Group Phase 1 (MPEG-1), the second video and audio lossy compression standard organized by the Moving Picture Experts Group (MPEG-2), the fourth video and audio lossy compression standard organized by the Moving Picture Experts Group (MPEG-4), Essential Video Coding (MPEG-5 / EVC), the low bit rate video coding standard H.263 for video conferencing developed by the ITU Telecommunication Standardization Sector (ITU-T), Advanced Video Coding (H.264 / AVC), High Efficiency Video Coding (H.265 / HEVC), Versatile Video Coding ( Coding, H.266 / VVC), Video Predictor 8, VP8, Video Predictor 9, VP9, ​​and Alliance for Open Media Video 1, AV1.

[0050] However, with the emergence of new video standards, such as H.265 / HEVC, H.266 / VVC, AV1, MPEG-5, etc., which target larger and larger screen sizes and higher and higher frame rates, increasing DRAM bandwidth or increasing DRAM frequency is usually used to accelerate data throughput.

[0051] In an embodiment of the present application, a current video bitstream is obtained, and the video bitstream may include one or more groups of pictures (Group of pictures, GOP), and one group of pictures includes multiple image frames. In an embodiment of the present application, an example is taken in which the current video bitstream includes one group of pictures. The current video bitstream obtained is an encoded video bitstream, and the image frames in the current video bitstream may not have been decoded, and some image frames may have been decoded, and other image frames may be waiting to be decoded. It should be noted that the decoded image frames can be used as reference image frames when decoding other subsequent image frames.

[0052] Step 102: Generate a power consumption limit instruction, and determine a target frame rate according to the power consumption limit instruction.

[0053] In an embodiment of the present application, when decoding, the decoder can decide whether to reduce the frame rate of the decoded video according to the power consumption requirements. Specifically, if a power consumption instruction limit instruction is received, the target frame rate corresponding to the power consumption instruction limit instruction can be further obtained. For example, if the initial frame rate of the video stream is 120FPS, the electronic device receives a power consumption instruction limit instruction and the target frame rate corresponding to the instruction is 30FPS, that is, the video obtained after decoding the video stream only needs a frame rate of 30FPS, thereby reducing the pressure and power consumption of the video decoder.

[0054] It should be noted that if no power consumption limit instruction is received, that is, the current power of the electronic device is sufficient and there is no need to reduce power consumption, the video decoder can be controlled to perform a normal decoding process according to the initial rate of the video bit stream, such as 120FPS.

[0055] In another embodiment, after determining the target frame rate corresponding to the power consumption limit instruction, it is further determined whether it is less than the initial frame rate of the video stream. If it is less than, the subsequent steps are continued to limit the power consumption of the video decoder. If it is not less than, the video decoder is also controlled to perform a normal decoding process according to the initial rate of the video stream. For example, if the initial frame rate of the video stream is 60FPS, and the target frame rate corresponding to the power consumption limit instruction is also 60FPS, then decoding according to the initial frame rate can meet the current power consumption requirements, so there is no need to perform subsequent steps and follow the normal decoding process. If the initial frame rate of the video stream is 60FPS, and the target frame rate corresponding to the power consumption limit instruction is also 30FPS, which is less than the initial frame rate, it is determined that power consumption control is required, and it is necessary to continue to perform the subsequent step 103.

[0056] In one embodiment, the power consumption limit instruction may be automatically generated by the electronic device, for example, when the remaining power of the device is less than a preset value, the power consumption limit instruction carrying the target frame rate is automatically generated. That is, before generating the power consumption limit instruction, the method further includes:

[0057] Get the current remaining power of the electronic device;

[0058] Determining whether the remaining power is less than a preset value;

[0059] If it is less than the preset value, a power consumption limiting instruction is generated according to the remaining power.

[0060] Further, after determining whether the remaining power is less than the preset value, the corresponding target frame rate can be set according to the preset interval corresponding to the remaining power to generate a power consumption limit instruction. For example, the above preset value is set to 30%, and three power intervals of 0-10%, 11%-20%, and 21%-30% are set below the preset value. If the current remaining power of the electronic device is 25%, which is in the 21%-30% power interval, the corresponding target frame rate can be set to 90FPS, and a power consumption limit instruction is generated according to the above target frame rate. If the current remaining power of the electronic device is 18%, which is in the 11%-20% power interval, the corresponding target frame rate can be set to 60FPS, and a power consumption limit instruction is generated according to the above target frame rate. If the current remaining power of the electronic device is 5%, which is in the 0-10% power interval, the corresponding target frame rate can be set to 30FPS, and a power consumption limit instruction is generated according to the above target frame rate.

[0061] Step 103 : determining the image frame to be decoded in the image group of the video code stream according to the initial frame rate and the target frame rate.

[0062] In an embodiment of the present application, if the target frame rate is less than the initial frame rate, for example, the target frame rate is 30 FPS and the initial frame rate is 120 FPS, then the video stream only needs to obtain 30 frames of images after decoding. Therefore, there will be some image frames in the image group of the video stream that do not need to be decoded. Therefore, it is necessary to perform different processing on the image frames to be decoded and the image frames not to be decoded in the image group.

[0063] For example, if Figure 4 As shown, Figure 4 A schematic diagram of a scene showing the reference relationship between image frames in an image group provided in an embodiment of the present application. This embodiment is described by taking an image group including 13 image frames as an example. In other implementations, the number of image frames included in an image group can be adjusted according to specific needs. In an image group, the display order of the image frames may be the same as the decoding order, or may be different. Figure 4 The display order of the image frames in the image group shown in is different from the decoding order. Assume that the initial frame rate of the video stream is 120FPS, that is, the time interval between I0 and B1 is 1 / 120 second, the time interval between B1 and B2 is 1 / 120 second, the time interval between B2 and B3 is 1 / 120 second, and so on. Assuming that the power consumption or energy consumption limit requires that VDEC can only run at 30FPS, that is, the target frame rate is 30FPS, then it is necessary to decode the frames that need to be played when the frame rate is 30FPS, that is, I0, P4, B8, P12 in the figure, and define the above frames that need to be played as key image frames. After analyzing the code stream, it is found that if you want to decode the above P4, B8, P12, you also need to refer to the image frames B2, B3, B6. In this way, the above key image frames and reference image frames can be determined as image frames to be decoded.

[0064] Therefore, in this step, it is necessary to determine the reference image frame of the key image frame in the image group according to the image group structure. Specifically, the reference frame can be determined by bitstream analysis, so as to select the bitstreams that the VDEC actually needs to decode, without decoding the image frames that are not to be decoded. In this way, the energy consumption and power consumption of the VDEC can be reduced as much as possible to cool down the entire system or maintain energy allocation to the most important modules.

[0065] Specifically, the above-mentioned reference picture frame and the number of times the reference frame is referenced can be determined by preset parameters, and the preset parameters can include network abstraction layer parsing parameters, slice header parsing parameters, reference picture list modification parameters, and reference picture frame marking parameters, etc. For example, the network abstraction layer parsing parameters can be the nal_unit() function, the slice header parsing parameters can be the slice_header() function, the reference picture list modification parameters can be the ref_pic_list_modification() function, and the reference picture frame marking parameters can be the ref_pic_list_modification() function.

[0066] For example, taking H.264 as an example, when performing a rough frame-level analysis, after parsing the NAL unit header information or slice header information contained in multiple image frames, it is possible to distinguish which reference image frames will be referenced multiple times, such as by using the nal_ref_idc variable in the nal_unit() function, the num_ref_idx_active_override_flag variable in the slice_header() function, the ref_pic_list_modification() function, the dec_ref_pic_marking() function and other information to make a preliminary judgment.

[0067] For example, the nal_unit() function analyzes the NAL units starting with 00 00 00 01 and 00 00 01 from the H.264 image frame, and then directly fills in the length of the NAL unit. The nal_ref_idc variable represents the reference level, which represents the reference status of other image frames. The higher the reference level, the more important the reference image frame is.

[0068] The num_ref_idx_active_override_flag variable indicates whether the number of actually available reference frames for the current image frame needs to be overridden. The syntax elements num_ref_idx_l0_active_minus1 and num_ref_idx_l1_active_minus1 that already appear in the image parameter set specify the number of actually available reference frames in the current reference image queue. This pair of syntax elements can be overridden in the slice header to give a greater flexibility to a particular image frame. The position of the slice can be known through the num_ref_idx_active_override_flag variable.

[0069] The ref_pic_list_modification() function is a reference picture list modification function, which can be saved in the structure of the slice header. The definition of the ref_pic_list_modification() function is as follows: when ref_pic_list_modification_flag_l0 is 1, the reference picture list RefPicList0 is modified, and when ref_pic_list_modification_flag_l1 is 1, the reference picture list RefPicList1 is modified. The dec_ref_pic_marking() function is the decoded reference picture frame mark, and the marking operation is used to move the reference picture frame into or out of the reference picture frame queue and specify the symbol of the reference picture.

[0070] Continue reading Figure 4 In this embodiment, taking H.264 video code stream as an example, a rough analysis is performed on the H.264 video code stream. By analyzing the header information of the NAL unit or the slice header information, it is possible to analyze whether certain image frames are referenced and the number of times they are referenced. By roughly analyzing the code streams corresponding to these 13 image frames, the reference image list can be obtained. Figure 4 The arrow direction in the figure shows the reference relationship and reference times between each image frame and other image frames. When an image frame is referenced by other image frames once or multiple times, the image frame can be used as a reference image frame. For example, Figure 4 The reference number of I0 frame is 4 times, the reference number of B2 frame with display order 2 is 4 times, the reference number of B3 frame with display order 3 is 2 times, the reference number of P4 frame with display order 4 is 3 times, the reference number of B6 frame with display order 6 is 1 time, the reference number of B8 frame with display order 8 is 5 times, the reference number of P12 frame with display order 12 is 4 times, and so on. That is, in this embodiment, after analyzing the code stream, it is found that the frames that really need to be decoded are the image frames I0, B2, B3, P4, B6, B8, and P12 to be decoded. The remaining image frames B1, B5, B7, P9, P10, and P11 are image frames that are not to be decoded. It should be noted that Figure 4 In the above, only a rough frame-level analysis is performed by the relevant hardware or software of the video decoder. The reference relationship of each specific block in the reference frame image cannot be analyzed through this rough analysis method.

[0071] Step 104: dynamically adjust the working state of the video decoder according to the decoding order of the image frames to be decoded in the image group, so as to decode the image frames to be decoded.

[0072] For example, if a power consumption limit instruction is received and the image frame to be decoded is determined according to the target frame rate of the power consumption limit instruction, the working state of the video decoder can be dynamically adjusted to decode the image frame to be decoded without processing the non-to-be-decoded image frames in the image group.

[0073] It should be noted that if no power consumption limit instruction is received, or the target frame rate in the power consumption limit instruction is the same as the initial frame rate, all image frames of the image group in the video stream can be decoded. Figure 5 , Figure 5 A schematic diagram of a decoding scenario of an image group provided in an embodiment of the present application without power consumption restriction. In this embodiment, if no power consumption restriction instruction is received, that is, the electronic device currently has sufficient power and there is no need to reduce the frame rate to reduce power consumption, then all image frames in the image group can be decoded when decoding the block to be decoded. In this figure, the initial frame rate of the video code stream is 120FPS, so because each image frame in the image group can be displayed, the VDEC will strive to decode each image frame and send it to the screen of the electronic device for display, that is, to maintain the voltage of the video decoder at the voltage value required for decoding the 120FPS frame rate until all image frames in the image group are decoded.

[0074] In one embodiment, the working state of the video decoder may include an on state and an off state. Figure 6 , Figure 6 A schematic diagram of a flow chart of adjusting the decoder state for decoding provided in an embodiment of the present application. In this embodiment, the above step 104 may include the following steps:

[0075] Step 1041, calculating the decoding unit time in the picture group according to the target frame rate.

[0076] In the embodiment of the present application, the above decoding unit time needs to be calculated according to the target frame rate. For example, if the initial frame rate of the video stream is 120FPS and the target frame rate is 30FPS, then the decoding unit time in the image group can be determined to be 1 / 30 second. Figure 7 , Figure 7 A schematic diagram of a first decoding scenario of an image group under power consumption limitation provided in an embodiment of the present application. In this embodiment, the IO frame is the first image frame to be decoded, and no reference frame is required, so the decoding unit time can be calculated from the completion of the I0 frame decoding.

[0077] Step 1042: Determine the number of image frames to be decoded within a decoding unit time according to the decoding order of the image frames to be decoded in the image group.

[0078] like Figure 7As shown, within the first decoding unit time, i.e. 1 / 30 second, after the decoding of the I0 frame is completed, there are three image frames to be decoded, namely B2, B3, and P4; within the second decoding unit time, i.e. 1 / 30 second, there are two image frames to be decoded, namely B6 and B8; within the third decoding unit time, i.e. 1 / 30 second, there is one image frame to be decoded, namely P12, and so on.

[0079] Step 1043 , dividing the decoding unit time into a first time period and a second time period according to the number of image frames to be decoded within the decoding unit time.

[0080] For example, in the first decoding unit time, since there are 3 image frames to be decoded, the 1 / 30 second can be divided into the first 1 / 120 second and the last 3 / 120 second. In the second decoding unit time, since there are 2 image frames to be decoded, the 1 / 30 second can be divided into the first 2 / 120 second and the last 2 / 120 second. In the third decoding unit time, since there is 1 image frame to be decoded, the 1 / 30 second can be divided into the first 3 / 120 second and the last 1 / 120 second, and so on.

[0081] Step 1044: During the first time period, the video decoder is adjusted to a closed state.

[0082] Step 1045 , within the second time period, adjusting the video decoder to be turned on to decode the image frames to be decoded corresponding to the number of image frames to be decoded.

[0083] In this embodiment, for example, in the first time period, i.e., the first 1 / 120 second, the video decoder does not need to work, so the power supply can be turned off. At this time, the voltage of the video decoder is 0. In the second time period, i.e., the last 3 / 120 seconds, the video decoder needs to decode B2, B3, and P4. At this time, the video decoder is turned on and the voltage of the video decoder is set to the voltage value required for decoding the 120FPS frame rate until the current decoding unit time ends, and then the decoding process in the next decoding unit time is started.

[0084] In another application embodiment, if Figure 8 As shown, Figure 8A schematic diagram of a second decoding scenario of an image group provided in an embodiment of the present application under power consumption constraints. After determining that the decoding unit time in the image group is 1 / 30 second, the decoding unit time can be calculated from before the decoding of the I0 frame begins. Therefore, in this embodiment, within the first decoding unit time, i.e., 1 / 30 second, there is an image frame to be decoded, i.e., I0. Within the second decoding unit time, i.e., 1 / 30 second, there are three image frames to be decoded, namely, B2, B3, and P4. Within the third decoding unit time, i.e., 1 / 30 second, there are two image frames to be decoded, namely, B6 and B8. Within the fourth decoding unit time, i.e., 1 / 30 second, there is an image frame to be decoded, i.e., P12, and so on. The subsequent division of the first time period and the second time period can refer to the above steps, which will not be repeated here.

[0085] In other application embodiments, the working state of the video decoder may include voltage strength. For example, the voltage strength of the video decoder is adjusted to a voltage value corresponding to decoding a 120FPS frame rate, or a voltage value corresponding to decoding a 90FPS frame rate, or a voltage value corresponding to decoding a 60FPS frame rate, or a voltage value corresponding to decoding a 30FPS frame rate, etc. Fig. 9 , Fig. 9 A schematic diagram of a flow chart of adjusting the decoder state for decoding provided in an embodiment of the present application. In this embodiment, the above step 104 may include the following steps:

[0086] Step 1046, calculating the decoding unit time in the picture group according to the target frame rate.

[0087] In the embodiment of the present application, the above decoding unit time needs to be calculated according to the target frame rate. For example, if the initial frame rate of the video stream is 120FPS and the target frame rate is 30FPS, then the decoding unit time in the image group can be determined to be 1 / 30 second. Fig.10 , Fig.10 A schematic diagram of a third decoding scenario of an image group under power consumption limitation provided in an embodiment of the present application. In this embodiment, the IO frame is the first image frame to be decoded, and no reference frame is required, so the decoding unit time can be calculated from the completion of the I0 frame decoding.

[0088] Step 1047: Determine the number of image frames to be decoded within a decoding unit time according to the decoding order of the image frames to be decoded in the image group.

[0089] like Fig.10As shown, within the first decoding unit time, i.e. 1 / 30 second, after the decoding of the I0 frame is completed, there are three image frames to be decoded, namely B2, B3, and P4; within the second decoding unit time, i.e. 1 / 30 second, there are two image frames to be decoded, namely B6 and B8; within the third decoding unit time, i.e. 1 / 30 second, there is one image frame to be decoded, namely P12, and so on.

[0090] Step 1048: Adjust the operating voltage of the video decoder according to the number of image frames to be decoded.

[0091] For example, Fig.10 In the first decoding unit time after the I0 frame decoding is completed, since there are 3 image frames to be decoded, the working voltage of the video decoder in the decoding unit time can be set to the voltage value corresponding to the decoding 90FPS frame rate. In the second decoding unit time, since there are 2 image frames to be decoded, the working voltage of the video decoder in the decoding unit time can be set to the voltage value corresponding to the decoding 60FPS frame rate. In the third decoding unit time, since there is 1 image frame to be decoded, the working voltage of the video decoder in the decoding unit time can be set to the voltage value corresponding to the decoding 30FPS frame rate, and so on.

[0092] Step 1049 , within the decoding unit time, decode the image frames to be decoded corresponding to the number of image frames to be decoded using the working voltage of the video decoder.

[0093] Specifically, in the first decoding unit time, the video decoder operates at a voltage value corresponding to decoding a 90FPS frame rate, and decodes B2, B3, and P4. In the second decoding unit time, the video decoder operates at a voltage value corresponding to decoding a 60FPS frame rate, and decodes B6 and B8. In the third decoding unit time, the video decoder operates at a voltage value corresponding to decoding a 30FPS frame rate, and decodes P12, and so on.

[0094] In another application embodiment, if Fig.11 As shown, Fig.11A schematic diagram of the fourth decoding scenario of the image group provided in an embodiment of the present application under power consumption limitation. Among them, after determining that the decoding unit time in the image group is 1 / 30 second, the decoding unit time can be calculated from before the decoding of the I0 frame begins. Therefore, in this embodiment, within the first decoding unit time, that is, 1 / 30 second, there is an image frame to be decoded, that is, I0. Within the second decoding unit time, that is, 1 / 30 second, there are three image frames to be decoded, namely B2, B3, and P4, and within the third decoding unit time, that is, 1 / 30 second, there are two image frames to be decoded, namely B6 and B8, and within the fourth decoding unit time, that is, 1 / 30 second, there is an image frame to be decoded, that is, P12, and so on. The subsequent steps of adjusting the working voltage of the video decoder for decoding according to the number of image frames to be decoded within the decoding unit time can refer to the above content and will not be repeated here.

[0095] As can be seen from the above, the image processing method provided by the embodiment of the present application can obtain the current video code stream and determine the initial frame rate of the video code stream, generate a power consumption limit instruction, and determine the target frame rate according to the power consumption limit instruction, determine the image frame to be decoded in the image group of the video code stream according to the initial frame rate and the target frame rate, and dynamically adjust the working state of the video decoder according to the decoding order of the image frame to be decoded in the image group to decode the image frame to be decoded. The embodiment of the present application can select the image frame to be decoded according to the target frame rate when power consumption limit is required, and decode the image frame to be decoded by dynamically adjusting the working state of the video decoder, thereby reducing the power consumption of the electronic device.

[0096] The following will further introduce the image processing method of the present application based on the method described in the above embodiment. Fig.12 , Fig.12 Another flowchart of an image processing method provided in an embodiment of the present application is provided, and the image processing method includes:

[0097] Step 201, obtain the current video stream and determine the initial frame rate of the video stream.

[0098] In an embodiment of the present application, a current video bitstream is obtained, and the video bitstream may include one or more groups of pictures (Group of pictures, GOP), and one group of pictures includes multiple image frames. In an embodiment of the present application, an example is taken in which the current video bitstream includes one group of pictures. The current video bitstream obtained is an encoded video bitstream, and the image frames in the current video bitstream may not have been decoded, and some image frames may have been decoded, and other image frames may be waiting to be decoded. It should be noted that the decoded image frames can be used as reference image frames when decoding other subsequent image frames.

[0099] After obtaining the video bitstream, the frame rate of the video bitstream, that is, FPS, is further determined, such as 30FPS, 60FPS, or even 120FPS.

[0100] Step 202: If a power consumption limit instruction is generated, a target frame rate is determined according to the power consumption limit instruction.

[0101] In an embodiment of the present application, when decoding, the decoder can decide whether to reduce the frame rate of the decoded video according to the power consumption requirements. Specifically, if a power consumption instruction limit instruction is received, the target frame rate corresponding to the power consumption instruction limit instruction can be further obtained. For example, if the initial frame rate of the video stream is 120FPS, the electronic device receives a power consumption instruction limit instruction and the target frame rate corresponding to the instruction is 30FPS, that is, the video obtained after decoding the video stream only needs a frame rate of 30FPS, thereby reducing the pressure and power consumption of the video decoder.

[0102] Step 203: If no power consumption limit instruction is received, the refresh rate of the electronic device screen is obtained.

[0103] In one embodiment, if the power consumption limit instruction is not received, it is considered that if the decoded video frame rate is greater than the screen refresh rate of the electronic device, it will cause a waste of resources and increase the power consumption of the video decoder. For example, the initial frame rate of the video stream is 120FPS, and the screen refresh rate of the electronic device is only 60FPS. Even if the electronic device has sufficient power and does not need to limit power consumption, that is, if the power consumption limit instruction is not received, the current refresh rate cannot meet the playback requirements of the 120FPS frame rate video. Therefore, the target frame rate can be determined according to the refresh rate of the electronic device screen.

[0104] Among them, the refresh rate of the electronic device screen can be obtained by calling the electronic device screen parameters. In addition to the screen refresh rate, the above-mentioned electronic device screen parameters can also include resolution, brightness, sRGB (standard Red GreenBlue) color gamut, NTSC (NTSC color gamut) color gamut, PPI (Pixels Per Inch, pixel density) and the like.

[0105] Step 204: determine the target frame rate according to the refresh rate of the screen and the initial frame rate of the video stream.

[0106] Specifically, if the screen refresh rate is lower than the initial frame rate of the video stream, the screen refresh rate can be set to the target frame rate. For example, if the initial frame rate of the video stream is 120FPS and the screen refresh rate of the electronic device is 60FPS, if no power consumption limit instruction is received, 60FPS can be set as the target frame rate.

[0107] Step 205 : determining a key image frame in the image group of the video code stream according to the initial frame rate and the target frame rate.

[0108] In an embodiment of the present application, if the target frame rate is less than the initial frame rate, for example, the target frame rate is 30 FPS and the initial frame rate is 120 FPS, then the video stream only needs to obtain 30 frames of images after decoding. Therefore, there will be some image frames in the image group of the video stream that do not need to be decoded. Therefore, it is necessary to perform different processing on the image frames to be decoded and the image frames not to be decoded in the image group.

[0109] In this step, you first need to determine the key image frame to be played. Figure 4 Assuming that the power consumption or energy consumption limit requires that VDEC can only run at 30FPS, that is, the target frame rate is 30FPS, it is necessary to decode the key image frames required for 30FPS playback, that is, I0, P4, B8, and P12 in the figure.

[0110] Step 206: Obtain the decoding dependency relationship corresponding to the image group.

[0111] In actual use, it is found that after bitstream analysis, if you want to decode the above P4, B8, and P12, you also need to refer to image frames B2, B3, and B6. Therefore, you need to first obtain the decoding dependency corresponding to the image group.

[0112] Step 207 , determining the reference image frame referenced by the key image frame according to the decoding dependency relationship, and determining the image frame to be decoded according to the key image frame and the reference image frame of the key image frame.

[0113] The above decoding dependency relationship includes the reference relationship between each image frame and other image frames. Specifically, it can be determined in sequence whether the current frame is referenced by the key image frame according to the decoding dependency relationship. If so, the current frame is determined to be a reference image frame. If not, the current frame is determined to be a non-to-be-decoded image frame. That is, the steps of determining the reference image frame referenced by the key image frame according to the decoding dependency relationship, and determining the image frame to be decoded according to the key image frame and the reference image frame of the key image frame include:

[0114] Determining whether the current frame is referenced by a key image frame according to the decoding dependency relationship;

[0115] If yes, determining that the current frame is the reference image frame;

[0116] The image frame to be decoded is determined according to the key image frame and the reference image frame of the key image frame.

[0117] In one embodiment, VDEC-related hardware or software can determine that certain image frames will be used when decoding other frames after analysis or estimation, that is, the above-mentioned decoding dependency. Specifically, the decoding dependency of each image frame can be determined based on the frame header information of each image frame in the current video code stream. Among them, the data of each image frame can be regarded as a network abstraction layer (Network Abstraction Layer, NAL) unit, and the frame header information is used to distinguish the beginning of an image frame. The frame header information can also be regarded as NAL unit header information. The frame header information can be used to determine which image frame it is, that is, the reference image frame can be determined.

[0118] Step 208: dynamically adjust the working state of the video decoder according to the decoding order of the image frames to be decoded in the picture group, so as to decode the image frames to be decoded.

[0119] For example, if a power consumption limit instruction is received and the image frame to be decoded is determined according to the target frame rate of the power consumption limit instruction, the working state of the video decoder can be dynamically adjusted to decode the image frame to be decoded without processing the non-to-be-decoded image frames in the image group.

[0120] In one embodiment, the working state of the video decoder may include an on state and an off state. The electronic device needs to calculate the decoding unit time in the image group according to the target frame rate, determine the number of image frames to be decoded in the decoding unit time according to the decoding order of the image frames to be decoded in the image group, divide the decoding unit time into a first time period and a second time period according to the number of image frames to be decoded in the decoding unit time, and then adjust the video decoder to a off state in the first time period, and adjust the video decoder to an on state in the second time period to decode the image frames to be decoded corresponding to the number of image frames to be decoded.

[0121] In another embodiment, the working state of the video decoder may include voltage intensity. For example, the voltage intensity of the video decoder is adjusted to a voltage value corresponding to a decoding frame rate of 120FPS, or a voltage value corresponding to a decoding frame rate of 90FPS, or a voltage value corresponding to a decoding frame rate of 60FPS, or a voltage value corresponding to a decoding frame rate of 30FPS, etc. The decoding unit time in the image group is calculated according to the target frame rate, the number of image frames to be decoded in the decoding unit time is determined according to the decoding order of the image frames to be decoded in the image group, and then the working voltage of the video decoder is adjusted according to the number of image frames to be decoded, and the image frames to be decoded corresponding to the number of image frames to be decoded are decoded at the working voltage of the video decoder within the decoding unit time.

[0122] In one embodiment, the above step of decoding the image frame to be decoded may include:

[0123] Acquire a reference motion vector according to the current video code stream;

[0124] Acquire a corresponding reference block from an image frame of the current video code stream according to the reference motion vector;

[0125] The to-be-decoded block of the to-be-decoded image frame is decoded according to the reference block.

[0126] Specifically, the current video bitstream is parsed, and one or more reference motion vectors can be obtained from the current video bitstream. Each reference motion vector corresponds to a reference block. When decoding the block to be decoded, the reference block needs to be referenced. The relative displacement between the reference block and the block to be encoded can be used as a reference motion vector. By obtaining the corresponding reference block through the reference motion vector, a refined analysis can be performed, and finally the block to be decoded is decoded according to the image data of the reference block.

[0127] Among them, the current video bitstream can be entropy decoded to obtain one or more motion vector differences (Motion Vector Difference, MVD), and then one or more reference motion vectors can be obtained based on the one or more motion vector differences and the corresponding motion vector prediction values. For example, after entropy decoding the current video bitstream to obtain one or more motion vector differences, one or more reference motion vectors can be obtained based on the one or more motion vector differences and the corresponding motion vector prediction values, such as adding the motion vector difference to the motion vector prediction value as the reference motion vector. According to the one or more reference motion vectors, the corresponding one or more reference blocks are obtained from one or more image frames of the current video bitstream, and finally the image data of the required reference blocks are read, and the current block to be decoded or the sub-block in the current block to be decoded is decoded according to the read image data of the reference block.

[0128] When decoding, the image frame can be divided into a plurality of non-overlapping blocks, which form a rectangular array, wherein each block is a block of N×N pixels, for example, a block of 4×4 pixels, a block of 32×32 pixels, a block of 128×128 pixels, etc. When decoding the block to be decoded in the current image frame to be decoded, it is necessary to read the image data of the required reference image frame from the memory.

[0129] For example, see Fig.13 , Fig.13A schematic diagram of a decoding scene of a video decoder provided in an embodiment of the present application. The current video code stream is illustrated by taking the H.264 video code stream as an example. After entropy decoding of the current video code stream, one or more motion vector differences and a quantized first residual are obtained. According to the motion vector difference and the corresponding motion vector prediction value, a reference motion vector can be obtained, so that the reference block used for motion compensation can be more precisely known. Among them, entropy decoding can be implemented by an independent hardware design or by software. The parsing of the current video code stream and image buffering can be implemented by software through a driver or an open multimedia acceleration layer framework (Open MediaAcceleration, OpenMAX). After the first residual is inversely quantized and inversely transformed, a second residual can be obtained. According to the reference motion vector (the relative displacement between the reference block and the current block to be decoded or the sub-block in the current block to be decoded) and the reference block, the predicted value of the current block to be decoded or the sub-block in the current block to be decoded can be obtained. It should be noted that the predicted value of the current block to be decoded or the sub-block in the current block to be decoded can be obtained by intra-frame prediction or motion compensation. Inverse quantization and inverse transformation, intra / inter mode selection, intra prediction, motion compensation and deblocking filtering in the decoding process can be implemented through an application specific integrated circuit (ASIC).

[0130] After obtaining the prediction value of the current block to be decoded or the sub-block in the current block to be decoded, the second residual is added to the prediction value of the current block to be decoded or the sub-block in the current block to be decoded to obtain the decoded block of the current block to be decoded or the decoded sub-block of the sub-block in the current block to be decoded (actual value), and after filtering by a blocking effect filter according to the decoded block of the current block to be decoded or the decoded sub-block of the sub-block in the current block to be decoded, smooth video stream decoded data can be obtained.

[0131] If there are other blocks to be decoded or sub-blocks of other blocks to be decoded in the current image frame to be decoded or the current strip to be decoded that need to be decoded, then decode the other blocks to be decoded or sub-blocks of other blocks to be decoded. If all blocks to be decoded or sub-blocks of blocks to be decoded in the current image frame to be decoded or the current strip to be decoded have been decoded, decode the other image frames or strips until the decoding of all image frames or strips that need to be decoded is completed.

[0132] As can be seen from the above, the image processing method provided by the embodiment of the present application can obtain the current video code stream and determine the initial frame rate of the video code stream. If a power consumption limit instruction is generated, the target frame rate is determined according to the power consumption limit instruction. If the power consumption limit instruction is not received, the refresh rate of the electronic device screen is obtained, and the target frame rate is determined according to the refresh rate of the screen and the initial frame rate of the video code stream. The key image frame is determined in the image group of the video code stream according to the initial frame rate and the target frame rate, and the decoding dependency corresponding to the image group is obtained. The reference image frame referenced by the key image frame is determined according to the decoding dependency, and the image frame to be decoded is determined according to the key image frame and the reference image frame of the key image frame, and the working state of the video decoder is dynamically adjusted according to the decoding order of the image frame to be decoded in the image group to decode the image frame to be decoded. The embodiment of the present application can select the image frame to be decoded according to the target frame rate when power consumption limitation is required, and decode the image frame to be decoded by dynamically adjusting the working state of the video decoder, thereby reducing the power consumption of the electronic device during video decoding.

[0133] See also Fig.14 , Fig.14 A schematic diagram of a structure of an image processing device provided in an embodiment of the present application. The image processing device 30 is applied to an electronic device, wherein the electronic device includes a video decoder, wherein the image processing device 30 includes:

[0134] The acquisition module 301 is used to acquire the current video code stream and determine the initial frame rate of the video code stream;

[0135] The receiving module 302 is used to generate a power consumption limit instruction and determine a target frame rate according to the power consumption limit instruction;

[0136] A determination module 303, configured to determine a to-be-decoded image frame in the image group of the video code stream according to the initial frame rate and the target frame rate;

[0137] The decoding module 304 is used to dynamically adjust the working state of the video decoder according to the decoding order of the image frames to be decoded in the image group, so as to decode the image frames to be decoded.

[0138] In one embodiment, see Fig.15 In this embodiment, the determining module 303 may include:

[0139] A first determining submodule 3031 is used to determine a key image frame in the image group of the video code stream according to the initial frame rate and the target frame rate;

[0140] An acquisition submodule 3032 is used to acquire a decoding dependency relationship corresponding to the image group;

[0141] The second determining submodule 3033 is used to determine the reference image frame referenced by the key image frame according to the decoding dependency relationship, and determine the image frame to be decoded according to the key image frame and the reference image frame of the key image frame.

[0142] In one embodiment, the working state of the video decoder includes an on state and an off state, and the decoding module 304 may include:

[0143] A calculation submodule 3041, configured to calculate a decoding unit time in the picture group according to the target frame rate;

[0144] A third determining submodule 3042 is used to determine the number of image frames to be decoded within the decoding unit time according to the decoding order of the image frames to be decoded in the image group;

[0145] A processing submodule 3043, configured to divide the decoding unit time into a first time period and a second time period according to the number of image frames to be decoded in the decoding unit time;

[0146] The decoding submodule 3044 is used to adjust the video decoder to a closed state during the first time period, and to adjust the video decoder to an open state during the second time period, so as to decode the image frames to be decoded corresponding to the number of image frames to be decoded.

[0147] In another embodiment, the working state of the video decoder includes voltage intensity, wherein:

[0148] The processing submodule 3043 is further configured to adjust the operating voltage of the video decoder according to the number of image frames to be decoded;

[0149] The decoding submodule 3044 is further configured to decode the image frames to be decoded corresponding to the number of image frames to be decoded using the working voltage of the video decoder within the decoding unit time.

[0150] As can be seen from the above, the image processing device 30 of the embodiment of the present application can obtain the current video code stream and determine the initial frame rate of the video code stream, generate a power consumption limit instruction, and determine the target frame rate according to the power consumption limit instruction, determine the image frame to be decoded in the image group of the video code stream according to the initial frame rate and the target frame rate, and dynamically adjust the working state of the video decoder according to the decoding order of the image frame to be decoded in the image group to decode the image frame to be decoded. The embodiment of the present application can select the image frame to be decoded according to the target frame rate when power consumption limit is required, and decode the image frame to be decoded by dynamically adjusting the working state of the video decoder, thereby reducing the power consumption of the electronic device.

[0151] In the embodiment of the present application, the image processing device and the image processing method in the above embodiment belong to the same concept. Any method provided in the image processing method embodiment can be run on the image processing device. The specific implementation process is detailed in the embodiment of the image processing method and will not be repeated here.

[0152] The term "module" used herein can be regarded as a software object executed on the computing system. The different components, modules, engines and services described herein can be regarded as implementation objects on the computing system. The apparatus and method described herein can be implemented in software, and of course can also be implemented in hardware, all within the scope of protection of this application.

[0153] An embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed on a computer, the computer is enabled to execute the above-mentioned image processing method.

[0154] The embodiment of the present application also provides an electronic device, which includes a video decoder, such as a tablet computer, a mobile phone, etc. The processor in the electronic device will load instructions corresponding to the processes of one or more application programs into the memory according to the following steps, and the processor will run the application programs stored in the memory to implement various functions:

[0155] Obtaining a current video stream and determining an initial frame rate of the video stream;

[0156] generating a power consumption limit instruction, and determining a target frame rate according to the power consumption limit instruction;

[0157] Determining a to-be-decoded image frame in the image group of the video code stream according to the initial frame rate and the target frame rate; and

[0158] The working state of the video decoder is dynamically adjusted according to the decoding order of the image frames to be decoded in the image group to decode the image frames to be decoded.

[0159] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0160] See also Fig.16 , the electronic device 400 includes a processor 401 and a memory 402. The processor 401 is electrically connected to the memory 402.

[0161] The processor 401 is the control center of the electronic device 400. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device 400 and processes data by running or loading computer programs stored in the memory 402 and calling data stored in the memory 402, thereby monitoring the electronic device 400 as a whole.

[0162] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the computer programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0163] In the embodiment of the present application, the processor 401 in the electronic device 400 will load instructions corresponding to the processes of one or more computer programs into the memory 402 according to the following steps, and the processor 401 will run the computer program stored in the memory 402, thereby realizing various functions, as follows:

[0164] Obtaining a current video stream and determining an initial frame rate of the video stream;

[0165] generating a power consumption limit instruction, and determining a target frame rate according to the power consumption limit instruction;

[0166] Determining a to-be-decoded image frame in the image group of the video code stream according to the initial frame rate and the target frame rate; and

[0167] The working state of the video decoder is dynamically adjusted according to the decoding order of the image frames to be decoded in the image group to decode the image frames to be decoded.

[0168] Please also read Fig.17 In some implementations, the electronic device 400 may further include: a display 403, a radio frequency circuit 404, an audio circuit 405, a power supply 406, and a video decoder 407. The display 403, the radio frequency circuit 404, the audio circuit 405, the power supply 406, and the video decoder 407 are electrically connected to the processor 401, respectively.

[0169] The display 403 may be used to display information input by the user or information provided to the user and various graphical user interfaces, which may be composed of graphics, text, icons, videos, and any combination thereof. The display 403 may include a display panel, and in some embodiments, the display panel may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0170] The radio frequency circuit 404 can be used to send and receive radio frequency signals to establish wireless communication with network devices or other electronic devices through wireless communication, and to send and receive signals between network devices or other electronic devices. Generally, the radio frequency circuit 501 includes but is not limited to an antenna, at least one amplifier, a tuner, one or more oscillators, a subscriber identity module (SIM) card, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc.

[0171] The audio circuit 405 can be used to provide an audio interface between the user and the electronic device through a speaker and a microphone. The audio circuit 506 can convert the received audio data into an electrical signal, transmit it to the speaker, and convert it into a sound signal for output by the speaker.

[0172] The power supply 406 can be used to supply power to various components of the electronic device 400. In some embodiments, the power supply 406 can be logically connected to the processor 401 through a power management system, so that the power management system can manage charging, discharging, and power consumption. The power supply 406 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0173] Video decoder 407 Video decoder is a program (video player) or device that performs decompression operation on the encoded digital video. For example, if the system is installed with real encoder, it can convert other format files into rm or rmvb format. If real decoder is installed, it can play rm or rmvb format files. Among them, video decoders are divided into software decoders, hardware decoders and wireless decoders.

[0174] although Fig.17 Not shown, the electronic device 400 may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0175] In the embodiment of the present application, the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0176] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0177] It should be noted that, for the image processing method of the embodiment of the present application, ordinary testers in the field can understand that all or part of the process of implementing the image processing method of the embodiment of the present application can be completed by controlling the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, such as a memory of an electronic device, and executed by at least one processor in the electronic device, and the execution process may include the process of the embodiment of the image processing method. Among them, the storage medium can be a disk, an optical disk, a read-only memory, a random access memory, etc.

[0178] For the image processing device of the embodiment of the present application, each functional module can be integrated into a processing chip, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk.

[0179] The above is a detailed introduction to an image processing method, device, storage medium and electronic device provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, applied to an electronic device, wherein the electronic device comprises a video decoder, characterized in that: The method comprises the following steps: Obtaining a current video stream and determining an initial frame rate of the video stream; generating a power consumption limit instruction, and determining a target frame rate according to the power consumption limit instruction; Determining a to-be-decoded image frame in the image group of the video code stream according to the initial frame rate and the target frame rate; and Dynamically adjusting the working state of the video decoder according to the decoding order of the image frames to be decoded in the image group to decode the image frames to be decoded, wherein the working state of the video decoder includes an on state and an off state or a voltage intensity; The dynamically adjusting the working state of the video decoder according to the decoding order of the image frames to be decoded in the image group to decode the image frames to be decoded includes: Calculating a decoding unit time in the picture group according to the target frame rate; Determining the number of image frames to be decoded within the decoding unit time according to the decoding order of the image frames to be decoded in the image group; Dividing the decoding unit time into a first time period and a second time period according to the number of image frames to be decoded within the decoding unit time; During the first time period, adjusting the video decoder to a closed state; In the second time period, the video decoder is adjusted to be turned on to decode the image frames to be decoded corresponding to the number of image frames to be decoded; or, adjusting the operating voltage of the video decoder according to the number of image frames to be decoded; In the decoding unit time, the image frames to be decoded corresponding to the number of image frames to be decoded are decoded with the adjusted working voltage of the video decoder.

2. The image processing method according to claim 1, characterized in that: After obtaining the current video code stream and determining the initial frame rate of the video code stream, the method further includes: If the power consumption limiting instruction is not received, obtaining a refresh rate of the screen of the electronic device; The target frame rate is determined according to the refresh rate of the screen and the initial frame rate of the video code stream.

3. The image processing method according to claim 1, characterized in that: The step of determining the image frame to be decoded in the image group of the video code stream according to the initial frame rate and the target frame rate includes: Determining a key image frame in an image group of the video code stream according to the initial frame rate and the target frame rate; Obtaining a decoding dependency relationship corresponding to the image group; The reference image frame referenced by the key image frame is determined according to the decoding dependency relationship, and the image frame to be decoded is determined according to the key image frame and the reference image frame of the key image frame.

4. The image processing method according to claim 1, characterized in that: The decoding of the image frame to be decoded includes: Acquire a reference motion vector according to the current video code stream; Acquire a corresponding reference block from an image frame of the current video code stream according to the reference motion vector; The to-be-decoded block of the to-be-decoded image frame is decoded according to the reference block.

5. The image processing method according to claim 1, characterized in that: Before generating the power consumption limit instruction, the method further includes: Obtaining the current remaining power of the electronic device; Determining whether the remaining power is less than a preset value; If it is less than the preset value, a power consumption limiting instruction is generated according to the remaining power.

6. An image processing device, applied to an electronic device, wherein the electronic device comprises a video decoder, characterized in that: The device comprises: An acquisition module, used to acquire a current video code stream and determine an initial frame rate of the video code stream; A receiving module, configured to generate a power consumption limit instruction and determine a target frame rate according to the power consumption limit instruction; A determination module, configured to determine an image frame to be decoded in an image group of the video code stream according to the initial frame rate and the target frame rate; A decoding module, used for dynamically adjusting the working state of the video decoder according to the decoding order of the image frame to be decoded in the image group, so as to decode the image frame to be decoded, wherein the working state of the video decoder includes an on state and an off state or a voltage intensity; The decoding module comprises: A calculation submodule, configured to calculate a decoding unit time in the image group according to the target frame rate; A third determining submodule, configured to determine the number of image frames to be decoded within the decoding unit time according to a decoding order of the image frames to be decoded in the image group; A processing submodule, configured to divide the decoding unit time into a first time period and a second time period according to the number of image frames to be decoded within the decoding unit time; a decoding submodule, configured to adjust the video decoder to a closed state in the first time period, and to adjust the video decoder to an open state in the second time period, so as to decode the image frames to be decoded corresponding to the number of image frames to be decoded; or The processing submodule is further used to adjust the working voltage of the video decoder according to the number of image frames to be decoded; The decoding submodule is further configured to decode the image frames to be decoded corresponding to the number of image frames to be decoded using the working voltage of the video decoder within the decoding unit time.

7. The image processing device according to claim 6, characterized in that: The determination module comprises: A first determination submodule, configured to determine a key image frame in an image group of the video code stream according to the initial frame rate and the target frame rate; An acquisition submodule, used for acquiring a decoding dependency relationship corresponding to the image group; The second determination submodule is used to determine the reference image frame referenced by the key image frame according to the decoding dependency relationship, and determine the image frame to be decoded according to the key image frame and the reference image frame of the key image frame.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed on a computer, the computer is caused to execute the image processing method according to any one of claims 1 to 5.

9. An electronic device comprising a processor, a memory and a video decoder, wherein the memory stores a plurality of instructions, characterized in that: The processor loads the instructions in the memory for executing the following steps: Obtaining a current video stream and determining an initial frame rate of the video stream; generating a power consumption limit instruction, and determining a target frame rate according to the power consumption limit instruction; Determine a to-be-decoded image frame in the image group of the video code stream according to the initial frame rate and the target frame rate; as well as Dynamically adjusting the working state of the video decoder according to the decoding order of the image frames to be decoded in the image group to decode the image frames to be decoded, wherein the working state of the video decoder includes an on state and an off state or a voltage intensity; The dynamically adjusting the working state of the video decoder according to the decoding order of the image frames to be decoded in the image group to decode the image frames to be decoded includes: Calculating a decoding unit time in the picture group according to the target frame rate; Determining the number of image frames to be decoded within the decoding unit time according to the decoding order of the image frames to be decoded in the image group; Dividing the decoding unit time into a first time period and a second time period according to the number of image frames to be decoded within the decoding unit time; During the first time period, adjusting the video decoder to a closed state; In the second time period, the video decoder is adjusted to be turned on to decode the image frames to be decoded corresponding to the number of image frames to be decoded; or, adjusting the operating voltage of the video decoder according to the number of image frames to be decoded; In the decoding unit time, the image frames to be decoded corresponding to the number of image frames to be decoded are decoded with the adjusted working voltage of the video decoder.

Images