Video coding method and device, video decoding method and device, electronic equipment and medium

Abstract

The invention discloses a video coding method and device, a video decoding method and device, electronic equipment and a medium, and belongs to the technical field of electronic equipment. The video coding method comprises the following steps: performing highlight detection on a video stream; and coding the video frame according to the video stream obtained by the highlight detection to obtain a coded video stream.

Description

Technical Field

[0001] This application belongs to the field of electronic equipment technology, and specifically relates to a video encoding and decoding method, apparatus, electronic equipment and medium. Background Technology

[0002] With the increasing demand for video from mobile devices such as smartphones and cameras, the pursuit of video quality is also gradually rising.

[0003] Currently, there are two main types of encoding methods. One is BP frame encoding, which encodes the image by predicting intermediate frames. This results in high video compression rates, but relatively lower video quality. The other is full keyframe (I-frame) encoding, which does not perform predictions and renders each frame as a real image. The trade-off is higher encoder requirements, reduced power consumption, and significantly larger video storage space requirements.

[0004] Current video coding technologies generally adopt a fixed group of pictures (GOP) structure, which leads to a rigid allocation of coding resources between non-keyframes (B-frames, P-frames) and keyframes (I-frames). Summary of the Invention

[0005] This application provides a video encoding and decoding method, apparatus, electronic device, and medium to solve the problem that current encoding methods result in rigid allocation of encoding resources, which cannot effectively save resources or improve resource utilization.

[0006] In a first aspect, embodiments of this application provide a video encoding method, including:

[0007] Perform specular detection on the video stream;

[0008] Based on the video stream obtained from specular detection, video frames are encoded to obtain the encoded video stream.

[0009] Optionally, the step of encoding video frames based on the video stream obtained from specular detection to obtain an encoded video stream includes:

[0010] Obtain the highlight tags of video frames in the video stream. For the first video frame that has a highlight tag, encode the first video frame as an I-frame.

[0011] For the second video frame that does not have a highlight tag, encoding is performed based on a target method, which is to extend the group of pictures (GOP) interval and use non-keyframe encoding.

[0012] Optionally, the step of encoding video frames based on the video stream obtained from specular detection to obtain an encoded video stream further includes:

[0013] Video frames whose time difference from the time of the first video frame is within a preset time range are encoded as I-frames.

[0014] Optionally, the method further includes:

[0015] During video frame encoding, the size of the buffer is dynamically adjusted according to the number of I-frames being encoded;

[0016] After all I-frames associated with the first video frame have been encoded, the buffer is released.

[0017] Optionally, the highlight detection of the video stream includes:

[0018] Perform specular detection on each video frame in the video stream;

[0019] If a detection point related to the target video frame exists, the target video frame is marked with a highlight tag;

[0020] The detection points are pre-configured and / or user-set.

[0021] Secondly, embodiments of this application also provide a video decoding method, including:

[0022] Decode the video stream;

[0023] Obtain the highlight tags of the decoded video frames;

[0024] Displays the decoded video stream and the highlight tags associated with the video frames.

[0025] Thirdly, embodiments of this application also provide a video encoding apparatus, comprising:

[0026] The detection module is used to perform highlight detection on the video stream;

[0027] An encoding module is used to encode video frames based on the video stream obtained from specular detection, to obtain an encoded video stream. Optionally, the encoding module includes:

[0028] The first encoding unit is used to obtain the highlight tags of video frames in the video stream, and to encode the first video frame as an I-frame for the first video frame that has a highlight tag.

[0029] The second encoding unit is used to encode the second video frame that does not have a highlight tag based on a target method, which is to extend the group of pictures (GOP) interval and use non-keyframe encoding.

[0030] Fourthly, embodiments of this application also provide a video decoding apparatus, including:

[0031] The decoding module is used to decode the video stream;

[0032] The acquisition module is used to acquire the highlight tags of the decoded video frames;

[0033] The display module is used to display the decoded video stream and the highlight tags associated with the video frames.

[0034] Fifthly, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the first or second aspect described above.

[0035] In a sixth aspect, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the first or second aspect.

[0036] In a seventh aspect, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the first aspect or the second aspect.

[0037] In this embodiment of the application, by performing highlight detection on the video stream, and encoding video frames based on the video stream obtained from the highlight detection, an encoded video stream is obtained. By performing highlight detection before encoding and encoding based on the results of the highlight detection, resources can be effectively saved and resource utilization can be improved. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the video encoding method according to an embodiment of this application;

[0039] Figure 2 This is a flowchart illustrating application scenario one;

[0040] Figure 3 This is a schematic diagram of the video display interface for application scenario one;

[0041] Figure 4 This is a flowchart illustrating application scenario two;

[0042] Figure 5 This is a schematic diagram of the video display interface in application scenario two;

[0043] Figure 6 This is a schematic diagram of the video encoding apparatus according to an embodiment of this application;

[0044] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0045] Figure 8This is a flowchart illustrating the video decoding method according to an embodiment of this application;

[0046] Figure 9 This is a schematic diagram of the video decoding device according to an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0049] The video encoding and decoding methods, apparatuses, electronic devices, and media provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0050] like Figure 1 As shown, this application provides a video encoding method, including:

[0051] Step 101: Perform specular detection on the video stream;

[0052] Step 102: Based on the video stream obtained from specular detection, encode the video frames to obtain the encoded video stream.

[0053] It should be noted that, in this embodiment of the application, highlight detection is performed on the video stream, and video frames are encoded based on the video stream obtained from the highlight detection to obtain the encoded video stream. By performing highlight detection before encoding and encoding based on the results of the highlight detection, resources can be effectively saved and resource utilization can be improved.

[0054] Alternatively, the video stream to be encoded can be a recorded video stream obtained based on the user's recording operation.

[0055] Optionally, the highlight detection of the video stream includes steps 11 and 12:

[0056] Step 11: Perform specular detection on each video frame in the video stream;

[0057] It should be noted that, in this embodiment of the application, highlight detection can be performed on video frames based on pre-set detection points.

[0058] Optionally, the detection point can be pre-configured and / or user-set. For example, the detection point may be pre-set within the application during setup; or it may be set by the user based on their needs.

[0059] For example, the detection point could be the amplitude of a person's movements, their facial expressions, or other key elements in the video frame. Alternatively, the detection point could be user interaction data, bullet comments, audio spikes, etc.

[0060] Step 12: If there are detection points related to the target video frame, mark the target video frame with a highlight tag;

[0061] Optionally, this step involves filtering video frames, that is, identifying which video frame in the video stream corresponds to the relevant detection, and then assigning a highlight tag to that video frame to distinguish it from other video frames. Optionally, the video frames with the highlight tag can be called highlight frames, keyframes, or important frames.

[0062] It should be noted that, in this embodiment of the application, after the video stream is acquired, highlight detection needs to be performed on the video frames contained in the video stream. If a detection point related to the video frame can be detected (detecting a detection point related to the video frame can be understood as the video frame containing a detection point, or it can also be understood as the moment when the detection point appears corresponds to the video frame), then the video frame is a frame that needs to be focused on, and the video frame is considered a highlight frame, and a highlight label needs to be set for the video frame.

[0063] Alternatively, in one implementation, detection points in the video frame can be directly detected. If the video frame includes the area corresponding to the detection point, then the video frame is considered a highlight frame. For example, the detection point could be the amplitude of a person's movement in the video frame; if the amplitude of the movement exceeds a specific threshold, the video frame is considered a highlight frame. Or, the detection point could be a person's facial expression; if the person's expression is one of hearty laughter, then the video frame is considered a highlight frame. In another implementation, the presence of key content requiring attention in the environment scene can be detected. If key content requiring attention exists, then that content is considered important. The video frame corresponding to the content is a highlight frame. For example, if the detection point is user interaction data, and the user interaction data is "like", then the video frame corresponding to that interaction data is a highlight frame; if the detection point is bullet screen content, and the user interaction data is "very good", then the video frame corresponding to that bullet screen content is a highlight frame; if the detection point is the number of times a certain word appears in the user interaction data, and the number of times a certain word appears exceeds a threshold, then the video frame corresponding to that bullet screen content is a highlight frame; if the detection point is an audio spike, and the audio reaches an audio spike, then the video frame corresponding to that audio spike is a highlight frame.

[0064] Optionally, the highlight tag needs to be encoded into the bitstream and passed to the decoding end during encoding. After receiving the video stream, the decoding end decodes the video stream, obtains the highlight tags of the video frames, and displays the decoded video stream and the highlight tags associated with the video frames. By displaying the highlight tags, users can easily view the highlight frames and perform operations on them.

[0065] It should be noted that after the highlight frames are divided, video frame encoding can be performed. Optionally, the specific implementation of encoding video frames based on the video stream obtained from highlight detection to obtain the encoded video stream includes steps 21 and 22:

[0066] Step 21: Obtain the highlight tags of video frames in the video stream. For the first video frame with a highlight tag, encode the first video frame as an I-frame.

[0067] It should be noted that current encoding technologies encode I-frames at specific locations based on pre-set I-frame rules, i.e., encoding video frames at specific locations as I-frames. However, in this embodiment, it is necessary to first determine the highlight frames. After determining the highlight frames, they need to be encoded as I-frames. This allows for I-frame encoding based on requirements, ensuring that the image quality of the highlight frames meets the necessary specifications. Optionally, encoding highlight frames as I-frames can be understood as forcibly inserting I-frames into the corresponding highlight frames.

[0068] Step 22: For the second video frame that does not have a highlight tag, encode it based on the target method, which is to extend the group of pictures (GOP) interval and use non-keyframe encoding.

[0069] It should be noted that the non-keyframe encoding mentioned in the embodiments of this application can be understood as encoding the second video frame as a non-I frame, for example, the second video frame can be encoded as a B / P frame.

[0070] Optionally, step 12 can be implemented as follows: for the second video frame, the overall bitrate is calculated, and if the bitrate remains basically unchanged or changes little, the interval time of the GOP is extended and B-frames or P-frames are used for encoding, without inserting I-frames.

[0071] It should be noted that steps 11 and 12 can transform the original uniform encoding of I-frames into encoding I-frames concentrated on the highlight parts (i.e., the key parts or critical parts), thereby improving the image quality of the highlight parts of the video.

[0072] Optionally, the specific implementation of encoding video frames from the video stream obtained by specular detection to obtain the encoded video stream further includes:

[0073] Video frames whose time difference from the time of the first video frame is within a preset time range are encoded as I-frames.

[0074] It should be noted that, in order to further improve the image quality of the highlight frame, in this embodiment of the application, other video frames around the highlight frame are also encoded as I-frames. For example, video frames within a range of 1 second or 0.5 seconds before and after the highlight frame (optionally, this value can be set according to the requirements) are all encoded as I-frames, so as to meet the image quality requirements of the highlight frame.

[0075] For example, a video stream may consist of 200 video frames. Using existing methods, an I-frame needs to be encoded every 5 frames, requiring a total of 40 I-frames. However, in this embodiment, after highlight detection, frames 20 and 70 are identified as highlight frames. For instance, if the video frames within 1 second before and after a highlight frame constitute the 50 frames before and after the highlight frame, then during encoding, frames 15 to 25 and 65 to 75 are encoded as I-frames, while the remaining video frames are encoded as B-frames or P-frames. Only 12 I-frames need to be encoded, reducing the number of I-frames required. The encoding method in this embodiment allows highlight frames to be concentrated in a specific area, effectively saving resources and improving resource utilization.

[0076] It should be noted that the above is merely an example of an embodiment of this application and does not constitute a limitation on the embodiments of this application.

[0077] Optionally, the method further includes:

[0078] During video frame encoding, the size of the buffer is dynamically adjusted according to the number of I-frames to be encoded.

[0079] After all I-frames associated with the first video frame have been encoded, the buffer is released.

[0080] Optionally, because the encoding method of this application embodiment increases the number of I-frames in a short period of time, in order to avoid the I-frame surge causing encoding abnormalities, the number of I-frames to be encoded can be known before video encoding in this application embodiment. During the encoding process, the size of the buffer is dynamically adjusted in real time based on the number of I-frames to avoid the I-frame surge causing overflow, thereby ensuring that the buffer demand is met. Furthermore, in order to avoid the buffer being occupied for a long time, the buffer needs to be released in a timely manner after the highlight frames are encoded in this application embodiment. For example, during highlight detection, if frames 50 and 150 out of 500 video frames are identified as highlight frames, then when encoding frames 50 and 100, the preceding and following 10 frames need to be encoded as I-frames. Therefore, the number of I-frames for encoding frames 50 and 100 is 21 video frames. During the encoding process, the buffer size is adjusted to 21 before encoding frame 40, and the buffer is released after encoding frame 60. Encoding continues, and the buffer size is adjusted to 21 before encoding frame 100, and the buffer is released after encoding frame 110. This achieves dynamic adjustment of the buffer, ensuring encoding requirements while avoiding prolonged buffer occupation.

[0081] The specific implementation of the embodiments of this application is illustrated below with examples.

[0082] Application Scenario 1: Specific Implementation in Video Recording Scenarios

[0083] like Figure 2 As shown, the main implementation process for this application includes:

[0084] Step 201: The user turns on the camera and enters video recording mode to record.

[0085] Specifically, the user opens the camera, enters video recording mode, and clicks the video recording button to start recording.

[0086] Step 202: Perform highlight detection on the video frames obtained from the video recording.

[0087] Optionally, the video frames obtained in this step can be video frames captured after processing by an Image Signal Processor (ISP). It should be noted that processing by an ISP yields the best video frames, which are easier for image detection and other algorithms to recognize and process.

[0088] The purpose of detecting key moments in a video is to identify the most exciting parts of the video by analyzing the detection points in the video frames.

[0089] Optionally, the detection point may include, but is not limited to, at least one of the following:

[0090] Movement amplitude, audio peak, facial expressions, and user interaction data.

[0091] It should be noted that, based on the highlight detection results, the detection results for each video frame are output. Optionally, the detection results can be an indication of whether it is a highlight frame; or, no detection results are output for non-highlight frames, and detection results indicating that it is a highlight frame are output for highlight frames, and highlight frames can be labeled with highlight tags.

[0092] Step 203: Based on the video frames after dividing the highlight frames, encode the video frames to obtain the encoded video stream.

[0093] Based on step 202, the video stream output after highlight detection is sent to the encoder's differential encoding engine. After reading the detection results of each video frame in step 202, the encoder forcibly inserts I-frames into highlight frames and increases the quantization parameter (QP). For non-highlight frames, the GOP interval is extended and B-frames or P-frames are used for encoding. That is, for highlight frames, I-frames are inserted within 1 second before and after the highlight frame (for example, if the frame rate is 30fps, 30 consecutive I-frames are encoded within the 1-second video segment corresponding to the highlight frame). For non-highlight frames, based on the overall bitrate calculation, if the bitrate remains basically unchanged or changes only slightly, the GOP interval is extended and B / P frames are used for encoding; for example, if GOP=30, no I-frames are inserted. This is equivalent to transforming the original evenly distributed I-frames into a distribution where I-frames are concentrated in the highlight frames.

[0094] Optionally, a buffer compensation mechanism can be added to the encoder during the encoding process to prevent anomalies caused by a surge in I-frames. Specifically, based on the detection results of highlight frames, highlight frames are identified in advance, indicating that an I-frame is about to enter encoding. The size of the buffer area is dynamically increased to prevent overflow caused by a surge in I-frames. The buffer is released immediately after the highlight segment ends to avoid prolonged memory spikes.

[0095] Step 204: Output video stream.

[0096] Output the video stream and mark the video with highlight tags (e.g., add stars to highlight frames) to make it easier for users to view highlight frames and facilitate post-production editing.

[0097] like Figure 3 As shown, on the display interface 31 of the electronic device, a highlight label 33 (for example, a star) is added above the video frames of the video stream 32 to the highlight frames, so that the user can clearly know the position of the highlight frames for easy use later.

[0098] It should be noted that this application can improve the image quality of key parts of the video by performing differential encoding on the highlight parts of the video frames, while also saving resources and improving resource utilization.

[0099] Application Scenario 2: Specific Implementation in Live Streaming Scenarios

[0100] like Figure 4 As shown, the main implementations of this application scenario include:

[0101] Step 401: The user opens the live streaming interface to start the live stream.

[0102] Users open an application that enables live streaming and record the live stream by accessing their camera.

[0103] Step 402: Perform highlight detection on the video frames obtained from the live recording.

[0104] Optionally, the video frames obtained in this step can be video frames captured after processing by an Image Signal Processor (ISP). It should be noted that processing by an ISP yields the best video frames, which are easier for image detection and other algorithms to recognize and process.

[0105] The purpose of detecting key moments in a video is to identify the most exciting parts of the video by analyzing the detection points in the video frames.

[0106] Optionally, the detection point may include, but is not limited to, at least one of the following:

[0107] The detection targets, audio spikes, human actions, and bullet screen information (e.g., bullet screen content, content density, etc.) are combined with the live stream subject (e.g., product sales, outdoor lighting) to set the detection targets.

[0108] It should be noted that, based on the highlight detection results, the detection results for each video frame are output. Optionally, the detection results can be an indication of whether it is a highlight frame; or, no detection results are output for non-highlight frames, and detection results indicating that it is a highlight frame are output for highlight frames, and highlight frames can be labeled with highlight tags.

[0109] Step 403: Based on the video frames after dividing the highlight frames, encode the video frames to obtain the encoded video stream.

[0110] Based on step 402, the video stream output after highlight detection is sent to the encoder's differential encoding engine. After reading the detection results of each video frame in step 402, the encoder forcibly inserts I-frames into highlight frames and increases the quantization parameter (QP). For non-highlight frames, the GOP interval is extended and B-frames or P-frames are used for encoding. That is, for highlight frames, I-frames are inserted within 1 second before and after the highlight frame (for example, if the frame rate is 30fps, 30 consecutive I-frames are encoded within the 1-second video segment corresponding to the highlight frame). For non-highlight frames, based on the overall bitrate calculation, if the bitrate remains basically unchanged or changes only slightly, the GOP interval is extended and B / P frames are used for encoding; for example, if GOP=30, no I-frames are inserted. This is equivalent to transforming the original evenly distributed I-frames into a distribution where I-frames are concentrated in the highlight frames.

[0111] Optionally, a buffer compensation mechanism can be added to the encoder during the encoding process to prevent anomalies caused by a surge in I-frames. Specifically, this can be achieved by identifying highlight frames in advance based on their detection results; that is, if an I-frame is about to enter encoding, the buffer area size can be dynamically increased to prevent overflow caused by a surge in I-frames. Once the highlight segment ends, the buffer is immediately released to prevent prolonged memory spikes.

[0112] Step 404: Output video stream.

[0113] Output the video stream and mark the video with highlight tags (for example, add a star to the highlight frame), send it back to the live streaming software and push it to the client. When the client decodes and displays the video, it not only displays the video stream, but also the highlight tags corresponding to the video frames.

[0114] like Figure 5 As shown, on the display interface of the electronic device, a highlight tag 52 (for example, a star tag) is added to the highlight frame on the live broadcast interface 51, so that the user can clearly know the location of the highlight frame and focus on it.

[0115] Step 405: Client-side decoding and rendering, specifically rendering the video.

[0116] The client reads highlight video clips, performs targeted rendering, and enhances the image to improve the user experience.

[0117] It should be noted that this application can improve the quality of live streaming and enhance the user's live streaming experience by performing differentiated encoding and decoding rendering of the highlight parts of video frames.

[0118] It should be noted that the embodiments of this application can solve the problem of insufficient highlight frame quality in video footage and improve resource utilization.

[0119] like Figure 6As shown, at least one embodiment of this application also provides a video encoding apparatus 600, comprising:

[0120] Detection module 601 is used to perform highlight detection on the video stream;

[0121] The encoding module 602 is used to encode video frames based on the video stream obtained from specular detection to obtain the encoded video stream.

[0122] Optionally, the encoding module 602 includes:

[0123] The encoding module includes:

[0124] The first encoding unit is used to obtain the highlight tags of video frames in the video stream, and to encode the first video frame as an I-frame for the first video frame that has a highlight tag.

[0125] The second encoding unit is used to encode the second video frame that does not have a highlight tag based on a target method, which is to extend the group of pictures (GOP) interval and use non-keyframe encoding.

[0126] Optionally, the encoding module 602 further includes:

[0127] The third encoding unit is used to encode video frames whose time difference from the time of the first video frame is within a preset time range into I-frames.

[0128] Optionally, the device further includes:

[0129] The adjustment module is used to dynamically adjust the size of the buffer according to the number of I-frames being encoded during the video frame encoding process;

[0130] The release module is used to release the buffer after all I-frames associated with the first video frame have been encoded.

[0131] Optionally, the detection module 601 includes:

[0132] The detection unit is used to perform highlight detection on each video frame in the video stream.

[0133] A tagging unit is used to tag the target video frame with a highlight tag when there are detection points associated with the target video frame;

[0134] The detection points are pre-configured and / or user-set.

[0135] It should be noted that this device embodiment is a device corresponding to the above method. All implementation methods in the above method embodiment are applicable to this device embodiment and can achieve the same technical effect.

[0136] The video encoding device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method embodiments achieve the same technical effect, and will not be described again here to avoid repetition.

[0137] This application embodiment also provides an electronic device, including a processor and a communication interface, wherein the processor is used to perform highlight detection on a video stream;

[0138] Based on the video stream obtained from specular detection, video frames are encoded to obtain the encoded video stream.

[0139] Optionally, the processor is used to:

[0140] Obtain the highlight tags of video frames in the video stream. For the first video frame that has a highlight tag, encode the first video frame as an I-frame.

[0141] For the second video frame that does not have a highlight tag, encoding is performed based on a target method, which is to extend the group of pictures (GOP) interval and use non-keyframe encoding.

[0142] Optionally, the processor is used to:

[0143] Video frames whose time difference from the time of the first video frame is within a preset time range are encoded as I-frames.

[0144] Optionally, the processor is further configured to:

[0145] During video frame encoding, the size of the buffer is dynamically adjusted according to the number of I-frames being encoded;

[0146] After all I-frames associated with the first video frame have been encoded, the buffer is released.

[0147] Optionally, the processor is used to:

[0148] Perform specular detection on each video frame in the video stream;

[0149] If a detection point related to the target video frame exists, the target video frame is marked with a highlight tag;

[0150] The detection points are pre-configured and / or user-set.

[0151] Preferably, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When executed by the processor, the program or instructions implement the various processes of the video encoding method embodiments described above, and achieve the same technical effect. Specifically, Figure 7A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0152] The electronic device 700 includes, but is not limited to, at least some of the following components: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710.

[0153] Those skilled in the art will understand that the electronic device 700 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 710 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0154] It should be understood that, in this embodiment, the input unit 704 may include a graphics processor 7041 and a microphone 7042. The graphics processor 7041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 706 may include a display panel 7061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts: a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0155] In this embodiment, after receiving downlink data from the access network device, the radio frequency unit 701 can transmit it to the processor 710 for processing; in addition, the radio frequency unit 701 can send uplink data to the network-side device. Typically, the radio frequency unit 701 includes, but is not limited to, antennas, amplifiers, transceivers, couplers, low-noise amplifiers, duplexers, etc.

[0156] The memory 709 can be used to store software programs or instructions, as well as various data. The memory 709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 709 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 709 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0157] Processor 710 may include one or more processing units; optionally, processor 710 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 710.

[0158] The processor 710 is used for:

[0159] Perform specular detection on the video stream;

[0160] Based on the video stream obtained from specular detection, video frames are encoded to obtain the encoded video stream.

[0161] Optionally, the processor 710 is used for:

[0162] Obtain the highlight tags of video frames in the video stream. For the first video frame that has a highlight tag, encode the first video frame as an I-frame.

[0163] For the second video frame that does not have a highlight tag, encoding is performed based on a target method, which is to extend the group of pictures (GOP) interval and use non-keyframe encoding.

[0164] Optionally, the processor 710 is used for:

[0165] Video frames whose time difference from the time of the first video frame is within a preset time range are encoded as I-frames.

[0166] Optionally, the processor 710 is further configured to:

[0167] During video frame encoding, the size of the buffer is dynamically adjusted according to the number of I-frames being encoded;

[0168] After all I-frames associated with the first video frame have been encoded, the buffer is released.

[0169] Optionally, the processor 710 is used for:

[0170] Perform specular detection on each video frame in the video stream;

[0171] If a detection point related to the target video frame exists, the target video frame is marked with a highlight tag;

[0172] The detection points are pre-configured and / or user-set.

[0173] like Figure 8 As shown in the embodiments of this application, a video decoding method is also provided, including:

[0174] Step 801: Decode the video stream;

[0175] Step 802: Obtain the highlight tags of the decoded video frames;

[0176] Step 803: Display the decoded video stream and the highlight tags associated with the video frames.

[0177] It should be noted that, in order to facilitate users' viewing of highlight frames and make post-production easier, highlight frames are marked with highlight tags during video encoding and the highlight tags are passed to the decoding end. After decoding the video stream, the decoding end displays the highlight tags on the video stream playback interface, thereby facilitating users' viewing of highlight frames.

[0178] like Figure 9 As shown in the illustration, this application also provides a video decoding device, including:

[0179] Decoding module 901 is used to decode the video stream;

[0180] The acquisition module 902 is used to acquire the highlight tags of the decoded video frames;

[0181] Display module 903 is used to display the decoded video stream and the highlight tags associated with the video frames.

[0182] It should be noted that this device embodiment is a device corresponding to the above method. All implementation methods in the above method embodiment are applicable to this device embodiment and can achieve the same technical effect.

[0183] The video decoding device provided in this application embodiment can achieve... Figure 8 The various processes implemented in the method embodiments achieve the same technical effect, and will not be described again here to avoid repetition.

[0184] This application also provides an electronic device, including a processor and a communication interface, wherein the processor is used to decode a video stream; obtain highlight tags of the decoded video frames; and display the decoded video stream and the highlight tags associated with the video frames.

[0185] Preferably, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When executed by the processor, the program or instructions implement the various processes of the video decoding method embodiments described above, and achieve the same technical effect. The structure of this electronic device can be found in [reference needed]. Figure 7 As shown, it will not be elaborated further here.

[0186] The processor is used for:

[0187] Decode the video stream;

[0188] Obtain the highlight tags of the decoded video frames;

[0189] Displays the decoded video stream and the highlight tags associated with the video frames.

[0190] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described video encoding method or video decoding method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0191] The readable storage medium can be non-volatile or non-transient. The readable storage medium can include computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0192] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described video encoding method or video decoding method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0193] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0194] This application also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described video encoding method or video decoding method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0195] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A video encoding method, characterized in that, include: Perform specular detection on the video stream; Based on the video stream obtained from specular detection, video frames are encoded to obtain the encoded video stream.

2. The method according to claim 1, characterized in that, The step of encoding video frames from the video stream obtained by specular detection to obtain an encoded video stream includes: Obtain the highlight tags of video frames in the video stream. For the first video frame that has a highlight tag, encode the first video frame as an I-frame. For the second video frame that does not have a highlight tag, encoding is performed based on a target method, which is to extend the group of pictures (GOP) interval and use non-keyframe encoding.

3. The method according to claim 2, characterized in that, The step of encoding video frames from the video stream obtained by specular detection to obtain an encoded video stream further includes: Video frames whose time difference from the time of the first video frame is within a preset time range are encoded as I-frames.

4. The method according to claim 2 or 3, characterized in that, Also includes: During video frame encoding, the size of the buffer is dynamically adjusted according to the number of I-frames being encoded; After all I-frames associated with the first video frame have been encoded, the buffer is released.

5. The method according to claim 1, characterized in that, The process of performing highlight detection on the video stream includes: Perform specular detection on each video frame in the video stream; If a detection point related to the target video frame exists, the target video frame is marked with a highlight tag; The detection points are pre-configured and / or user-set.

6. A video decoding method, characterized in that, include: Decode the video stream; Obtain the highlight tags of the decoded video frames; Displays the decoded video stream and the highlight tags associated with the video frames.

7. A video encoding device, characterized in that, include: The detection module is used to perform highlight detection on the video stream; The encoding module is used to encode video frames based on the video stream obtained from specular detection, resulting in an encoded video stream.

8. The apparatus according to claim 7, characterized in that, The encoding module includes: The first encoding unit is used to obtain the highlight tags of video frames in the video stream, and to encode the first video frame as an I-frame for the first video frame that has a highlight tag. The second encoding unit is used to encode the second video frame that does not have a highlight tag based on a target method, which is to extend the group of pictures (GOP) interval and use non-keyframe encoding.

9. A video decoding device, characterized in that, include: The decoding module is used to decode the video stream; The acquisition module is used to acquire the highlight tags of the decoded video frames; The display module is used to display the decoded video stream and the highlight tags associated with the video frames.

10. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of the video encoding method as described in any one of claims 1-5 or the steps of the video decoding method as described in claim 6.

11. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the video encoding method as described in any one of claims 1-5 or the steps of the video decoding method as described in claim 6.

Images