Video encoding method, apparatus, device, and medium
Patent Information
- Application Number
- CN202510438786.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-08
- Publication Date
- 2026-10-16
Smart Images
Figure CN122933897A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video processing technology, and more specifically, to a video encoding method, apparatus, device, and medium. Background Technology
[0002] Currently, video transmission between devices is widely used in scenarios such as live streaming, video calls, and online conferencing. Therefore, as users' demands for real-time performance and image quality continue to increase, how to effectively utilize limited bandwidth resources to achieve high-quality video transmission has become an urgent technical problem to be solved. Summary of the Invention
[0003] One objective of this application is to provide a new technical solution for video encoding.
[0004] According to a first aspect of this application, a video coding method is provided, comprising:
[0005] Acquire a first image and a second image from the video, wherein the first image is the frame preceding the second image;
[0006] Based on the first image and the second image, the video scene type is determined, and the video scene type includes: static scene and dynamic scene;
[0007] Based on the video scene type, the parameter values of the encoding parameters of the video encoder are set to parameter values that match the video scene type;
[0008] The video encoder is controlled to encode the first image and the image of a preset number of subsequent frames to obtain the encoded video segment to be transmitted.
[0009] Optionally, it includes: determining the video scene type based on the first image and the second image, including:
[0010] Based on the first image and the second image, determine the corner motion quantization value and / or the motion region quantization value;
[0011] The video scene type is determined based on the corner motion quantization value and / or motion region quantization value;
[0012] Wherein, the corner motion quantization value is the quantization value of the motion vector formed by the corner point in the second image moving from the corresponding position in the first image to the corresponding position in the second image, and the motion region quantization value is the quantization value of the region size of the pixel-formed area describing the motion of the second image compared to the first image.
[0013] Optionally, before determining the corner motion quantization value and / or motion region quantization value based on the first image and the second image, the method further includes:
[0014] The first image and the second image are respectively subjected to grayscale processing to obtain a first grayscale image corresponding to the first image and a second grayscale image corresponding to the second image;
[0015] Update the first image to the first grayscale image;
[0016] Update the second image to the second grayscale image.
[0017] Optionally, determining the corner motion quantization value and / or motion region quantization value based on the first image and the second image includes:
[0018] Based on the second image, determine the first position of each corner point in the second image;
[0019] For any of the corner points, determine the second position of the corner point in the first image from the first image;
[0020] The motion quantization value of the corner point is determined based on the first position of the corner point in the second image and the second position of the corner point in the first image.
[0021] Optionally, determining the corner motion quantization value and / or motion region quantization value based on the first image and the second image includes:
[0022] Based on the first image and the second image, a difference image is determined, wherein the pixel value of any pixel in the difference image is the absolute value of the difference between the pixel values of the corresponding pixels in the first image and the second image;
[0023] The difference image is binarized according to a preset threshold to obtain a binarized image. The preset threshold is the minimum change in pixel value caused by the movement of an object in the image.
[0024] Based on the binarized image, determine the quantization value of the motion region.
[0025] Optionally, determining the video scene type based on the corner motion quantization value and / or motion region quantization value includes:
[0026] If the corner motion quantization value is greater than or equal to the preset corner motion change amount, the video scene type is determined to be a dynamic scene;
[0027] Alternatively, if the motion region quantization value is greater than or equal to the preset motion region quantization value, the video scene type is determined to be a dynamic scene;
[0028] Alternatively, normalization processing can be performed on the corner motion quantization value and the motion region quantization value respectively to obtain normalized corner motion quantization value and normalized motion region quantization value;
[0029] The motion quantization value is determined based on the normalized corner motion quantization value and the normalized motion region quantization value;
[0030] If the motion quantization value is greater than or equal to the preset motion quantization value, the video scene type is determined to be a dynamic scene.
[0031] Optionally, the encoding parameters include at least one of frame rate, bit rate, and GOP, and setting the parameter values of the video encoder's encoding parameters to match the video scene type according to the video scene type includes:
[0032] When the video scene type is a static scene, the frame rate of the video encoder is set to a first frame rate, the bit rate to a first bit rate, and the GOP to a first GOP;
[0033] When the video scene type is a dynamic scene, the frame rate of the video encoder is set to a second frame rate, the bit rate is set to a second bit rate, and the GOP is set to a second GOP at least one of the following:
[0034] Wherein, the first frame rate is less than the second frame rate, the first bitrate is less than the second bitrate, and the first GOP is greater than the second GOP.
[0035] According to a second aspect of this application, a video encoding apparatus is provided, comprising:
[0036] The acquisition module is used to acquire a first image and a second image from the video, wherein the first image is the frame preceding the second image;
[0037] The determining module is used to determine the video scene type based on the first image and the second image, wherein the video scene type includes: static scene and dynamic scene;
[0038] The setting module is used to set the parameter values of the encoding parameters of the video encoder to parameter values that match the video scene type, according to the video scene type.
[0039] The control module is used to control the video encoder to encode the first image and the image with a preset number of subsequent frames to obtain the encoded video segment to be transmitted.
[0040] According to a third aspect of this application, an electronic device is provided, the electronic device comprising the means as described in the second aspect;
[0041] Alternatively, the electronic device includes a memory and a processor, the memory for storing computer instructions and the processor for retrieving the computer instructions from the memory to perform the method as described in any one of the first aspects.
[0042] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the first aspects.
[0043] This application provides a video encoding method, which includes: acquiring a first image and a second image from a video, wherein the first image is the frame preceding the second image; determining a video scene type based on the first image and the second image, wherein the video scene type includes a static scene and a dynamic scene; setting the encoding parameters of the video encoder to values matching the video scene type; and controlling the video encoder to encode the first image and a preset number of subsequent frames to obtain an encoded video segment to be transmitted. This method provides a foundation for achieving high-quality video transmission by utilizing effective bandwidth resources.
[0044] Other features and advantages of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present application and, together with their description, serve to explain the principles of the present application.
[0046] Figure 1 This is a hardware configuration framework for an electronic device implementing a video encoding method according to an embodiment of this application. Figure 1 ;
[0047] Figure 2 This is a flowchart illustrating a video encoding method according to an embodiment of this application;
[0048] Figure 3 This is a schematic diagram of a video encoding device according to an embodiment of this application;
[0049] Figure 4 This is a hardware configuration framework for an electronic device implementing a video encoding method according to an embodiment of this application. Figure 2 . Detailed Implementation
[0050] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present application.
[0051] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.
[0052] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.
[0053] In all the examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.
[0054] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.
[0055] Figure 1 This is a hardware configuration framework for an electronic device implementing a video encoding method according to an embodiment of this application. Figure 1 .
[0056] Electronic device 1000 can be a terminal or a server. Furthermore, the terminal can be a head-mounted device (such as an AR device, MR device, or VR device), a portable computer, a tablet computer, a PDA, etc. The server can be a cloud server, etc.
[0057] Electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and an image acquisition device 1900, etc. The processor 1100 may be a central processing unit (CPU), a microprocessor (MCU), etc. The memory 1200 may include, for example, ROM (Read-Only Memory), RAM (Random Access Memory), or non-volatile memory such as a hard disk. The interface device 1300 may include, for example, a USB interface, a headphone jack, etc. The communication device 1400 may be capable of wired or wireless communication. The display device 1500 may be, for example, an LCD screen, a touch screen, etc. The input device 1600 may include, for example, a touch screen, a keyboard, etc. Users can input / output voice information through the speaker 1700 and the microphone 1800. The image acquisition device 1900 can acquire environmental images; the image acquisition device 1900 is exemplarily a camera.
[0058] Despite Figure 1 The electronic device 1000 shows multiple devices, but this application may only involve some of them. For example, electronic device 1000 may only involve memory 1200 and processor 1100.
[0059] In the embodiments applied in this application, the memory 1200 of the electronic device 1000 is used to store instructions for controlling the processor 1100 to execute the video encoding method provided in the embodiments of this application.
[0060] In the above description, those skilled in the art can design instructions based on the scheme disclosed in this application. How the instructions control the processor to operate is well known in the art, and therefore will not be described in detail here.
[0061] This application provides a video encoding method, applicable to, for example... Figure 1 The electronic devices shown, such as Figure 2 As shown, the process includes steps S210 to S240.
[0062] Step S210: Obtain the first image and the second image from the video.
[0063] The first image is the frame preceding the second image.
[0064] In one embodiment of this application, the image acquisition device in the electronic device continuously acquires images multiple times to form the video in step S210 above. In one example, the second image is the image acquired by the image acquisition device at the current image acquisition moment, and the first image is the image acquired by the image acquisition device at the previous image acquisition moment.
[0065] In another embodiment of this application, the video in step S210 above is pre-stored by an electronic device, for example, downloaded and stored by the electronic device at a historical moment. In one example, the first image is the first frame of the video, and the second image is the second frame of the video.
[0066] The first and second images can be in YUV format. Of course, the first and second images can also be in RGB format.
[0067] Step S220: Determine the video scene type based on the first image and the second image.
[0068] The video scene types include static scenes and dynamic scenes.
[0069] In this embodiment, a static scene refers to a scene where, compared to the first image, the objects in the second image are relatively stationary; that is, the second image is essentially the same as the first image. A static scene, for example, is one where the position of the image acquisition device does not change, and there are no objects moving significantly in the environment captured by the image acquisition device.
[0070] Correspondingly, a dynamic scene refers to an image in which objects are in relative motion compared to the first image, meaning the second image changes compared to the first image. Examples of dynamic scenes include: the image acquisition device acquiring the first image and then acquiring the second image after its position changes; or, the image acquisition device's position remains unchanged, but there are moving objects in the environment it captures; or, the image acquisition device's position changes, but the movement of objects in the environment it captures is asynchronous.
[0071] In one embodiment of this application, the specific implementation of step S220 above can be: generating an input instruction for the AI big data model, such as "Please determine whether the video scene type corresponding to the first image and the second image is a dynamic scene or a static scene?"; inputting the aforementioned input instruction, the first image, and the second image into the AI big data model, and having the AI big data model output the video scene type.
[0072] Step S230: Based on the video scene type, set the parameter values of the encoding parameters of the video encoder to parameter values that match the video scene type.
[0073] In one embodiment of this application, the video encoder may specifically be an H.265 encoder, and the encoding parameters of the video encoder are at least one of frame rate, bit rate, and GOP. GOP is used to describe the image frame interval.
[0074] Furthermore, the specific implementation of step S230 above is as follows: based on the video scene type, determine the parameter values of the encoder's encoding parameters that match the video scene type; set the encoder's parameter values to the previously determined parameter values.
[0075] Taking a video encoder with at least one of frame rate, bit rate, and GOP as an example, when the video scene type is static, the parameter value matching the video scene type is one that can encode video segments that satisfy at least one of low frame rate, low bit rate, and high GOP. This is because in static scenes, the image change rate is relatively low. When encoding video using a video encoder with parameter values that can encode video segments satisfying at least one of low frame rate, low bit rate, and high GOP, the smoothness and clarity of the encoded video will not be significantly affected, i.e., the quality of the encoded video will not be significantly affected, and bandwidth resources used during transmission of the encoded video segments can be saved. Conversely, in dynamic scenes, the parameter value matching the video scene type is one that can encode video segments satisfying at least one of high frame rate, high bit rate, and low GOP. Thus, when encoding video using a video encoder with parameter values that can encode video segments satisfying at least one of high frame rate, high bit rate, and low GOP, the smoothness and clarity of the encoded video can be guaranteed, i.e., the quality of the encoded video can be guaranteed.
[0076] Based on the above, in one embodiment of this application, the encoding parameters include at least one of frame rate, bit rate and GOP, and the above step S230 can be specifically implemented by the following steps S2301 and S2302.
[0077] Step S2301: When the video scene type is a static scene, set the video encoder's frame rate to at least one of the following: first frame rate, first bit rate, and first GOP.
[0078] Step S2302: When the video scene type is dynamic scene, set the video encoder frame rate to at least one of the following: second frame rate, second bit rate, and GOP to second GOP.
[0079] Among them, the first frame rate is less than the second frame rate, the first bitrate is less than the second bitrate, and the first GOP is greater than the second GOP.
[0080] In this embodiment, the first frame rate, the second frame rate, the first bitrate, the second bitrate, the first GOP, and the second GOP can be set based on experience.
[0081] Step S240: Control the video encoder to encode the first image and the preset number of frames following the first image to obtain the encoded video segment to be transmitted.
[0082] In this embodiment, the video segment encoded by the video encoder is referred to as the encoded video segment, which is to be transmitted to another electronic device.
[0083] The first image and the subsequent preset number of frames total the preset number of frames plus one frame. This preset number of frames plus one frame corresponds to the number of frames where the image change rate is less than or equal to the preset change rate. When the image change rate is less than the preset change rate, the video scene type corresponding to the first image and the subsequent preset number of frames is considered unchanged.
[0084] The preset frame number plus 1 frame corresponds to the frame rate of the images in the video, and can be set based on experience, for example, 10 frames.
[0085] In this embodiment, after step S240, the next frame of the first image after a preset number of frames can be used as the first frame, and the next frame after that can be used as the second frame. Steps S210 to S240 can be repeated to complete the full encoding of the video.
[0086] Furthermore, in one embodiment of this application, if the first image and its subsequent preset frame number image are images of a first format, and the video encoder is used to encode the image of a second format, the specific implementation of step S240 is as follows: converting the first image of the first format and its subsequent preset frame number image of the first image into the first image of the second format and its subsequent preset frame number image of the first image; controlling the video encoder to encode the first image of the second format and its subsequent preset frame number image of the first image to obtain the encoded video segment to be transmitted. In one example, the first format is RGB format and the second format is YUV format.
[0087] Through steps S210 to S240 above, the video encoding method provided by this application can adaptively encode images in a video according to the video scene type to obtain an encoded video segment to be transmitted. Specifically, in the case of a static video scene, by using a video encoder that satisfies at least one of low frame rate, low bit rate, and high GOP, bandwidth resources can be saved during transmission of the encoded video segment. In the case of a dynamic video scene, by using a video encoder that satisfies at least one of high frame rate, high bit rate, and low GOP, the smoothness and clarity of the encoded video segment to be transmitted can be guaranteed, that is, the quality of the encoded video can be guaranteed. In other words, the video encoding method provided by this application can provide a foundation for achieving high-quality video transmission by utilizing effective bandwidth resources.
[0088] This application provides a video encoding method, which includes: acquiring a first image and a second image from a video, wherein the first image is the frame preceding the second image; determining a video scene type based on the first image and the second image, wherein the video scene type includes a static scene and a dynamic scene; setting the encoding parameters of the video encoder to values matching the video scene type; and controlling the video encoder to encode the first image and a preset number of subsequent frames to obtain an encoded video segment to be transmitted. This method provides a foundation for achieving high-quality video transmission by utilizing effective bandwidth resources.
[0089] In another embodiment of this application, step S220 can also be implemented by the following steps S2201 and S2202.
[0090] Step S2201: Determine the corner motion quantization value and / or motion region quantization value based on the first image and the second image.
[0091] The corner motion value is the quantized value of the motion vector formed by the movement of a corner point in the second image from its corresponding position in the first image to its corresponding position in the second image. The motion region quantization value is the quantized value of the region size describing the area formed by pixels that produce motion in the second image compared to the first image.
[0092] In one embodiment of this application, in order to reduce the amount of data in the first image and the second image, and to enhance the feature information in the first image and the second image, based on the embodiments shown in steps S2201 and S2202, the video encoding method provided in this application further includes the following steps S2203 and S2205 before the above step S220.
[0093] Step S2203: Perform grayscale processing on the first image and the second image respectively to obtain the first grayscale image corresponding to the first image and the second grayscale image corresponding to the second image.
[0094] Step S2204: Update the first image to the first grayscale image.
[0095] Step S2205: Update the second image to a second grayscale image.
[0096] In this embodiment, when the first image and the second image are YUV format images, the specific implementation of the above step S2203 is as follows: retain the value of the Y channel of each pixel in the first image to form a first grayscale image; similarly, retain the value of the Y channel of each pixel in the second image to form a second grayscale image.
[0097] When both the first and second images are RGB format images, step S2203 is implemented as follows: For the pixel at position a in the first image, the grayscale value of the pixel at position a is determined according to the formula for converting RGB pixels to grayscale pixels, and used as the pixel value of the pixel at position a in the first grayscale image. Based on this, the first grayscale image corresponding to the first image is obtained. Similarly, the second grayscale image corresponding to the second image can be obtained. Specifically, the formula for converting RGB pixels to grayscale pixels can be gray = (R + G + B) / 3, where R is the specific value of the R channel of the pixel, G is the specific value of the G channel of the pixel, B is the specific value of the B channel of the pixel, and gray is the grayscale value of the pixel.
[0098] After obtaining the first grayscale image, the first image is updated to the first grayscale image. Similarly, after obtaining the second grayscale image, the second image is updated to the second grayscale image. It is understood that in this embodiment, after step S2205, the first image in the video encoding method provided by this application is essentially the first grayscale image, and the second image is essentially the second grayscale image.
[0099] In this embodiment, step S2201 includes determining the corner motion quantization value based on the first image and the second image, and / or determining the motion region quantization value based on the first image and the second image. The step of determining the corner motion quantization value based on the first image and the second image is implemented through steps S22010 to S22012 described below.
[0100] Step S22010: Based on the second image, determine the first position of each corner point in the second image.
[0101] In one embodiment of this application, step S22010 specifically involves: extracting each corner point in the second image using the Harris corner detection algorithm; and determining the position of any corner point in the second image, which is denoted as the first position of the corner point in the second image.
[0102] Step S22011: For any corner point, determine the second position of the corner point in the first image from the first image.
[0103] In one embodiment of this application, the specific implementation of step S22011 is as follows: using an optical flow method, such as the Lucas-Kanade optical flow algorithm, to track the position of the corner point in the first image, and denoted as the second position of the corner point in the first image.
[0104] Step S22012: Determine the motion quantization value of the corner point based on the first position of the corner point in the second image and the second position of the corner point in the first image.
[0105] In one embodiment of this application, the specific implementation of step S22012 above can be as follows: for any corner point, calculate the displacement between the first position of the corner point in the second image and the second position in the first image; use the average displacement between the displacements corresponding to all corner points as the corner point motion quantization value; or determine the first score corresponding to the aforementioned determined average displacement according to the first preset mapping relationship, and record the first score as the corner point motion quantization value, wherein the first preset mapping is used to reflect the correspondence between different average displacements and corresponding scores, and is set according to experience.
[0106] It should be noted that in the embodiments shown in steps S22010 to S22012 above, the first image can essentially be the first image obtained in step S210, and the second image can essentially be the second image obtained in step S210. Alternatively, the first image can also be the first grayscale image obtained in step S2204 above, and the second image can essentially be the second grayscale image obtained in step S2205 above.
[0107] Furthermore, the step of determining the motion region quantization value based on the first image and the second image is implemented by the following steps S22013 to S22015.
[0108] Step S22013: Determine the difference image based on the first image and the second image.
[0109] In this differential image, the pixel value of any pixel is the absolute value of the difference between the pixel values of the corresponding pixel in the first image and the second image.
[0110] In this embodiment, the first image can essentially be the first image obtained based on step S210, and the second image can essentially be the second image obtained based on step S210. Of course, the first image can also essentially be the first grayscale image obtained based on step S2204, and the second image can essentially be the second grayscale image obtained based on step S2205.
[0111] Furthermore, for a pixel at position b in the first image, the pixel value at position b in the first image is subtracted from the pixel value at position b in the second image to obtain the difference value. The absolute value of the difference is then used as the pixel value at position b in the difference image. Based on this, the difference image can be obtained.
[0112] Step S22014: Perform binarization processing on the difference image according to the preset threshold to obtain the binarized image.
[0113] The preset threshold is the minimum change in pixel value caused by the movement of an object in the image. Based on experience or experimentation, this application does not limit the specific value of the preset threshold. It is understood that the movement of an object in the image includes: the position of the image acquisition device does not change, but objects in the environment captured by the image acquisition device move; or the position of the image acquisition device changes, but there are no moving objects in the environment captured by the image acquisition device; or the change in the position of the image acquisition device is asynchronous with the movement of objects in the environment captured by the image acquisition device, resulting in relative movement between the objects in the environment and the image acquisition device.
[0114] In this embodiment, the specific implementation of step S22014 is as follows: for a pixel at position c in the difference image, if the pixel value of the pixel is greater than or equal to a preset threshold, the pixel value of the pixel at position c in the binarized image is determined to be 255; otherwise, the pixel value of the pixel at position c in the binarized image is determined to be 0, and a binarized image is obtained accordingly.
[0115] Step S22015: Determine the quantization value of the motion region based on the binarized image.
[0116] In one embodiment of this application, the number of pixels with a pixel value of 255 in the binarized image is used as the motion region quantization value. Alternatively, according to a second preset mapping relationship, a second score corresponding to the previously determined number of pixels with a pixel value of 255 is determined, and the second score is recorded as the motion region quantization value. The second preset mapping is used to reflect the correspondence between the number of pixels with different pixel values of 255 and the corresponding score, and is set based on experience.
[0117] Step S2202: Determine the video scene type based on the corner motion quantization value and / or motion region quantization value.
[0118] In one embodiment of this application, the above step S2202 is specifically implemented by the following steps S22020, or step S22021, or steps S22022 to S22024.
[0119] Step S22020: If the corner motion quantization value is greater than or equal to the preset corner motion change, the video scene type is determined to be a dynamic scene.
[0120] The preset corner point motion change amount is the minimum change amount of the corner points in the image caused by the movement of objects in the image.
[0121] Corresponding to step S22020 above, if the corner motion quantization value is less than the preset corner motion change amount, the video scene type is determined to be a static scene.
[0122] Step S22021: If the motion region quantization value is greater than or equal to the preset motion region quantization value, determine that the video scene type is a dynamic scene.
[0123] The preset motion region quantization value is the minimum change in the motion region quantization value caused by the motion of an object in the image.
[0124] Corresponding to step S22021 above, if the motion region quantization value is less than the preset motion region quantization value, the video scene type is determined to be a static scene.
[0125] Step S22022: Normalize the corner motion quantization value and the motion region quantization value respectively to obtain normalized corner motion quantization value and normalized motion region quantization value.
[0126] In this embodiment, a traditional normalization algorithm can be used to normalize the corner motion quantization value and the motion region quantization value respectively, so as to obtain the normalized corner motion quantization value and the normalized motion region quantization value.
[0127] Step S22023: Determine the motion quantization value based on the normalized corner motion quantization value and the normalized motion region quantization value.
[0128] In this embodiment, step S22021 is implemented by weighted summation. Specifically, the product of the first weight and the normalized corner motion quantization value is calculated and recorded as the first product; the product of the second weight and the normalized motion region quantization value is calculated and recorded as the second product; the sum of the first product and the second product is used as the determined motion quantization value; wherein, the sum of the first weight and the second weight is 1, and the values of the first weight and the second weight are set according to experience.
[0129] Step S22024: If the motion quantization value is greater than or equal to the preset motion quantization value, determine that the video scene type is a dynamic scene.
[0130] The preset motion quantization value is the minimum change in motion quantization value caused by the movement of an object in the image.
[0131] Corresponding to step S22024, if the motion quantization value is less than the preset motion quantization value, the video scene type is determined to be a static scene.
[0132] This application also provides a video encoding apparatus 300, such as... Figure 3 As shown, it includes:
[0133] The acquisition module 310 is used to acquire a first image and a second image in the video, wherein the first image is the frame preceding the second image;
[0134] The determining module 320 is used to determine the video scene type based on the first image and the second image, wherein the video scene type includes: static scene and dynamic scene;
[0135] Setting module 330 is used to set the parameter values of the encoding parameters of the video encoder to parameter values that match the video scene type according to the video scene type;
[0136] The control module 340 is used to control the video encoder to encode the first image and the image with a preset number of subsequent frames of the first image to obtain the encoded video segment to be transmitted.
[0137] In one embodiment of this application, the determining module 320 is specifically used for:
[0138] Based on the first image and the second image, determine the corner motion quantization value and / or the motion region quantization value;
[0139] The video scene type is determined based on the corner motion quantization value and / or motion region quantization value;
[0140] Wherein, the corner motion quantization value is the quantization value of the motion vector formed by the corner point in the second image moving from the corresponding position in the first image to the corresponding position in the second image, and the motion region quantization value is the quantization value of the region size of the pixel-formed area describing the motion of the second image compared to the first image.
[0141] In one embodiment of this application, the video encoding apparatus 300 provided in this application further includes:
[0142] The update module is used to perform grayscale processing on the first image and the second image respectively to obtain a first grayscale image corresponding to the first image and a second grayscale image corresponding to the second image;
[0143] Update the first image to the first grayscale image;
[0144] Update the second image to the second grayscale image.
[0145] In one embodiment of this application, the determining module 320 is specifically used to: determine the first position of each corner point in the second image based on the second image;
[0146] For any of the corner points, determine the second position of the corner point in the first image from the first image;
[0147] The motion quantization value of the corner point is determined based on the first position of the corner point in the second image and the second position of the corner point in the first image.
[0148] In one embodiment of this application, the determining module 320 is specifically used to: determine a difference image based on the first image and the second image, wherein the pixel value of any pixel in the difference image is the absolute value of the difference between the pixel values of the corresponding pixels in the first image and the second image;
[0149] The difference image is binarized according to a preset threshold to obtain a binarized image. The preset threshold is the minimum change in pixel value caused by the movement of an object in the image.
[0150] Based on the binarized image, determine the quantization value of the motion region.
[0151] In one embodiment of this application, the determining module 320 is specifically used to: determine the video scene type as a dynamic scene when the corner motion quantization value is greater than or equal to a preset corner motion change amount;
[0152] Alternatively, if the motion region quantization value is greater than or equal to the preset motion region quantization value, the video scene type is determined to be a dynamic scene;
[0153] Alternatively, normalization processing can be performed on the corner motion quantization value and the motion region quantization value respectively to obtain normalized corner motion quantization value and normalized motion region quantization value;
[0154] The motion quantization value is determined based on the normalized corner motion quantization value and the normalized motion region quantization value;
[0155] If the motion quantization value is greater than or equal to the preset motion quantization value, the video scene type is determined to be a dynamic scene.
[0156] In one embodiment of this application, the encoding parameters include at least one of frame rate, bit rate, and GOP. The setting module 330 is specifically used to: when the video scene type is a static scene, set the frame rate of the video encoder to a first frame rate, the bit rate to a first bit rate, and the GOP to a first GOP.
[0157] When the video scene type is a dynamic scene, the frame rate of the video encoder is set to a second frame rate, the bit rate is set to a second bit rate, and the GOP is set to a second GOP at least one of the following:
[0158] Wherein, the first frame rate is less than the second frame rate, the first bitrate is less than the second bitrate, and the first GOP is greater than the second GOP.
[0159] This application also provides an electronic device, which includes a video encoding device 300 as provided in any of the above device embodiments;
[0160] Or, such as Figure 4 As shown, the electronic device 400 includes a memory 410 and a processor 420. The memory 410 is used to store computer instructions, and the processor 420 is used to retrieve the computer instructions from the memory 410 to execute any of the video encoding methods provided in the above method embodiments.
[0161] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the video encoding methods provided in the above-described method embodiments.
[0162] This application may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.
[0163] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.
[0164] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.
[0165] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.
[0166] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0167] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.
[0168] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0169] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be well known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.
[0170] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.
Claims
1. A video encoding method, characterized in that, include: Acquire a first image and a second image from the video, wherein the first image is the frame preceding the second image; Based on the first image and the second image, the video scene type is determined, and the video scene type includes: static scene and dynamic scene; Based on the video scene type, the parameter values of the encoding parameters of the video encoder are set to parameter values that match the video scene type; The video encoder is controlled to encode the first image and the image of a preset number of subsequent frames to obtain the encoded video segment to be transmitted.
2. The method according to claim 1, characterized in that, The step of determining the video scene type based on the first image and the second image includes: Based on the first image and the second image, determine the corner motion quantization value and / or the motion region quantization value; The video scene type is determined based on the corner motion quantization value and / or motion region quantization value; Wherein, the corner motion quantization value is the quantization value of the motion vector formed by the corner point in the second image moving from the corresponding position in the first image to the corresponding position in the second image, and the motion region quantization value is the quantization value of the region size of the pixel-formed area describing the motion of the second image compared to the first image.
3. The method according to claim 2, characterized in that, Before determining the corner motion quantization value and / or motion region quantization value based on the first image and the second image, the method further includes: The first image and the second image are respectively subjected to grayscale processing to obtain a first grayscale image corresponding to the first image and a second grayscale image corresponding to the second image; Update the first image to the first grayscale image; Update the second image to the second grayscale image.
4. The method according to claim 2 or 3, characterized in that, The step of determining the corner motion quantization value and / or motion region quantization value based on the first image and the second image includes: Based on the second image, determine the first position of each corner point in the second image; For any of the corner points, determine the second position of the corner point in the first image from the first image; The motion quantization value of the corner point is determined based on the first position of the corner point in the second image and the second position of the corner point in the first image.
5. The method according to claim 2 or 3, characterized in that, The step of determining the corner motion quantization value and / or motion region quantization value based on the first image and the second image includes: Based on the first image and the second image, a difference image is determined, wherein the pixel value of any pixel in the difference image is the absolute value of the difference between the pixel values of the corresponding pixels in the first image and the second image; The difference image is binarized according to a preset threshold to obtain a binarized image. The preset threshold is the minimum change in pixel value caused by the movement of an object in the image. Based on the binarized image, determine the quantization value of the motion region.
6. The method according to claim 2, characterized in that, The step of determining the video scene type based on the corner motion quantization value and / or motion region quantization value includes: If the corner motion quantization value is greater than or equal to the preset corner motion change amount, the video scene type is determined to be a dynamic scene; Alternatively, if the motion region quantization value is greater than or equal to the preset motion region quantization value, the video scene type is determined to be a dynamic scene; Alternatively, normalization processing can be performed on the corner motion quantization value and the motion region quantization value respectively to obtain normalized corner motion quantization value and normalized motion region quantization value; The motion quantization value is determined based on the normalized corner motion quantization value and the normalized motion region quantization value; If the motion quantization value is greater than or equal to the preset motion quantization value, the video scene type is determined to be a dynamic scene.
7. The method according to claim 1, characterized in that, The encoding parameters include at least one of frame rate, bit rate, and GOP. The step of setting the parameter values of the video encoder to match the video scene type includes: When the video scene type is a static scene, the frame rate of the video encoder is set to a first frame rate, the bit rate to a first bit rate, and the GOP to a first GOP; When the video scene type is a dynamic scene, the frame rate of the video encoder is set to a second frame rate, the bit rate is set to a second bit rate, and the GOP is set to a second GOP at least one of the following: Wherein, the first frame rate is less than the second frame rate, the first bitrate is less than the second bitrate, and the first GOP is greater than the second GOP.
8. A video encoding device, characterized in that, include: The acquisition module is used to acquire a first image and a second image from the video, wherein the first image is the frame preceding the second image; The determining module is used to determine the video scene type based on the first image and the second image, wherein the video scene type includes: static scene and dynamic scene; The setting module is used to set the parameter values of the encoding parameters of the video encoder to parameter values that match the video scene type, according to the video scene type. The control module is used to control the video encoder to encode the first image and the image with a preset number of subsequent frames to obtain the encoded video segment to be transmitted.
9. An electronic device, characterized in that, The electronic device includes the apparatus as described in claim 8; Alternatively, the electronic device includes a memory and a processor, the memory for storing computer instructions, and the processor for retrieving the computer instructions from the memory to perform the method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.