Video encoding method, device, equipment, readable storage medium and product

Abstract

The embodiment of the present application discloses a video encoding method, device, equipment, readable storage medium and product, wherein the method includes: obtaining an image group contained in the video data to be processed, and a target scene to which the image group belongs; the image group contains multiple video frames; based on the target scene to which the image group belongs, determining the video encoding parameters for the image group; the video encoding parameters of the image group are used to perform video encoding processing on the multiple video frames contained in the image group; respectively obtaining the types of the multiple video frames, and based on the video encoding parameters of the image group and the types of the multiple video frames, respectively performing video encoding processing on the multiple video frames contained in the image group, and obtaining the encoded data of the multiple video frames contained in the image group. By adopting the embodiment of the present application, video data can be encoded in combination with specific scenes to enrich the video encoding method.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video encoding method, device, equipment, readable storage medium and product. Background Art

[0002] In the video live broadcast scenario, it is necessary to collect the live video data of the anchor, encode the collected video data and send it to the client for video display. However, the current video encoding method generally uses fixed video encoding parameters to encode the video data. This video encoding method is relatively simple, so that the encoded video data cannot meet the business needs in different live broadcast scenarios. Therefore, how to encode the video data in combination with specific scenarios and enrich the video encoding methods is an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present application provide a video encoding method, apparatus, device, readable storage medium and product, which can encode video data in combination with specific scenarios to enrich video encoding methods.

[0004] In a first aspect, the present application provides a video encoding method, comprising:

[0005] Acquire an image group included in the video data to be processed and a target scene to which the image group belongs; the image group includes multiple video frames;

[0006] Based on the target scene to which the image group belongs, determining a video encoding parameter for the image group; the video encoding parameter for the image group is used to perform video encoding processing on a plurality of video frames included in the image group;

[0007] The types of the multiple video frames are respectively obtained, and based on the video encoding parameters of the image group and the types of the multiple video frames, video encoding processing is performed on the multiple video frames included in the image group to obtain encoded data of the multiple video frames included in the image group.

[0008] In a second aspect, the present application provides a video encoding device, including:

[0009] A data acquisition unit, used to acquire an image group included in the video data to be processed and a target scene to which the image group belongs; the image group includes a plurality of video frames;

[0010] A parameter determination unit, used to determine a video encoding parameter for the image group based on a target scene to which the image group belongs; the video encoding parameter of the image group is used to perform video encoding processing on a plurality of video frames included in the image group;

[0011] The video encoding unit is used to respectively obtain the types of the multiple video frames, and based on the video encoding parameters of the image group and the types of the multiple video frames, respectively perform video encoding processing on the multiple video frames included in the image group to obtain the encoded data of the multiple video frames included in the image group.

[0012] In a third aspect, the present application provides a computer device, including: a processor, a memory, and a network interface;

[0013] The processor is connected to a memory and a network interface, wherein the network interface is used to provide a data communication function, the memory is used to store a computer program code, and the processor is used to call the computer program code so that a computer device including the processor executes the video encoding method.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the above-mentioned video encoding method.

[0015] In a fifth aspect, the present application provides a computer program product or a computer program, which includes computer instructions. When the computer instructions are executed by a processor, the video encoding method provided in various optional methods in the first aspect of the present application is implemented.

[0016] In an embodiment of the present application, an image group included in the video data to be processed and a target scene to which the image group belongs are obtained; the image group includes multiple video frames; based on the target scene to which the image group belongs, video encoding parameters for the image group are determined; the video encoding parameters of the image group are used to perform video encoding processing on the multiple video frames included in the image group; the types of the multiple video frames are respectively obtained, and based on the video encoding parameters of the image group and the types of the multiple video frames, video encoding processing is performed on the multiple video frames included in the image group respectively to obtain the encoded data of the multiple video frames included in the image group. Since the corresponding video encoding parameters are selected in combination with the scene to which the image group belongs when performing video encoding processing on the video frames in the image group, it is possible to implement video encoding of the video data in combination with a specific scene, thereby enriching the video encoding method. Moreover, since the corresponding video encoding method is selected in combination with the type of each video frame in the image group for video encoding processing, the accuracy of video encoding can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a network architecture diagram of a video encoding system provided in an embodiment of the present application;

[0019] Figure 2 It is a flowchart of a video encoding method provided by an embodiment of the present application;

[0020] Figure 3 is a scene schematic diagram of a video processing method provided in an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a video frame provided in an embodiment of the present application;

[0022] Figure 5 It is a flowchart of another video encoding method provided by an embodiment of the present application;

[0023] Figure 6 It is a schematic diagram of the composition structure of a video encoding device provided in an embodiment of the present application;

[0024] Figure 7 It is a schematic diagram of the composition structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The technical solution of the present application can be applied to the scene of video encoding of video data in the live broadcast scene. For example, it can be applied to a video interaction platform, by determining the corresponding video encoding parameters in combination with the scene to which the video data belongs, and performing video encoding in combination with the video encoding parameters corresponding to the scene, it is possible to achieve targeted video encoding and enrich the video encoding method. The technical solution of the embodiment of the present application can also be applied to any scene that requires video encoding, for example, it can include but is not limited to a video interaction platform, a social platform, a content interaction platform, an audio interaction platform, etc., and the embodiment of the present application does not limit this.

[0027] It is understandable that the collection, use and processing of relevant data involved in the embodiments of the present application (such as video data to be processed, video encoding parameters, target scenes, etc.) require the consent of the corresponding personal information subject and need to comply with the provisions of relevant laws and regulations.

[0028] Please refer to Figure 1 , Figure 1 is a network architecture diagram of a video encoding system provided in an embodiment of the present application, such as Figure 1 As shown, the computer device can exchange data with the terminal device, and the number of the terminal devices can be one or at least two. For example, when the number of the terminal devices is multiple, the terminal device may include Figure 1 The terminal device 101a, the terminal device 101b and the terminal device 101c, etc. Among them, taking the terminal device 101a as an example, the computer device 102 can obtain the image group contained in the video data to be processed, and the target scene to which the video data to be processed belongs. Further, the computer device 102 can determine the video encoding parameters for the image group based on the target scene to which the image group belongs. In addition, the computer device 102 can obtain the types of multiple video frames respectively, and based on the video encoding parameters of the image group and the types of the multiple video frames, perform video encoding processing on the multiple video frames contained in the image group respectively to obtain the encoded data of the multiple video frames contained in the image group. Optionally, the computer device 102 can also send the encoded data of the multiple video frames contained in the image group to the terminal device 101a, so that the terminal device 101a performs video decoding processing on the encoded data and displays the corresponding video data.

[0029] The computer device mentioned in the embodiments of the present application may refer to a server or a terminal device, or may also be a system composed of a server and a terminal device, which is not limited in the embodiments of the present application. The terminal device may be an electronic device, including but not limited to a mobile phone, a tablet computer, a laptop computer, a vehicle-mounted device, etc. The server mentioned above may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, and big data and artificial intelligence platforms.

[0030] For further information, see Figure 2 , Figure 2 is a flow chart of a video encoding method provided by an embodiment of the present application; Figure 2 As shown, the video encoding method can be applied to a computer device, and the video encoding method includes but is not limited to the following steps:

[0031] S101, obtaining an image group included in video data to be processed, and a target scene to which the image group belongs.

[0032] See also Figure 3 , Figure 3 is a scene diagram of a video processing method provided in an embodiment of the present application, such as Figure 3 As shown, after the host end starts the live broadcast, the video acquisition device at the host end can collect the video data of the host's live broadcast, and obtain the pre-processed video data by pre-processing the video data, for example, the pre-processing can include video optimization processing such as beauty and video special effects. Further, the pre-processed video data can be subjected to video encoding processing to obtain encoded data, and the encoded data can be packaged and transmitted, that is, the encoded data is packaged into data packets, and the data packets are transmitted to the packet receiving and framing through the network, so as to realize the transmission of the encoded data. Further, by post-processing the data packets, for example, checking the data packets to determine whether the format of the data packets is correct and whether frames are lost, and in the case of determining that the format of the data packets is correct and no frames are lost, the video decoder at the audience end performs video decoding processing on the data packets to obtain decoded data, and the decoded data is the pre-processed video data. Further, the audience end can render the decoded data, and display the rendered decoded data on the audience end, so that the live video data after the video optimization processing such as beauty and video special effects on the video data of the host end can be viewed through the audience end.

[0033] In the embodiment of the present application, the video data to be processed may refer to video data in a video live broadcast scene, for example, it may include but is not limited to live broadcast data of a talent show of a live broadcast host, live broadcast data of a game, live broadcast data of a competition with other hosts, etc. After the video acquisition device at the host end acquires the video data in the live broadcast scene at the host end, the acquired video data may be uploaded to obtain the video data to be processed. Furthermore, the video data to be processed may be subjected to video division processing, and the video data to be processed may be divided into image groups.

[0034] A picture group may include multiple video frames, and the multiple video frames included in a picture group are continuous. A picture group may refer to a basic video unit, that is, a group of pictures (Group Of Picture, GOP), and a picture group is a group of continuous image pictures. In the process of video encoding of the video data to be processed, by dividing the video data to be processed into picture groups, subsequent video encoding processing is facilitated.

[0035] Further, the target scene may refer to a live broadcast scene, and the target scene may include multiple live broadcast scenes, for example, the live broadcast scene may include a first scene, a second scene, and a third scene. Among them, the first scene may refer to a scene where a live broadcast with microphones is required, the second scene may refer to a scene where a live broadcast with microphones is not required and the number of live broadcast viewers is less than or equal to a preset number, and the third scene may refer to a scene where a live broadcast with microphones is not required and the number of live broadcast viewers is greater than a preset number. For example, the first scene may refer to a live broadcast with microphones scene, and the live broadcast with microphones scene may refer to a scene where a host end and an audience end conduct a live broadcast with microphones, or may refer to a scene where a host end and another host end conduct a live broadcast with microphones, and so on. The second scene may refer to an ordinary live broadcast scene, for example, an ordinary live broadcast scene may refer to a scene where the number of live broadcast viewers is less than or equal to a preset number, such as a game live broadcast, a singing live broadcast, a dancing live broadcast, a live broadcast with goods, etc. The third scene may refer to a popular live broadcast scene, for example, a popular live broadcast scene may refer to a scene where a popular host conducts a live broadcast and the number of live broadcast viewers is greater than a preset number, for example, a popular live broadcast scene may refer to a scene where a celebrity live broadcast, an authoritative expert live broadcast, etc.

[0036] In the live broadcast scenario, for the video data in the live broadcast scenario, the three indicators of latency, clarity, and fluency are more important. Latency can be used to indicate the data transmission speed of the video transmitter transmitting the video data to the video receiver in the live broadcast. The smaller the latency, the faster the data transmission speed, and the faster the client can view the video data in the live broadcast scenario. Clarity can refer to the video quality in the video data. The higher the clarity, the higher the video quality in the video data. The lower the clarity, the lower the video quality in the video data. Fluency can refer to the frame rate. The lower the frame rate, the worse the video continuity of the video data. The higher the frame rate, the higher the video continuity of the video data.

[0037] However, generally speaking, these three indicators need to be weighed for video data in any live broadcast scenario. For example, the smaller the delay, the lower the clarity and the lower the fluency, or the higher the clarity, the greater the delay and the lower the fluency, or the higher the fluency, the greater the delay and the lower the clarity. The requirements for these three indicators in different live broadcast scenarios are different. Therefore, different video encoding methods can be used to encode the video data in various live broadcast scenarios to improve the corresponding indicators in various live broadcast scenarios. For example, in the first scenario, if a live broadcast with a microphone is required, the delay needs to be reduced to achieve a better live broadcast with a microphone effect. That is, the lower the delay in the live broadcast scenario with a microphone, the smoother the microphone connection process and the better the live broadcast effect. In the second scenario, the video fluency needs to be improved to achieve a better live broadcast effect. In the third scenario, the video clarity needs to be improved to achieve a better live broadcast effect.

[0038] Since the current solution uses a unified encoding method when encoding video data, the video data in all scenarios has the same picture quality displayed on the client. Since the live broadcast scene can be subdivided into multiple live broadcast scenes such as live broadcast with microphone, ordinary live broadcast and popular live broadcast, the requirements for video data in each live broadcast scene are different. For example, for live broadcast with microphone, the latency requirement is high, and the live video data needs to reduce the latency as much as possible to avoid reducing the user experience. For popular live broadcasts, such as popular anchors selling goods, there are many viewers, and there are no microphone requests, so the video quality requirements are high and the picture quality requirements are relatively fine. For normal live broadcasts, there are neither microphone requests nor many viewers, such as game live broadcasts or sports live broadcasts, and the requirements for smoothness are high. Therefore, for each live broadcast scene, targeted video encoding processing is required to improve the user's subjective experience.

[0039] Since the indicator requirements for each scenario are different, and the video parameters in the video data to be processed recorded by the anchor end are fixed, different video encoding parameters can be selected for video encoding when the video data to be processed is encoded. This can make the same video data after video encoding processing, when the decoded video data is subsequently displayed on the client such as the anchor end or the audience end, when the video data in different live broadcast scenarios is displayed on the client, the performance of the three indicators of smoothness, delay and clarity will be different.

[0040] In the embodiment of the present application, for example, if the target scene to which the image group belongs is the first scene, the delay of the video data viewed by the audience can be reduced by performing video encoding processing on the image group included in the video data to be processed in combination with the scene. For example, if the target scene to which the image group belongs is the second scene, the smoothness of the video data viewed by the audience can be improved by performing video encoding processing on the image group included in the video data to be processed in combination with the scene. For example, if the target scene to which the image group belongs is the third scene, the clarity of the video data viewed by the audience can be improved by performing video encoding processing on the image group included in the video data to be processed in combination with the scene, and the video quality can be improved, so that targeted video encoding processing can be performed in combination with the scene corresponding to the video data to be processed, and the video encoding method can be enriched.

[0041] In an embodiment of the present application, if the video data to be processed contains one image group, the target scene to which the image group belongs is the target scene to which the video data to be processed belongs. If the video data to be processed contains multiple image groups, the target scene to which the video data to be processed belongs includes the target scenes to which multiple image groups belong. For example, if the video duration of the acquired video data to be processed is relatively long, then the scene to which the first period of time belongs in the video data to be processed is the first scene, and the scene to which the second period of time belongs is the second scene. The target scenes to which the multiple image groups contained in the video data to be processed belong may be different. Then, the target scene to which each image group belongs can be obtained for each image group, thereby performing targeted video encoding on the target scene to which each image group belongs to, and obtaining corresponding encoded data.

[0042] In the embodiment of the present application, when obtaining the target scene to which the image group belongs, the target scene to which the image group belongs can be detected by the service end; or, image recognition can be performed on multiple video frames contained in the image group by image recognition to determine the target scene to which the image group belongs; or, a specified scene can be obtained as the target scene to which the image group belongs, for example, the specified scene can refer to a scene specified by the client. The specific method of determining the target scene to which the image group belongs can be found in the specific description in step S201 below, and no further description is given here.

[0043] S102: Determine video encoding parameters for the image group based on the target scene to which the image group belongs.

[0044] In the embodiment of the present application, the principle of video encoding processing is to control the video encoding parameters in the video encoder, and the video encoder performs video encoding based on the video encoding parameters. Different video encoding parameter settings will result in different video playback effects on the viewer end after the subsequent video decoding process of the encoded video data. Since the current encoding method generally sets fixed video encoding parameters, the video data in each scenario encoded based on the fixed video encoding parameters will have the same video effect displayed on the viewer end, and targeted video display cannot be achieved.

[0045] Therefore, in the embodiment of the present application, according to the different target scenes to which the image group in the video data to be processed belongs, the video encoding parameters of the corresponding scene can be selected, so as to control the video encoder to perform video encoding processing using the video encoding parameters of the corresponding scene, so that the encoded video data can meet the business requirements in the corresponding scene. Among them, the video encoding parameters of the image group are used to perform video encoding processing on multiple video frames contained in the image group.

[0046] In one embodiment, the video encoding parameters of the image group include at least one of the following: the number of reference frames, fractional pixel search accuracy, and the number of cached frames. The number of reference frames refers to the number of reference video frames required for encoding each video frame contained in the image group. The fractional pixel search accuracy is used to search for reference image blocks that match the target image blocks in each video frame in the reference frame, and the reference image blocks refer to image blocks in the reference frame whose similarity between pixels and the target image blocks is greater than a similarity threshold. The number of cached frames refers to the number of cached video frames, and the number of cached frames is used to allocate the bit rate of each video frame in the image group.

[0047] Among them, the number of reference frames is different, and the accuracy of the motion of the moving object in the encoded data obtained by encoding is different. The more reference frames, the finer the video encoding, but correspondingly, the lower the video encoding efficiency. The fewer reference frames, the higher the video encoding efficiency. The target image block can refer to the image block in the current frame, that is, the current image block. The image block can refer to a macroblock. In video encoding technology, video compression is performed by dividing the image screen in the video frame into macroblocks. Fractional pixel search accuracy means that when searching for the motion vector of the current image block on the reference frame, fractional pixel search is supported. The fractional pixel is obtained by filtering and synthesizing the integer pixel, because the motion in the video is not necessarily integer pixel motion. The finer the supported fractional pixel, the greater the amount of calculation, so the more accurate the motion vector obtained, and the higher the encoding quality. For example, fractional pixel search accuracy refers to searching for the image block in the reference frame that is most similar to the image block pixel in the current video frame. The current search method is generally integer pixel search. However, in reality, the motion of objects varies greatly. Therefore, in the technical solution of this application, by setting the fractional pixel search accuracy, that is, the fractional pixel search accuracy is no longer searched in whole pixel units, but in 1 / 2 or 1 / 4 pixel units, the accuracy of the motion description can be improved. The more cached frames there are, the more video frames can be referenced during video encoding, the more refined the video encoding, and the lower the video encoding efficiency. By caching video frames, the bit rate of the cached video frames can be allocated, so that when the video encoding process is performed later, the video encoding can be combined with the bit rate of each video frame.

[0048] In the embodiment of the present application, by controlling the video encoding parameters in the video encoder, such as the number of reference frames, the fractional pixel search accuracy, and the number of cached frames, video encoding processing can be achieved for video data. The three parameters of the number of reference frames, the fractional pixel search accuracy, and the number of cached frames are set differently, and the video encoding effect is different. The video playback effect of the video data played on the client later is different. For example, the larger the number of reference frames, the more referenced video frames are used when each video frame is subjected to video encoding processing, and the encoded data obtained by encoding has a higher video clarity when it is subsequently displayed. The larger the fractional pixel search accuracy, the more refined the search for image blocks in the reference frame can be searched in a more refined search method when each video frame is subjected to video encoding processing, so that the more refined the description of the motion, the higher the video fluency of the encoded data obtained by encoding when it is subsequently displayed. The larger the number of cached frames, the more cached video frames are used when the video encoding is processed, and before the video encoding is processed, each cached frame can be estimated separately, so that different bit rates are assigned to each video frame, so that the video clarity can be improved.

[0049] In one embodiment, if the target scene to which the image group belongs is the first scene, for example, the first scene is a scene that requires live broadcast with microphones connected, it can be determined that the number of reference frames belongs to the first number interval, the fractional pixel search accuracy belongs to the first search accuracy interval, and the number of cached frames belongs to the first cached number interval. The first number interval may include one or more values, for example, 1 and 2. The first search accuracy interval may include one or more values, for example, 1 and 1.5, and the first cached number interval may include one or more values, for example, 1, 2, 3, 4, 5, and so on.

[0050] For example, in a live broadcast scenario, if only the latency needs to be reduced, the number of reference frames can be determined as a smaller value in the first number interval, such as 1. If the video clarity needs to be appropriately improved on the basis of reducing latency, the number of reference frames can be determined as a larger value in the first number interval, such as 2. By appropriately reducing the requirement for latency, the requirement for clarity can be appropriately improved, thereby achieving a trade-off.

[0051] In one embodiment, if the target scene to which the image group belongs is the second scene, for example, the second scene is an ordinary live broadcast scene, it can be determined that the number of reference frames belongs to the first number interval, the fractional pixel search accuracy belongs to the second search accuracy interval, and the number of cached frames belongs to the second cache quantity interval. Among them, the minimum value of the second search accuracy interval can be greater than the maximum value of the first search accuracy interval, and the minimum value of the second cache quantity interval is greater than the maximum value of the first cache quantity interval. The second search accuracy interval can, for example, include one or more values, for example, 2 and 3, and so on. The second cache quantity interval can include one or more values, for example, 6, 7, 8, 9, 10, and so on.

[0052] For example, in a normal live broadcast scenario, if only the smoothness needs to be improved, the fractional pixel search accuracy can be determined as a larger value in the second search accuracy interval, such as 3. If it is necessary to appropriately improve the video clarity on the basis of improving the smoothness, the fractional pixel search accuracy can be determined as a smaller value in the second search accuracy interval, such as 2. By appropriately reducing the requirements for smoothness, the requirements for clarity can be appropriately improved, thereby achieving a trade-off.

[0053] In one embodiment, if the target scene to which the image group belongs is the third scene, for example, the second scene is a popular live broadcast scene, it is determined that the number of reference frames belongs to the second number interval, the fractional pixel search accuracy belongs to the second search accuracy interval, and the number of cached frames belongs to the third cached number interval. The minimum value of the second number interval is greater than the maximum value of the first number interval, and the minimum value of the third cached number interval is greater than the maximum value of the second cached number interval. The third cached number interval may include one or more values, for example, 11, 12, 13, 14, 15, ..., 20, and so on.

[0054] For example, in the scenario of popular live broadcasts, if only the definition needs to be improved, the number of cached frames can be determined as a larger value in the third number interval, such as 20. If it is necessary to appropriately reduce video latency while improving definition, the number of reference frames can be determined as a smaller value in the second number interval, such as 15. By appropriately reducing the definition requirement, the latency requirement can be appropriately increased, thereby achieving a trade-off.

[0055] For example, if the target scene is a live broadcast with microphones connected, in order to reduce latency, the number of reference frames can be 1, the fractional pixel search accuracy can be 1, and the number of cached frames can be 1. The fractional pixel search accuracy of 1 means that the search is performed in units of 1 / 2 pixels. The smaller the number of reference frames and cached frames, the faster the video encoding speed, thereby improving video encoding efficiency and reducing latency.

[0056] Or, for example, if the target scene is a common live broadcast scene, in order to improve the fluency, the number of reference frames can be 1, the fractional pixel search precision can be 2, and the number of cached frames can be 10. The fractional pixel search precision can be 2, which means searching at every 1 / 4 pixel unit, which can make the description of the motion more detailed. The larger the number of cached frames, the lower the video encoding efficiency. However, the smaller the number of reference frames and the larger the number of cached frames, the better the fluency.

[0057] Or, for example, if the target scene is a popular live broadcast scene, in order to improve the clarity, the number of reference frames can be 3, the fractional pixel search accuracy can be 2, and the number of cached frames can be 20. The larger the number of cached frames, the lower the video encoding efficiency, but the finer the encoding, the larger the number of reference frames, which means that more video frames are referenced to describe the image changes in the video frames, which can improve the clarity.

[0058] In the embodiment of the present application, different video encoding parameters can be determined for image groups in different scenarios, so that targeted video encoding processing can be implemented for image groups in different scenarios, thereby improving the video encoding effect.

[0059] Please refer to Figure 4 , Figure 4 is a schematic diagram of a video frame provided in an embodiment of the present application, Figure 4 The white box in the middle represents the current frame, that is, the video frame currently being encoded. The gray boxes represent the video frames before and after the current frame. The arrows corresponding to the white boxes point to the reference frames of the current frame. If the current frame is a key frame, there is no need to refer to other video frames when encoding the current frame. If the current frame is a bidirectional reference frame, the gray box pointed to by the arrow is the reference frame when encoding the current frame, that is, the previous video frame and the following video frame can be referenced. Figure 4 As shown in 4a, the reference frames of the current frame may include 3. If the current frame is a forward reference frame, when the current frame is encoded, the gray box pointed by the arrow is the reference frame, that is, the previous video frame can be referenced, such as Figure 4 As shown in 4b, the reference frame of the current frame may include 1. Figure 4 As shown in 4c, when the fractional pixel search precision is 1, it means that when searching for the image block that is most similar to the current image block, the search is performed by moving 1 / 2 pixel units each time. Figure 4 As shown in 4d, the number of cached frames may be 8, and the cached frames include the current frame and the continuous video frames before and after the current frame.

[0060] S103, respectively obtaining types of the multiple video frames, and based on the video encoding parameters of the image group and the types of the multiple video frames, respectively performing video encoding processing on the multiple video frames included in the image group to obtain encoded data of the multiple video frames included in the image group.

[0061] In an embodiment of the present application, since the image group contains multiple video frames and each video frame has a different type, the video encoding method of each video frame is different. When performing video encoding processing on each video frame, the encoding is performed in combination with the video encoding parameters and the video encoding method of each video frame. Therefore, by respectively obtaining the types of the multiple video frames and performing video encoding processing on the multiple video frames contained in the image group based on the video encoding parameters of the image group and the types of the multiple video frames, the encoded data of the multiple video frames contained in the image group can be obtained.

[0062] The type of the video frame may indicate whether the video frame needs to be encoded with reference to other video frames in the group of pictures during encoding. The types of video frames may include, for example, key frame types, forward reference frame types, and bidirectional reference frame types, and each type of video frame has a different video encoding method. A video frame of the key frame type may be referred to as a key frame, a video frame of the forward reference frame type may be referred to as a forward reference frame, and a video frame of the bidirectional reference frame type may be referred to as a bidirectional reference frame.

[0063] For example, when a key frame is being encoded for video, the image data in the key frame can be directly encoded. When a forward reference frame is being encoded for video, the encoding can be performed with reference to the previous video frame. When a bidirectional reference frame is being encoded, the encoding can be performed with reference to both the previous video frame and the subsequent video frame. For example, the video encoding method of a key frame can be encoded using intra-frame compression technology. The video encoding method of a forward reference frame can be encoded using inter-frame compression technology. The video encoding method of a bidirectional reference frame can be encoded using inter-frame compression technology. It can be understood that the type of the first video frame in a group of images is generally a key frame type.

[0064] In one embodiment, different types of video frames have different encoding methods. If there is a video frame in an image group that is a key frame type, the encoding method corresponding to the key frame is obtained, and the encoding method corresponding to the key frame and the video encoding parameters of the image group are used to perform video encoding processing on the key frame in the image group to obtain the encoded data of the key frame in the image group.

[0065] Furthermore, if there is a video frame in the image group whose type is a forward reference frame type, the encoding method corresponding to the forward reference frame is obtained, and the encoding method corresponding to the forward reference frame and the video encoding parameters of the image group are used to perform video encoding processing on the forward reference frame in the image group to obtain the encoded data of the forward reference frame in the image group.

[0066] Furthermore, if there is a video frame in the image group whose type is a bidirectional reference frame type, the encoding method corresponding to the bidirectional reference frame is obtained, and the encoding method corresponding to the bidirectional reference frame and the video encoding parameters of the image group are used to perform video encoding processing on the bidirectional reference frame in the image group to obtain the encoded data of the bidirectional reference frame in the image group.

[0067] Generally speaking, multiple video frames included in an image group belong to the same scene, and therefore the video encoding parameters of the multiple video frames in an image group are the same. However, since the types of the multiple video frames in an image group are different, the same video encoding parameters and the video encoding methods corresponding to the various types of video frames can be used to perform video encoding processing on each video frame to obtain corresponding encoded data.

[0068] Optionally, the key frame may refer to an I frame (Intra picture), the forward reference frame may refer to a P frame (Predictive-frame), and the bidirectional reference frame may refer to a B frame (Bi-directional interpolated prediction frame). After the I frame is encoded, there is a complete image screen; after the P frame is encoded, there is no complete video screen data, only the screen difference data with the previous video frame; after the B frame is encoded, there is no complete video screen data, only the difference data with the screen of the previous video frame and the screen of the next video frame. Therefore, when performing video decoding processing, the I frame itself can be decompressed into a separate complete video screen by a video decompression algorithm. The P frame can refer to the previous I frame or P frame to decode into a complete video screen. The B frame can refer to the previous I frame or P frame and the next P frame to generate a complete video screen. After the I frame is encoded, the redundant information of the video frame in the spatial dimension can be removed, and the P frame and the B frame can remove the temporal redundant information between the video frames. In addition, the P frame is referenced forward during encoding, and the residual is small; the B frame is referenced both forward and backward, and the encoding efficiency is higher.

[0069] In one embodiment, video encoding processing can also be performed on each video frame in combination with the bit rate of each video frame in the image group. Specifically, prediction processing can be performed on multiple video frames included in the image group to obtain the complexity of the multiple video frames included in the image group; based on the complexity of the multiple video frames included in the image group, the bit rate of the multiple video frames included in the image group is determined; based on the video encoding parameters of the image group, the types of the multiple video frames included in the image group, and the bit rates of the multiple video frames, video encoding processing is performed on the multiple video frames included in the image group respectively to obtain the encoded data of the multiple video frames included in the image group.

[0070] The complexity of a video frame can be used to indicate the amount of image information contained in the video frame. The higher the complexity of a video frame, the more image information is contained in the video frame. A higher bit rate can be allocated to the video frame for video encoding, which can improve the accuracy of encoding and make the encoding more refined. For video frames with lower complexity, a lower bit rate can be allocated to improve video encoding efficiency.

[0071] In a specific implementation, when a cached frame is obtained, the complexity of each frame in the cached frame can be estimated in advance to determine the complexity of each cached frame. When video encoding is performed on each image group later, encoding can be performed in combination with the complexity of each video frame in the image group. For example, the complexity of each video frame can be determined by calculating the similarity between adjacent video frames in a plurality of video frames. Optionally, for example, a complexity calculation algorithm can be used to perform prediction processing on the plurality of video frames contained in the image group to obtain the complexity of the plurality of video frames contained in the image group. For example, the complexity calculation algorithm can include but is not limited to the Sum of Absolute Transformed Difference (SATD) algorithm.

[0072] In an embodiment of the present application, for example, the complexity of each video frame can be calculated frame by frame, and an initial bit rate can be assigned to all video frames based on the complexity of each video frame in the image group. For a video encoder, after the bit rate is set, the encoder encodes according to the bit rate, but the bit rate output by the encoder is calculated per second, such as how many bits per second. Therefore, the bit rate assigned to each video frame to characterize each video frame is different. For example, 2000 megabits of bit rate is assigned as a whole for 1 second, and the bit rates assigned to multiple video frames in this 1 second are different. The sum of the bit rates assigned to all video frames in 1 second is equal to 2000 megabits. Optionally, an initial bit rate and a quantization parameter can also be assigned to all video frames based on the complexity of each video frame in the image group. The quantization parameter is a parameter that describes quality frame loss at a finer level of the frame or block level in the video encoder.

[0073] For example, during the encoding process, multiple video frames are first divided into image groups, and an image group is considered to be a basic unit of video. For example, if the image group is set to 32 frames, the type of each frame of video needs to be determined from frames 0-31. The first frame is an I frame, which is the most important video frame and the video frame with the best quality. Many subsequent video frames will select I frames for encoding. For example, when encoding a video, the more important video frames can be encoded first. For example, the 0th frame can be encoded first, and then the 31st frame. The 0th frame and the 31st frame are both video frames with higher importance in the image group, so they can be encoded first. Then encode the 16th frame. When encoding the 16th frame, the first frame can be used as a reference frame for encoding. If the scene switches, the current frame can be encoded as a new I frame.

[0074] In a possible implementation, the frame type of the video frame in the GOP can also be changed according to the GOP (group of pictures) length (i.e., the number of video frames in the GOP) and the external forced I frame request, that is, determine which frame type in each GOP is I, B, or P frame, and set the number of B frames of the frame type to be changed in each GOP. Insert B frames in the GOP according to the number of B frames, that is, change which P frames in the GOP to B frames, and inserting B frames means changing P frames to B frames. For example, if a GOP is set to insert 3 B frames, then select the positions in the GOP where the 3 frames are inserted, that is, change which P frames in the GOP. Change the frame type according to scene switching detection, that is, when a scene switching is detected in the live scene, such as the front camera is switched to the rear camera, or the new product causes the entire live screen to change, the current frame is changed to an I frame, so that it can be used as a reference frame during video encoding. By changing P frames or B frames to I frames in the GOP, or inserting B frames in the GOP, the video quality can be made clearer and the encoding efficiency can be higher. The total number of video frames in the GOP after the frame type is changed remains unchanged. After the frame type is changed, the video encoding method corresponding to the changed frame is also changed during subsequent video encoding, so the encoded video effect is better.

[0075] In one embodiment, after video encoding is performed in combination with the scene to obtain encoded data, the encoded data of multiple video frames included in the image group can be transmitted to the client, so that the client performs video decoding on the encoded data of multiple video frames included in the image group to obtain decoded data, and renders the decoded data, and outputs the decoded data after rendering. The client may include a viewer end or a host end.

[0076] In an embodiment of the present application, video encoding is performed on multiple video frames in combination with a specific live broadcast scenario. When the encoded data of the multiple video frames are transmitted to a client, the client can use a video decoding method corresponding to the video encoding method to decode the video encoded data to obtain decoded data, and then the decoded data can be rendered and the rendered decoded data is displayed on the client, so that the live broadcast host and audience of the client can view the live video data, thereby achieving the effect of live video broadcast.

[0077] In an optional implementation, if the number of image groups included in the video data to be processed is multiple, any two consecutive image groups among the multiple image groups are the first image group and the second image group, and the first image group and the second image group belong to different target scenes; then based on the target scenes to which the image groups belong, a method for determining the video encoding parameters for the image groups may be as follows:

[0078] Based on the target scene to which the first image group belongs, video encoding parameters for the first image group are determined; based on the target scene to which the second image group belongs, video encoding parameters for the second image group are determined; the video encoding parameters of the second image group are different from the video encoding parameters of the first image group.

[0079] For example, when the collected video data to be processed happens to include video data of two live broadcast scenes, corresponding video encoding parameters can be selected for the video data of each live broadcast scene respectively, and corresponding video encoding processing can be performed, instead of using the same video processing method and video encoding parameters for video encoding processing, which can improve the corresponding video processing effect.

[0080] The technical solution in the embodiment of the present application is applied to the live broadcast service, and the stability, clarity, fluency and low latency of the live broadcast service must be ensured, such as the stability and normality of technical indicators and client indicators. Technical indicators may include, for example, the freeze rate, and client indicators may include, for example, the subjective experience of users, retention rate, and so on. By introducing a self-developed soft coding method, that is, setting different video coding parameters for encoding in combination with the scene, the picture quality of the live broadcast can be improved. In addition, the soft coding method can support the modification of coding parameters, save bit rate, and subdivide specific live broadcast scenes at a fixed bit rate, which can have better coding performance and coding effect. Using different video coding parameters for different business scenarios for video coding processing will obtain different coding qualities for multiple video frames in the image group under the premise that the total coding bit rate of the entire image group remains unchanged, thereby improving the experience of the audience. Selecting different video coding parameters for different live broadcast scenes can improve the picture quality, fluency or delay of the entire live broadcast, thereby improving the subjective experience of the audience. Distinguish specific live broadcast scenes in video live broadcast, and make adaptive adjustments to video coding parameters in different scenes, so that the video coding effect is efficient and meets expectations. Selecting different video encoding parameters for different live broadcast scenarios can improve the picture quality or smoothness of the entire live broadcast or reduce latency, thereby improving the subjective experience of the audience.

[0081] In the embodiments of the present application, by selecting different parameters for different live broadcast scenarios through self-developed software editing, it is possible to achieve targeted improvement of the live broadcast effect. For example, for the live broadcast scenario with microphone connection, by selecting the corresponding video encoding parameters for video encoding, the delay of the live video can be reduced and the video viewing experience can be improved. For popular live broadcasts, such as popular anchors selling goods, there are many viewers, and there are no microphone connection requests. By selecting the corresponding video encoding parameters to encode the video data, the video quality can be improved and the picture quality can be improved. For ordinary live broadcasts, when there is no microphone connection request and the number of viewers is small, such as game live broadcasts or sports live broadcasts, the video smoothness can be improved by selecting the corresponding video encoding parameters for encoding processing. Therefore, for each live broadcast scenario, targeted video encoding processing is required to improve the user's subjective experience.

[0082] In an embodiment of the present application, an image group included in the video data to be processed and a target scene to which the image group belongs are obtained; the image group includes multiple video frames; based on the target scene to which the image group belongs, video encoding parameters for the image group are determined; the video encoding parameters of the image group are used to perform video encoding processing on the multiple video frames included in the image group; the types of the multiple video frames are respectively obtained, and based on the video encoding parameters of the image group and the types of the multiple video frames, video encoding processing is performed on the multiple video frames included in the image group respectively to obtain the encoded data of the multiple video frames included in the image group. Since the corresponding video encoding parameters are selected in combination with the scene to which the image group belongs when performing video encoding processing on the video frames in the image group, it is possible to implement video encoding of the video data in combination with a specific scene, thereby enriching the video encoding method. Moreover, since the corresponding video encoding method is selected in combination with the type of each video frame in the image group for video encoding processing, the accuracy of video encoding can be improved.

[0083] For further information, see Figure 5 , Figure 5 is a flow chart of another video encoding method provided by an embodiment of the present application; Figure 5 As shown, the video encoding method can be applied to a video processing system, the video processing system is used to perform video encoding processing and video playback on video data, the video processing system includes a server and a client, and the video encoding method includes but is not limited to the following steps:

[0084] S201: The server obtains a first image group included in first video data and a target scene to which the first image group belongs.

[0085] In the embodiment of the present application, the first video data may be the same as the video data to be processed, or the first video data may be part of the video data to be processed. For example, the first video data may refer to the video data collected after the anchor terminal starts broadcasting, and when determining the scene to which the first video data belongs, the video encoding processing may be combined with the scene and displayed.

[0086] In one implementation, the target scene to which the image group belongs may be determined in the following manner. For example, scene indication information may be obtained; the scene indication information is used to indicate the scene to which the image group belongs; the scene indicated by the scene indication information is determined as the target scene to which the image group belongs; the scene indication information is obtained by the service end through detecting service data.

[0087] That is to say, during the live broadcast process, the business end can be used to detect business data, determine the current scene based on the detected business data, and generate scene indication information for indicating the current scene. The scene indication information can be used to indicate which scene the current scene detected by the business end is. By sending the scene indication information to the server, the server can determine the scene indicated by the scene indication information as the target scene to which the image group belongs. Among them, the business data can be used to indicate the business type of the current live broadcast room, and the live broadcast scene corresponding to the business type can be determined as the scene corresponding to the business data. The business end can quickly detect the live broadcast scene corresponding to the business data, thereby determining the target scene to which the image group belongs.

[0088] Optionally, the service end can detect service data during the entire live broadcast process to determine the current scene, thereby improving the accuracy of scene detection and ensuring the timeliness of scene detection. Alternatively, the service end can also detect service data once every target time to determine the current scene. Alternatively, the service end can also detect service data to determine the current scene when receiving a scene detection instruction, thereby saving overhead.

[0089] In another implementation, the target scene to which the image group belongs can be determined by image detection. For example, image recognition can be performed on multiple video frames in the image group respectively to obtain scene recognition results for the multiple video frames in the image group; the scene recognition result of each video frame is used to indicate the scene to which each video frame belongs; based on the scene recognition results of the multiple video frames, the target scene to which the image group belongs is determined.

[0090] Among them, by performing image recognition on each video frame in the image group, the scene recognition result of each video frame can be determined, so as to determine the target scene to which the image group belongs based on the scene recognition results of multiple video frames. Optionally, for example, if the number of video frames belonging to the first scene among the multiple video frames included in the image group is greater than the number of video frames belonging to the second scene, then it can be determined that the target scene to which the image group belongs is the first scene, so as to determine the scene to which the image group belongs. Alternatively, if the first number of video frames in the image group belong to the first scene and the second number of video frames belong to the second scene, then it can be determined that the target scene to which the image group belongs includes the first scene and the second scene. Subsequently, when determining the video encoding parameters of the image group, the video encoding parameters of the first number of video frames in the image group and the video encoding parameters of the second number of video frames in the image group can be determined respectively, so as to perform video encoding processing on the multiple video frames in the image group based on their respective video encoding parameters.

[0091] In an optional implementation, a scene recognition model may be used to perform image recognition on multiple video frames in the image group to obtain scene recognition results for the multiple video frames in the image group. The scene recognition model may include, for example, but is not limited to, a convolutional neural network model (CNN), a recurrent neural network model (RNN), a deep neural network model (DNN), and the like.

[0092] In an optional implementation, a large number of training samples and sample labels can be acquired in advance to train the scene recognition model. For example, the training samples can refer to a plurality of sample video frames contained in a large number of sample image groups, and the sample labels can be used to indicate the sample scene to which each sample video frame in the plurality of sample video frames actually belongs. When the scene recognition model is trained using the training samples, the scene recognition model is adjusted based on the difference between the sample scene to which the sample video frame output by the scene recognition model belongs and the sample label, so that the trained scene recognition model has the ability to recognize the scene to which the video frame belongs, so that the trained scene recognition model can be used to perform image recognition on the plurality of video frames in the image group, and obtain scene recognition results for the plurality of video frames in the image group.

[0093] In the embodiment of the present application, the method for determining the target scene to which the first image group included in the first video data belongs can refer to the above-mentioned method for determining the target scene to which the image group included in the video data to be processed belongs, and will not be described in detail here. By obtaining the target scene to which the first image group belongs, it is convenient to perform targeted video encoding processing in combination with the target scene to which the first image group belongs, thereby improving the video encoding effect.

[0094] S202: The server determines a video encoding parameter for the first image group based on a target scene to which the first image group belongs.

[0095] S203, the server obtains the types of multiple video frames included in the first image group respectively, and based on the video encoding parameters of the first image group and the types of video frames included in the multiple first image groups, performs video encoding processing on the multiple video frames included in the first image group respectively to obtain encoded data of the multiple video frames included in the first image group.

[0096] S204: The server transmits the encoded data of the multiple video frames included in the first image group to the client.

[0097] S205, the client performs video decoding processing on the encoded data of multiple video frames included in the first image group to obtain first decoded data, performs rendering processing on the first decoded data, and outputs the rendered first decoded data.

[0098] In the embodiment of the present application, the specific implementation of steps S201 to S205 can refer to the implementation of the above-mentioned steps S101 to S103, which will not be repeated here.

[0099] S206: If the second video data is acquired, and the target scene to which the second image group contained in the second video data belongs is different from the target scene to which the first image group belongs, the server determines the video encoding parameters for the second image group based on the target scene to which the second image group belongs.

[0100] Among them, the second video data may be the same as the video data to be processed, or the second video data may be part of the video data to be processed, and the target scene to which the second image group contained in the second video data belongs is different from the target scene to which the first image group belongs. For example, the first video data and the second video data are video data of the same live video broadcast in different time periods. For example, the collection period of the first video data may be earlier than the data collection period of the second video data. For example, after the anchor starts broadcasting, the first video data is first obtained, and the video encoding is processed and displayed in combination with the scene. After a period of time, the second video data is obtained, and the video encoding is processed and displayed in combination with the scene. For example, the first video data may refer to the video data obtained in the live broadcast scene with microphones connected, and the second video data may refer to the video data obtained in the ordinary live broadcast scene.

[0101] In an embodiment of the present application, the method for determining the target scene to which the second image group contained in the second video data belongs can refer to the above-mentioned method for determining the target scene to which the image group contained in the video data to be processed belongs, and will not be repeated here. By obtaining the target scene to which the second image group belongs, it is convenient to perform targeted video encoding processing in combination with the target scene to which the second image group belongs, thereby improving the video encoding effect. Furthermore, by obtaining video data of different time periods in a live broadcast scene, scene recognition can be performed on the video data of different time periods to determine whether the live broadcast scene has changed. If the live broadcast scene has changed, the video encoding parameters under the corresponding scene can be used for video encoding processing, instead of fixedly using the video encoding parameters before the live broadcast scene changes for video encoding processing, which can improve the video encoding effect.

[0102] S207, the server obtains the types of multiple video frames included in the second image group respectively, and based on the video encoding parameters of the second image group and the types of video frames included in the multiple second image groups, performs video encoding processing on the multiple video frames included in the second image group respectively, to obtain encoded data of the multiple video frames included in the second image group.

[0103] S208: The server transmits the encoded data of the multiple video frames included in the second image group to the client.

[0104] S209, the client performs video decoding processing on the encoded data of the multiple video frames included in the second image group to obtain second decoded data, performs rendering processing on the second decoded data, and outputs the rendered second decoded data.

[0105] In the embodiment of the present application, the specific implementation of steps S206 to S209 can refer to the implementation of the above-mentioned steps S101 to S103, which will not be repeated here.

[0106] In the embodiment of the present application, since the video encoding parameters of the corresponding scene are selected for encoding when the live broadcast scene changes, the corresponding video encoding parameters can be dynamically adjusted according to the dynamic changes of the live broadcast scene, thereby improving the video encoding effect. Selecting different video encoding parameters for different live broadcast scenes can improve the picture quality, smoothness or reduce latency of the entire live broadcast, thereby improving the subjective experience of the audience. In the live video broadcast, the specific live broadcast scenes are distinguished, and the video encoding parameters in different scenes are adaptively adjusted, so that the video encoding effect is efficient and meets expectations.

[0107] In an embodiment of the present application, an image group included in the video data to be processed and a target scene to which the image group belongs are obtained; the image group includes multiple video frames; based on the target scene to which the image group belongs, video encoding parameters for the image group are determined; the video encoding parameters of the image group are used to perform video encoding processing on the multiple video frames included in the image group; the types of the multiple video frames are respectively obtained, and based on the video encoding parameters of the image group and the types of the multiple video frames, video encoding processing is performed on the multiple video frames included in the image group respectively to obtain the encoded data of the multiple video frames included in the image group. Since the corresponding video encoding parameters are selected in combination with the scene to which the image group belongs when performing video encoding processing on the video frames in the image group, it is possible to implement video encoding of the video data in combination with a specific scene, thereby enriching the video encoding method. Moreover, since the corresponding video encoding method is selected in combination with the type of each video frame in the image group for video encoding processing, the accuracy of video encoding can be improved.

[0108] The method of the embodiment of the present application is introduced above, and the device of the embodiment of the present application is introduced below.

[0109] See also Figure 6 , Figure 6 : is a schematic diagram of the composition structure of a video encoding device provided in an embodiment of the present application. The above-mentioned video encoding device can be deployed on a computer device; the video encoding device can be used to execute the corresponding steps in the video encoding method provided in an embodiment of the present application. The video encoding device 60 includes:

[0110] The data acquisition unit 601 is used to acquire an image group included in the video data to be processed and a target scene to which the image group belongs; the image group includes multiple video frames;

[0111] A parameter determination unit 602 is used to determine a video encoding parameter for the image group based on the target scene to which the image group belongs; the video encoding parameter of the image group is used to perform video encoding processing on multiple video frames included in the image group;

[0112] The video encoding unit 603 is used to respectively obtain the types of the multiple video frames, and based on the video encoding parameters of the image group and the types of the multiple video frames, respectively perform video encoding processing on the multiple video frames included in the image group to obtain encoded data of the multiple video frames included in the image group.

[0113] Optionally, the video encoding parameters of the picture group include at least one of the following:

[0114] Number of reference frames, fractional pixel search accuracy, number of cached frames;

[0115] Among them, the number of reference frames refers to the number of reference video frames required to encode each video frame contained in the image group, the fractional pixel search accuracy is used to search for a reference image block in the reference frame that matches the target image block in each video frame, the reference image block refers to an image block in the reference frame whose similarity with the pixels of the target image block is greater than a similarity threshold, the number of cached frames refers to the number of cached video frames, and the number of cached frames is used to allocate the bit rate of each video frame in the image group.

[0116] Optionally, the target scene to which the image group belongs includes a first scene, a second scene or a third scene, the first scene is a scene where live broadcast with microphones connected is required, the second scene is a scene where live broadcast with microphones connected is not required and the number of live broadcast viewers is less than or equal to a preset number, and the third scene is a scene where live broadcast with microphones connected is not required and the number of live broadcast viewers is greater than the preset number; the parameter determination unit 602 is specifically used to:

[0117] If the target scene to which the image group belongs is the first scene, determining that the number of reference frames belongs to the first number interval, the fractional pixel search precision belongs to the first search precision interval, and the number of cached frames belongs to the first cached number interval;

[0118] If the target scene to which the image group belongs is the second scene, it is determined that the number of reference frames belongs to the first number interval, the fractional pixel search precision belongs to the second search precision interval, and the number of cached frames belongs to the second cache quantity interval; the minimum value of the second search precision interval is greater than the maximum value of the first search precision interval, and the minimum value of the second cache quantity interval is greater than the maximum value of the first cache quantity interval;

[0119] If the target scene to which the image group belongs is the third scene, it is determined that the number of reference frames belongs to the second number interval, the fractional pixel search accuracy belongs to the second search accuracy interval, and the number of cached frames belongs to the third cache number interval; the minimum value of the second number interval is greater than the maximum value of the first number interval, and the minimum value of the third cache number interval is greater than the maximum value of the second cache number interval.

[0120] Optionally, the types of video frames include key frame types, forward reference frame types, and bidirectional reference frame types, and each type of video frame has a different video encoding method; the video encoding unit 603 is specifically used to:

[0121] If the type of the video frame in the image group is a key frame type, then obtain the encoding mode corresponding to the key frame, use the encoding mode corresponding to the key frame and the video encoding parameters of the image group, perform video encoding processing on the key frame in the image group, and obtain encoding data of the key frame in the image group;

[0122] If the type of the video frame in the picture group is a forward reference frame type, then obtaining a coding mode corresponding to the forward reference frame, using the coding mode corresponding to the forward reference frame and the video coding parameters of the picture group, performing video coding processing on the forward reference frame in the picture group, and obtaining coded data of the forward reference frame in the picture group;

[0123] If there is a video frame in the image group that is a bidirectional reference frame type, the encoding method corresponding to the bidirectional reference frame is obtained, and the encoding method corresponding to the bidirectional reference frame and the video encoding parameters of the image group are used to perform video encoding processing on the bidirectional reference frame in the image group to obtain the encoded data of the bidirectional reference frame in the image group.

[0124] Optionally, the video encoding unit 603 is specifically configured to:

[0125] Performing prediction processing on multiple video frames included in the image group to obtain the complexity of the multiple video frames included in the image group;

[0126] Determining bit rates of the multiple video frames included in the group of images based on the complexity of the multiple video frames included in the group of images;

[0127] Based on the video encoding parameters of the image group, the types of multiple video frames included in the image group and the bit rates of the multiple video frames, video encoding processing is performed on the multiple video frames included in the image group to obtain encoded data of the multiple video frames included in the image group.

[0128] Optionally, the video data to be processed includes a plurality of image groups, any two consecutive image groups among the plurality of image groups are a first image group and a second image group, and the first image group and the second image group belong to different target scenes; the parameter determination unit 602 is specifically used to:

[0129] Determining a video encoding parameter for the first group of images based on a target scene to which the first group of images belongs;

[0130] Based on the target scene to which the second image group belongs, a video encoding parameter for the second image group is determined; the video encoding parameter for the second image group is different from the video encoding parameter for the first image group.

[0131] Optionally, the data acquisition unit 601 is specifically used to:

[0132] Acquire scene indication information; the scene indication information is used to indicate the scene to which the image group belongs;

[0133] The scene indicated by the scene indication information is determined as the target scene to which the image group belongs; the scene indication information is obtained by the service end through detecting service data.

[0134] Optionally, the data acquisition unit 601 is specifically used to:

[0135] Performing image recognition on the multiple video frames in the image group respectively to obtain scene recognition results for the multiple video frames in the image group; the scene recognition result of each video frame is used to indicate the scene to which each video frame belongs;

[0136] Based on the scene recognition results of the multiple video frames, a target scene to which the image group belongs is determined.

[0137] Optionally, the video encoding device 60 further includes a data sending unit 604, and the data sending unit 604 is used to:

[0138] The encoded data of multiple video frames included in the image group are transmitted to the client, so that the client performs video decoding processing on the encoded data of multiple video frames included in the image group to obtain decoded data, renders the decoded data, and outputs the decoded data after rendering.

[0139] It should be noted that Figure 6 For the contents not mentioned in the corresponding embodiments, please refer to the description of the method embodiments, which will not be repeated here.

[0140] In an embodiment of the present application, an image group included in the video data to be processed and a target scene to which the image group belongs are obtained; the image group includes multiple video frames; based on the target scene to which the image group belongs, video encoding parameters for the image group are determined; the video encoding parameters of the image group are used to perform video encoding processing on the multiple video frames included in the image group; the types of the multiple video frames are respectively obtained, and based on the video encoding parameters of the image group and the types of the multiple video frames, video encoding processing is performed on the multiple video frames included in the image group respectively to obtain the encoded data of the multiple video frames included in the image group. Since the corresponding video encoding parameters are selected in combination with the scene to which the image group belongs when performing video encoding processing on the video frames in the image group, it is possible to implement video encoding of the video data in combination with a specific scene, thereby enriching the video encoding method. Moreover, since the corresponding video encoding method is selected in combination with the type of each video frame in the image group for video encoding processing, the accuracy of video encoding can be improved.

[0141] See also Figure 7 , Figure 7 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 7 As shown, the above-mentioned computer device 70 may include: a processor 701, a network interface 704 and a memory 705. In addition, the above-mentioned computer device 70 may also include: a user interface 703 and at least one communication bus 702. Among them, the communication bus 702 is used to realize the connection and communication between these components. Among them, the user interface 703 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 703 may also include a standard wired interface and a wireless interface. The network interface 704 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 705 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 705 may optionally also be at least one storage device located away from the aforementioned processor 701. As Figure 7 As shown, the memory 705 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0142] exist Figure 7 In the computer device 70 shown, the network interface 704 can provide a network communication function; the user interface 703 is mainly used to provide an input interface for the user; and the processor 701 can be used to call the device control application stored in the memory 705 to achieve:

[0143] Acquire an image group included in the video data to be processed and a target scene to which the image group belongs; the image group includes multiple video frames;

[0144] Based on the target scene to which the image group belongs, determining a video encoding parameter for the image group; the video encoding parameter for the image group is used to perform video encoding processing on a plurality of video frames included in the image group;

[0145] The types of the multiple video frames are respectively obtained, and based on the video encoding parameters of the image group and the types of the multiple video frames, video encoding processing is performed on the multiple video frames included in the image group to obtain encoded data of the multiple video frames included in the image group.

[0146] It should be understood that the computer device 70 described in the embodiment of the present application can execute the above Figure 2 and Figure 5 The description of the above video encoding method in the corresponding embodiment can also be performed as described above. Figure 6 The description of the video encoding device in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here.

[0147] In an embodiment of the present application, an image group included in the video data to be processed and a target scene to which the image group belongs are obtained; the image group includes multiple video frames; based on the target scene to which the image group belongs, video encoding parameters for the image group are determined; the video encoding parameters of the image group are used to perform video encoding processing on the multiple video frames included in the image group; the types of the multiple video frames are respectively obtained, and based on the video encoding parameters of the image group and the types of the multiple video frames, video encoding processing is performed on the multiple video frames included in the image group respectively to obtain the encoded data of the multiple video frames included in the image group. Since the corresponding video encoding parameters are selected in combination with the scene to which the image group belongs when performing video encoding processing on the video frames in the image group, it is possible to implement video encoding of the video data in combination with a specific scene, thereby enriching the video encoding method. Moreover, since the corresponding video encoding method is selected in combination with the type of each video frame in the image group for video encoding processing, the accuracy of video encoding can be improved.

[0148] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, which includes program instructions, which, when executed by a computer, causes the computer to execute the method of the aforementioned embodiment, and the computer may be part of the computer device mentioned above. For example, the processor 701 mentioned above. As an example, the program instructions may be deployed on a computer device for execution, or deployed on multiple computer devices located at one location for execution, or, alternatively, executed on multiple computer devices distributed at multiple locations and interconnected by a communication network, and multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain network.

[0149] Those skilled in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0150] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A video encoding method, characterized in that: The method comprises: Obtaining an image group included in the video data to be processed, and a target scene to which each image group belongs; the image group includes a plurality of video frames, the target scene to which the image group belongs is one of a plurality of live broadcast scenes, the plurality of live broadcast scenes including a first scene requiring live broadcast with microphones connected, a second scene requiring no live broadcast with microphones connected and the number of live broadcast viewers being less than or equal to a preset number, and a third scene requiring no live broadcast with microphones connected and the number of live broadcast viewers being greater than the preset number; Based on the target scene to which the image group belongs, determining a video encoding parameter for the image group; the video encoding parameter of the image group is used to perform video encoding processing on a plurality of video frames included in the image group, and different video encoding parameters of the image group corresponding to different live broadcast scenes are different; respectively obtaining types of the plurality of video frames, and performing video encoding processing on the plurality of video frames included in the image group based on the video encoding parameters of the image group and the types of the plurality of video frames, to obtain encoded data of the plurality of video frames included in the image group; The video encoding parameters of the image group include the number of reference frames, fractional pixel search accuracy and the number of cached frames. The number of reference frames refers to the number of video frames required for reference to encode each video frame included in the image group. The fractional pixel search accuracy is used to search for a reference image block in the reference frame that matches a target image block in each video frame. The reference image block refers to an image block in the reference frame whose pixel similarity with the target image block is greater than a similarity threshold. The number of cached frames refers to the number of cached video frames. The number of cached frames is used to allocate a bit rate for each video frame in the image group. The determining of the video encoding parameters for the group of images based on the target scene to which the group of images belongs includes: If the target scene to which the image group belongs is the first scene, determining that the number of reference frames belongs to a first number interval, the fractional pixel search accuracy belongs to a first search accuracy interval, and the number of cached frames belongs to a first cached number interval; If the target scene to which the image group belongs is the second scene, it is determined that the number of reference frames belongs to the first number interval, the fractional pixel search precision belongs to the second search precision interval, and the number of cached frames belongs to the second cache quantity interval; the minimum value of the second search precision interval is greater than the maximum value of the first search precision interval, and the minimum value of the second cache quantity interval is greater than the maximum value of the first cache quantity interval; If the target scene to which the image group belongs is the third scene, it is determined that the number of reference frames belongs to the second number interval, the fractional pixel search accuracy belongs to the second search accuracy interval, and the number of cached frames belongs to the third cache number interval; the minimum value of the second number interval is greater than the maximum value of the first number interval, and the minimum value of the third cache number interval is greater than the maximum value of the second cache number interval.

2. The method according to claim 1, characterized in that The types of video frames include key frame types, forward reference frame types and bidirectional reference frame types, and the video encoding method of each type of video frame is different; The performing video encoding processing on the multiple video frames included in the group of images based on the video encoding parameters of the group of images and the types of the multiple video frames to obtain the encoded data of the multiple video frames included in the group of images includes: If the type of the video frame in the image group is a key frame type, obtaining a coding mode corresponding to the key frame, and performing video coding processing on the key frame in the image group using the coding mode corresponding to the key frame and the video coding parameters of the image group to obtain coding data of the key frame in the image group; If the type of the video frame in the image group is a forward reference frame type, obtaining a coding mode corresponding to the forward reference frame, and performing video coding processing on the forward reference frame in the image group using the coding mode corresponding to the forward reference frame and the video coding parameters of the image group to obtain coded data of the forward reference frame in the image group; If there is a video frame in the image group that is a bidirectional reference frame type, the encoding method corresponding to the bidirectional reference frame is obtained, and the encoding method corresponding to the bidirectional reference frame and the video encoding parameters of the image group are used to perform video encoding processing on the bidirectional reference frame in the image group to obtain the encoded data of the bidirectional reference frame in the image group.

3. The method according to any one of claims 1 to 2, characterized in that: The performing video encoding processing on the multiple video frames included in the group of images based on the video encoding parameters of the group of images and the types of the multiple video frames to obtain the encoded data of the multiple video frames included in the group of images includes: Performing prediction processing on a plurality of video frames included in the image group to obtain the complexity of the plurality of video frames included in the image group; Determining bit rates of the multiple video frames included in the group of images based on the complexity of the multiple video frames included in the group of images; Based on the video encoding parameters of the image group, the types of multiple video frames included in the image group and the bit rates of the multiple video frames, video encoding processing is performed on the multiple video frames included in the image group to obtain encoded data of the multiple video frames included in the image group.

4. The method according to any one of claims 1 to 2, characterized in that: The video data to be processed includes a plurality of image groups, any two consecutive image groups among the plurality of image groups are a first image group and a second image group, and the first image group and the second image group belong to different target scenes; The determining of the video encoding parameters for the group of images based on the target scene to which the group of images belongs includes: Determining video encoding parameters for the first group of images based on a target scene to which the first group of images belongs; Based on the target scene to which the second image group belongs, video encoding parameters for the second image group are determined; the video encoding parameters for the second image group are different from the video encoding parameters for the first image group.

5. The method according to any one of claims 1-2, characterized in that: The obtaining of the target scene to which the image group belongs includes: Acquire scene indication information; the scene indication information is used to indicate the scene to which the image group belongs, and the scene indication information is obtained by the service end through detecting service data; The scene indicated by the scene indication information is determined as the target scene to which the image group belongs.

6. The method according to any one of claims 1-2, characterized in that: The obtaining of the target scene to which the image group belongs includes: Performing image recognition on the multiple video frames in the image group respectively to obtain scene recognition results for the multiple video frames in the image group; the scene recognition result of each video frame is used to indicate the scene to which each video frame belongs; Based on the scene recognition results of the multiple video frames, a target scene to which the image group belongs is determined.

7. A video encoding device, characterized in that: The device comprises: a data acquisition unit, configured to acquire an image group included in the video data to be processed, and a target scene to which each image group belongs; the image group includes a plurality of video frames, the target scene to which the image group belongs is one of a plurality of live broadcast scenes, the plurality of live broadcast scenes including a first scene requiring live broadcast with microphones connected, a second scene requiring no live broadcast with microphones connected and the number of live broadcast viewers being less than or equal to a preset number, and a third scene requiring no live broadcast with microphones connected and the number of live broadcast viewers being greater than the preset number; A parameter determination unit, configured to determine a video encoding parameter for the image group based on a target scene to which the image group belongs; the video encoding parameter of the image group is used to perform video encoding processing on a plurality of video frames included in the image group, and the video encoding parameters of the image group corresponding to different live broadcast scenes are different; a video encoding unit, configured to respectively obtain types of the plurality of video frames, and based on the video encoding parameters of the group of pictures and the types of the plurality of video frames, respectively perform video encoding processing on the plurality of video frames included in the group of pictures to obtain encoded data of the plurality of video frames included in the group of pictures; The video encoding parameters of the image group include the number of reference frames, fractional pixel search accuracy and the number of cached frames. The number of reference frames refers to the number of video frames required for reference to encode each video frame included in the image group. The fractional pixel search accuracy is used to search for a reference image block in the reference frame that matches a target image block in each video frame. The reference image block refers to an image block in the reference frame whose pixel similarity with the target image block is greater than a similarity threshold. The number of cached frames refers to the number of cached video frames. The number of cached frames is used to allocate a bit rate for each video frame in the image group. The parameter determination unit is specifically used to determine, if the target scene to which the image group belongs is the first scene, whether the number of reference frames belongs to the first number interval, the fractional pixel search accuracy belongs to the first search accuracy interval, and the number of cached frames belongs to the first cache number interval; if the target scene to which the image group belongs is the second scene, whether the number of reference frames belongs to the first number interval, the fractional pixel search accuracy belongs to the second search accuracy interval, and the number of cached frames belongs to the second cache number interval; the minimum value of the second search accuracy interval is greater than the maximum value of the first search accuracy interval, and the minimum value of the second cache number interval is greater than the maximum value of the first cache number interval; if the target scene to which the image group belongs is the third scene, whether the number of reference frames belongs to the second number interval, the fractional pixel search accuracy belongs to the second search accuracy interval, and the number of cached frames belongs to the third cache number interval; the minimum value of the second number interval is greater than the maximum value of the first number interval, and the minimum value of the third cache number interval is greater than the maximum value of the second cache number interval.

8. A computer device, characterized in that: include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a data communication function, the memory is used to store program code, and the processor is used to call the program code so that the computer device executes the method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images