Coding method, coding device and electronic equipment

Abstract

The present application discloses a coding method, coding device, electronic device and computer-readable storage medium. The method includes: calculating the scene complexity of the current frame to be coded; determining the acceptable quantization parameter QP of the current frame to be coded based on the scene complexity and a preset first model, wherein the first model describes the mapping relationship between the scene complexity and the acceptable QP; determining the basic bit rate of the current frame to be coded based on the acceptable QP and a preset second model, wherein the second model describes the mutual mapping relationship between the bit rate and the QP; determining the target bit rate of the current frame to be coded within a specified bit rate range, wherein the minimum bit rate within the specified bit rate range is the basic bit rate, and the maximum bit rate within the specified bit rate range is the preset maximum bit rate; encoding the current frame to be coded according to the target bit rate. Through the solution of the present application, dynamic and intelligent adjustment of the video stream bit rate can be achieved, saving the storage space of the video stream.

Description

Technical Field

[0001] The present application belongs to the field of video processing technology, and in particular relates to an encoding method, an encoding device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of society, economy, science, and technology, more and more electronic devices are equipped with video capture capabilities. However, the rich video information contained in video streams often requires a large amount of storage space, which in turn increases the cost of electronic devices. Currently, electronic devices typically encode each frame of a video stream before storing it, and bitrate control is a key step in the encoding process. The bitrate control strategy adopted by electronic devices can be considered a trade-off between video quality and bitrate.

[0003] Currently, a common rate control strategy involves manually setting the target bitrate of an electronic device to a certain value, such as the maximum bitrate supported by the device. While this rate control strategy can maximize image quality, typical video streams often don't require such a high bitrate. Consequently, existing rate control strategies often waste storage space by setting the target bitrate too high. Summary of the Invention

[0004] The present application provides an encoding method, an encoding device, an electronic device and a computer-readable storage medium, which can realize dynamic intelligent adjustment of the video stream bit rate and save the storage space of the video stream.

[0005] In a first aspect, the present application provides an encoding method, comprising:

[0006] Calculate the scene complexity of the current frame to be encoded;

[0007] Determining an acceptable quantizer parameter (QP) of the current frame to be encoded based on the scene complexity and a preset first model, wherein the first model describes a mapping relationship between the scene complexity and the acceptable QP;

[0008] Determining a base bitrate for the current frame to be encoded based on the acceptable QP and a preset second model, wherein the second model describes a mapping relationship between bitrate and QP;

[0009] Determine the target bit rate of the current frame to be encoded within the specified bit rate range, wherein the minimum bit rate within the specified bit rate range is the base bit rate, and the maximum bit rate within the specified bit rate range is the preset maximum bit rate;

[0010] Encode the current frame to be encoded according to the target bit rate.

[0011] In a second aspect, the present application provides an encoding device, comprising:

[0012] A calculation module, used to calculate the scene complexity of the current frame to be encoded;

[0013] a first determining module, configured to determine an acceptable QP of a current frame to be encoded based on scene complexity and a preset first model, wherein the first model describes a mapping relationship between scene complexity and acceptable QP;

[0014] A second determination module is configured to determine a base bit rate of a current frame to be encoded based on an acceptable QP and a preset second model, wherein the second model describes a mapping relationship between bit rate and QP;

[0015] A third determination module is configured to determine a target bit rate for the current frame to be encoded within a specified bit rate range, wherein a minimum bit rate within the specified bit rate range is a base bit rate, and a maximum bit rate within the specified bit rate range is a preset maximum bit rate;

[0016] The encoding module is used to encode the current frame to be encoded according to the target bit rate.

[0017] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect when executing the computer program.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which implements the steps of the method of the first aspect when executed by a processor.

[0019] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by one or more processors, it implements the steps of the method of the first aspect.

[0020] Compared with the prior art, the present application has the following advantages: when encoding, the electronic device determines an acceptable QP based on the scene complexity of the current frame to be encoded, and then determines the bit rate corresponding to the acceptable QP as the basic bit rate. The electronic device then defines a specified bit rate range based on the basic bit rate and a preset maximum bit rate, and determines the most suitable target bit rate for the current frame to be encoded within the specified bit rate range. Finally, the electronic device encodes the current frame to be encoded according to the target bit rate. The above process takes into account the impact of the scene complexity of the current frame to be encoded on the bit rate, saving bit rate bandwidth and storage space to a certain extent.

[0021] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 This is a schematic diagram of the implementation flow of the encoding method provided in the embodiment of the present application;

[0024] Figure 2 This is a structural block diagram of the encoding device provided in an embodiment of the present application;

[0025] Figure 3 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0027] The encoding method provided in the embodiments of the present application is applied to an electronic device. As an example only, the electronic device may be a device with a video capture function such as a smartphone, a tablet computer, or a surveillance camera, which is not limited here.

[0028] The following uses a surveillance camera as an example to illustrate the encoding method. It can be understood that using a secure digital memory card (SD card) or cloud storage are two common ways for surveillance cameras to store surveillance videos. When the surveillance video is long, the surveillance video will take up a large storage space. For the method of using an SD card, it is easy to cause insufficient storage space; for the method of using cloud storage, the cost required is too high. Based on this, the embodiment of the present application provides an encoding method that can save storage space to a certain extent, and support surveillance cameras for ultra-long SD card recording or low cloud storage costs.

[0029] See also Figure 1 , the implementation process of the encoding method is detailed as follows:

[0030] Step 101: Calculate the scene complexity of the current frame to be encoded.

[0031] The current frame to be encoded refers to the original video frame that the surveillance camera is about to encode. The surveillance camera calculates the scene complexity of the current frame by statistically analyzing and processing the texture features of the current frame. Texture features include gradient, contrast, energy, entropy, and / or correlation, among other dimensions, which are not limited here. For ease of explanation, the scene complexity of the current frame to be encoded can be denoted as c.

[0032] Step 102: Determine an acceptable QP of the current frame to be encoded based on the scene complexity and a preset first model.

[0033] Surveillance cameras have a built-in first model that describes the mapping between scene complexity and acceptable quality-priority (QP). The acceptable QP is the minimum quality (QP) that users can accept after encoding, given the corresponding pre-encoding scene complexity. For ease of explanation, this mapping is represented by f1(), with the acceptable QP being QP0. QP0 = f1(c).

[0034] In some examples, when the scene complexity only considers single-dimensional texture features, the first model can be a simple function model (mapping model). In other examples, when the scene complexity considers multi-dimensional texture features, the first model can be a deep learning model or a machine learning model. The embodiments of the present application do not limit the type of the first model.

[0035] Step 103: Determine the basic bit rate of the current frame to be encoded according to the acceptable QP and the preset second model.

[0036] The surveillance camera has a built-in second model, which describes the mapping relationship between bit rate and QP. For ease of explanation, the mapping relationship between QP and bit rate is expressed as f2(). Let the base bit rate be R1, then R1 = f2(QP0).

[0037] In some embodiments, the training of the second model is briefly described as follows: The second model first sets the initial functional relationship between the bit rate R and QP, which can be expressed as: R = f(k1, k2, QP), where k1 and k2 are relationship parameters. These two relationship parameters are related to the scene and change with the dynamic changes of the scene. They will not be described here. When the encoding module of the electronic device sends QP and its corresponding bit rate R for encoding, the actual bit rate will be obtained. If the actual bit rate does not match the sent bit rate, it means that the scene has changed (usually reflected in changes in the video picture). Based on this, the electronic device can continuously adjust the relationship parameters k1 and k2 in real time according to this change.

[0038] It should be noted that after the second model is put into use, the surveillance camera can further fine-tune the second model based on its specific performance during application. The fine-tuning process is similar to the training process of the second model shown above and will not be repeated here. In other words, the second model can be a dynamic model.

[0039] It can be understood that the second model can be trained on other electronic devices and then deployed and transplanted to the surveillance camera; alternatively, the second model can be trained directly on the surveillance camera. There is no limitation here on the execution entity of the operation of training the second model.

[0040] Step 104: Determine the target bit rate of the current frame to be encoded within the specified bit rate range.

[0041] The minimum bitrate within the specified bitrate range can be set as follows: Since the base bitrate is calculated based on the acceptable QP, and the acceptable QP expresses the QP corresponding to the lowest image quality acceptable to the user under the scene complexity of the current frame to be encoded, the base bitrate can be used as the minimum bitrate within the specified bitrate range.

[0042] The maximum bitrate within a specified bitrate range can be set by using a preset maximum bitrate as the maximum bitrate within the specified bitrate range. Obviously, this preset maximum bitrate is a fixed value that does not change with changes in the scene captured by the surveillance camera. For example, this maximum bitrate can be determined based on the surveillance camera's memory, storage duration, or bandwidth, or it can be set by the user, without limitation here.

[0043] Let's assume the preset maximum bitrate is R2, and the specified bitrate range can be expressed as [R1, R2]. It can be understood that, based on this specified bitrate range, since the surveillance camera determines the target bitrate for the current frame to be encoded within this specified bitrate range, the target bitrate for the current frame to be encoded can be as low as R1 or as high as R2.

[0044] Step 105: Encode the current frame to be encoded according to the target bit rate.

[0045] After calculating the target bitrate for the current frame, the surveillance camera can encode it using this target bitrate as the basis for encoding. Specifically, based on the target bitrate, the surveillance camera calculates the corresponding coding parameters. These parameters are then sent to the camera's encoder, which then encodes the current frame based on the received coding parameters.

[0046] In some embodiments, the surveillance camera may determine a target bit rate suitable for the current frame to be encoded within a specified bit rate range based on motion information of the current frame to be encoded. Step 104 may include:

[0047] A1. Determine the real-time motion region area of the current frame to be encoded.

[0048] The real-time motion area of the current frame to be encoded can be obtained in the following ways:

[0049] The first method is to obtain the real-time motion region of the previous frame to be encoded based on the frame difference information between the current frame to be encoded and the previous encoded frame. Specifically, the real-time motion region of the previous frame to be encoded is determined based on the number of pixel pairs that differ between the current frame to be encoded and the previous encoded frame. It can be understood that since encoding does not change the size of the video frame (i.e., length and width), each pixel of the current frame to be encoded can find a pixel with a corresponding position (i.e., the same coordinates) in the previous encoded frame. That is, if the size of the current frame to be encoded is N*M, there are N*M pixel pairs between the current frame to be encoded and the previous encoded frame. By judging whether there are differences among these N*M pixel pairs, the number L of pixel pairs with differences can be found, and the real-time motion region of the current frame to be encoded can be determined based on this number L.

[0050] It should be noted that the coded frame specifically refers to the video frame obtained after encoding. That is, if the current frame to be encoded is the nth original video frame, then the coded frame before the current frame to be encoded refers to the video frame obtained by encoding the n-1th original video frame.

[0051] However, since the current frame to be encoded is the video frame before encoding, and the previous encoded frame is the video frame obtained after encoding, and the encoding operation inevitably brings certain encoding losses, it is not easy for the surveillance camera to determine the pixel difference between the current frame to be encoded and the previous encoded frame; that is, the real-time motion area of the current frame to be encoded obtained in this way may have a certain deviation.

[0052] The second method is to predict the motion information of the N frames encoded before the current frame to be encoded. The definition of encoded frames can be found in the previous section and will not be repeated here. The motion information may include, but is not limited to, the motion position, the speed of position change, and / or the area of the motion region, which is not limited here. It is understood that when the scene captured by the surveillance camera does not change significantly, the changes between adjacent video frames usually follow certain patterns. Based on this, a machine learning or deep learning model can be pre-trained and the motion information of the previous N frames encoded as input to achieve real-time prediction of the motion region area of the current frame to be encoded. The motion information of the previous N frames encoded can be directly obtained from the encoding process of the previous N frames; that is, the motion information of the previous N frames encoded is the real value. If the current frame to be encoded is the nth original video frame, the previous N frames encoded are: the n-1th encoded frame, the n-2th encoded frame, and so on.

[0053] However, when prediction is performed in this manner, the specific actual situation of the current frame to be encoded is not taken into consideration. Therefore, the real-time motion region area of the current frame to be encoded obtained in this manner may also have a certain deviation.

[0054] The third method is obtained by combining the above two methods. That is, on the one hand, the frame difference information between the current frame to be encoded and the previous encoded frame is determined, and on the other hand, the motion information of the N frames of encoded frames before the current frame to be encoded is obtained, and finally the real-time motion region area of the current frame to be encoded is predicted based on the frame difference information, the motion information and the preset third model, wherein the third model learns the change law of the motion region. It can be understood that the corresponding real-time motion region area can be obtained through the frame difference information, which is recorded as the first real-time motion region area; the corresponding real-time motion region area can also be obtained through the motion information of the N frames of encoded frames before the current frame to be encoded and the third model, which is recorded as the second real-time motion region area; finally, the third model integrates the first real-time motion region area and the second real-time motion region area, and obtains the final real-time motion region area by adjusting the weights of the first real-time motion region area and the second real-time motion region area and outputs it.

[0055] In some embodiments, the training of the third model is briefly described as follows: the third model processes the video stream samples, specifically weightedly predicting whether each image block of the to-be-encoded frame is in motion based on the frame difference information between the to-be-encoded frame and the previous encoded frame, as well as the motion information of the previous N encoded frames, thereby obtaining the predicted motion area of the to-be-encoded frame; when encoding the to-be-encoded frame according to specific coding parameters, the encoding module itself can calculate the accurate motion area of the to-be-encoded frame, and thus use the accurate motion area as a label to calculate the loss of the third model and optimize the model parameters of the third model to adjust the weights. After the loss reaches convergence, or after the training rounds have met the requirements, the trained third model can be obtained.

[0056] It should be noted that after the third model is implemented, the surveillance camera can fine-tune the third model based on its performance during application. This fine-tuning process is similar to the training process for the third model described above and will not be further described here. In other words, the third model can be a dynamic model.

[0057] It can be understood that the third model can be trained on other electronic devices and then deployed and transplanted to the surveillance camera; alternatively, the third model can be trained directly on the surveillance camera. No limitation is imposed here on the execution entity of the operation of training the third model.

[0058] A2. Calculate the target bit rate within the specified bit rate range based on the area of the real-time motion region.

[0059] The surveillance camera can pre-set the motion region area corresponding to the maximum bit rate it supports, and the motion region area is the maximum motion region area. The maximum bit rate has been described above and will not be repeated here. As an example only, the surveillance camera can set the maximum motion region area to the entire area (total area) of a video frame; or, it can also be set to a certain proportion of the overall area. For example, if the maximum motion region area is set to 60% of the overall area of a video frame, it means that when 60% of the area of the current frame to be encoded is in motion, the surveillance camera will increase the bit rate to the maximum bit rate.

[0060] The surveillance camera can calculate the ratio of the real-time motion area of the current frame to be encoded to the preset maximum motion area. For ease of explanation, the real-time motion area of the current frame to be encoded can be recorded as S p , the maximum motion area is S max , then we can calculate the ratio b=S p / S max Based on this ratio, the basic bit rate R1 and the maximum bit rate R2 mentioned above, the monitoring device can calculate the target bit rate R of the current frame to be encoded by the following formula: t :

[0061] R t =R1+b*(R2-R1)

[0062] As described above, R1 is the base bitrate based on the scene complexity of the current frame to be encoded, R2 is the maximum bitrate that the surveillance camera can provide, and b expresses the motion ratio of the current frame to be encoded. The above formula shows that in extreme cases: if the real-time motion area of the current frame to be encoded is 0, it means that the current frame to be encoded has hardly changed compared to the previous frame. Since b is 0, the surveillance camera will encode the current frame to be encoded at the base bitrate, which saves resources. If the real-time motion area of the current frame to be encoded reaches the maximum motion area, it means that the current frame to be encoded has the greatest degree of motion compared to the previous frame. Since b is 1, the surveillance camera will encode the current frame to be encoded at the maximum bitrate, which ensures image quality.

[0063] In some embodiments, step 105 specifically includes:

[0064] B1. Determine the target QP of the current frame to be encoded according to the target bit rate and the second model.

[0065] As mentioned above, the second model describes the mapping relationship between bit rate and QP. Based on this, when the target bit rate is known, the QP corresponding to the target bit rate can be determined by the second model, and this QP is the target QP of the current frame to be encoded. As described above, f2() expresses the mapping relationship between QP and bit rate, so the mapping relationship between bit rate and QP can be expressed as f2 -1 (). Let the target QP be QP t , then QP t =f2 -1 (R t ).

[0066] B2. Determine the target frame size of the current frame to be encoded according to the target bit rate.

[0067] The target frame size here does not refer to dimensions (i.e., length and width). Instead, it refers to the expected data size of the encoded frame after encoding the frame to be encoded, measured in bits or bytes. This target frame size can be roughly understood as the target space occupied.

[0068] The surveillance camera can obtain the number of video frames in one second through the preset frame rate. Based on the target bit rate and the number of video frames in one second, combined with the distribution ratio of frame types in the video stream, the target frame size of the frame to be encoded can be calculated.

[0069] B3. Send the target QP and target frame size to the encoder to instruct the encoder to encode the current frame to be encoded.

[0070] In surveillance cameras, the actual encoding operation is performed by the camera's hardware component—the encoder. After calculating the target QP and target frame size using the above method, the surveillance device sends these coding parameters to the encoder, which then encodes the current frame using these parameters.

[0071] In some embodiments, the first model also needs to be trained before it can be put into use. It should be noted that during the application of the first model, the surveillance camera will not fine-tune or optimize it. The training process of the first model is briefly described as follows:

[0072] C1. Construct the image dataset to be encoded.

[0073] Under different shooting environments, R&D personnel can use surveillance cameras or other electronic devices capable of shooting to obtain images under the corresponding shooting environments. These images are the images to be encoded. It is understandable that under the same shooting environment, multiple different images to be encoded may be obtained.

[0074] For example, in a library environment, an image can be captured every minute to be encoded. This means that multiple images can be captured in the library environment in a single day. Similarly, in a park environment, an image can be captured every minute to be encoded. This means that multiple images can be captured in the park environment in a single day. This can be done by analogy to construct a dataset to be encoded.

[0075] C2. For each image to be encoded in the image data set to be encoded, encode the image to be encoded according to at least two QPs to obtain at least two encoded images corresponding to the image to be encoded.

[0076] At least two QPs can be specified within a preset QP range (e.g., 25 to 52). Thus, for any image to be encoded, it can be encoded according to the at least two QPs, thereby obtaining at least two encoded images corresponding to the image to be encoded. Assuming that there are X images to be encoded in the image dataset to be encoded and Y QPs are preset, since each image to be encoded can obtain Y corresponding encoded images, ultimately X*Y encoded images are obtained.

[0077] For example, there are I1, I2, I3, ..., I in the image data set to be encoded. X These X pictures to be coded are assigned QP1, QP2, ..., QP YFor this Y QP, then: for the image to be coded I1, the image to be coded I1 is coded according to QP1, and the coded image I 11 ; Encode the image I1 to be encoded according to QP2 to obtain the encoded image I 12 ; And so on, we can get the coded image I 1Y The same encoding method is used for each other to-be-encoded image, and finally X*Y encoded images are obtained.

[0078] C3. Calculate the scene complexity of each image to be encoded.

[0079] For example, there are I1, I2, I3, ..., I in the image data set to be encoded. X The scene complexity C1 of the image to be encoded I1 and the scene complexity C2 of the image to be encoded I2 can be calculated from these X images to be encoded, and so on, which will not be repeated here.

[0080] C4. Get the score of each encoded image.

[0081] The rating of an encoded image is a subjective assessment of image quality by humans. Specifically, each encoded image can be pushed to an expert system, where multiple expert users will rate it. The average of the scores received by each expert user is the corresponding rating. Subsequent data capture from the expert system is then sufficient to obtain the rating for each encoded image.

[0082] C5. For each image to be encoded, screening is performed according to a preset target score and the score of each encoded image corresponding to the image to be encoded, to obtain a target encoded image corresponding to the image to be encoded.

[0083] As described above, each image to be encoded corresponds to Y coded images. Since the first model is trained here, it aims to describe the mapping between scene complexity and acceptable QP. The acceptable QP is defined as the QP corresponding to the minimum quality after encoding that the user can accept under the corresponding pre-encoding scene complexity. Therefore, for any image to be encoded, its corresponding target coded image can be determined as follows:

[0084] comparing the score of each coded image corresponding to the image to be coded with the target score to determine a candidate coded image, wherein the score of the candidate coded image is not less than the target score;

[0085] The candidate coded image with the lowest score is determined as the target coded image.

[0086] It can be understood that the above process is to find the encoded image with a score closest to the target score on the basis of not being lower than the target score.

[0087] For example, assuming Y is 5, there are five QPs: QP1, QP2, QP3, QP4, and QP5; accordingly, the image to be encoded I1 will have I 11 , I 12 , I 13 , I 14 , I 15 These five coded images. Assume that after step C4, the coded image I 11 The score is 62, the encoded image I 12 The score is 70, the encoded image I 13 The score is 83, the coded image I 14 The score is 81, the encoded image I 15 The score of the image to be coded is 89, and the target score is 80, then the candidate coded images of the image to be coded I1 can be determined as follows: 13 , I 14 and I 15 Since the coded image I 13 , I 14 and I 15 middle, I 14 has the lowest score; that is, the coded image I 14 It is the coded image with the score closest to the target score found on the basis of not less than the target score. 14 By analogy, the target coded image corresponding to each other image to be coded can be found.

[0088] C6. Train the first model according to the QP corresponding to each target encoded image and the scene complexity of each image to be encoded.

[0089] Each coded image is obtained by encoding the corresponding image to be coded according to a certain QP. Therefore, each coded image has a corresponding QP, and each coded image uniquely corresponds to a single image to be coded. Furthermore, the scene complexity of each image to be coded is also known. Therefore, given X images to be coded, X sets of mappings between scene complexity and QP can be formed. Using this mapping as label data and the scene complexity of each image to be coded as input data, the first model can be trained.

[0090] It can be understood that the first model can be trained on other electronic devices and then deployed and transplanted to the surveillance camera; alternatively, the first model can be trained directly on the surveillance camera. No limitation is imposed here on the execution entity of the operation of training the first model.

[0091] As can be seen from the above, the following technical effects are achieved through the embodiments of the present application: First, the encoding method combines scene complexity entropy masking and the impact of motion on bitrate, maximizing bitrate bandwidth and storage space savings while ensuring subjective visual quality. Second, the second model describes the mapping relationship between bitrate and QP, enabling the QP determined by scene complexity to be linked to the bitrate determined by motion, achieving continuous real-time dynamic adjustment of the bitrate and alleviating problems such as unstable bitrate control and image quality fluctuations. Third, through expert user ratings of each encoded image during the training of the first model, a mapping relationship between scene complexity and acceptable QP is established, which can maximize the human eye's subjective acceptability of image quality distortion and achieve a balance between image quality and bitrate bandwidth through flexible selection of target ratings. Fourth, both the second and third models are continuously trainable prediction models, and both can be continuously improved in accuracy through real-time optimization and fine-tuning during application, further ensuring the intelligence of the encoding process.

[0092] Corresponding to the encoding method provided above, the embodiment of the present application also provides an encoding device. Figure 2 As shown, the encoding device 200 includes:

[0093] Calculation module 201, used to calculate the scene complexity of the current frame to be encoded;

[0094] A first determination module 202 is configured to determine an acceptable QP for a current frame to be encoded based on scene complexity and a preset first model, wherein the first model describes a mapping relationship between scene complexity and acceptable QP;

[0095] A second determination module 203 determines a base bit rate of the current frame to be encoded based on the acceptable QP and a preset second model, wherein the second model describes a mapping relationship between bit rate and QP;

[0096] The third determination module 204 determines a target bit rate for the current frame to be encoded within a specified bit rate range, wherein the minimum bit rate within the specified bit rate range is the base bit rate, and the maximum bit rate within the specified bit rate range is the preset maximum bit rate;

[0097] The encoding module 205 is configured to encode the current frame to be encoded according to a target bit rate.

[0098] In some embodiments, the third determining module 204 includes:

[0099] A real-time motion region area determination unit, configured to determine the real-time motion region area of the current frame to be encoded;

[0100] The target bit rate calculation unit is used to calculate the target bit rate within a specified bit rate range according to the area of the real-time motion region.

[0101] In some embodiments, the real-time motion region area determination unit includes:

[0102] a frame difference information determination subunit, configured to determine frame difference information between a current frame to be encoded and a previous encoded frame;

[0103] The historical motion region area acquisition subunit is used to obtain the motion region area of N frames that have been encoded before the current frame to be encoded;

[0104] The real-time motion region area prediction subunit is used to predict the real-time motion region area of the current frame to be encoded based on the frame difference information, the motion region area and a preset third model, wherein the third model learns the change law of the motion region.

[0105] In some embodiments, the target bit rate calculation unit includes:

[0106] a ratio calculation subunit, configured to calculate the ratio of the real-time motion region area to the preset maximum motion region area;

[0107] The target bit rate calculation subunit is used to calculate the target bit rate according to the ratio, the basic bit rate and the maximum bit rate.

[0108] In some embodiments, the encoding module 205 includes:

[0109] a target QP determination unit, configured to determine a target QP of a current frame to be encoded based on a target bit rate and a second model;

[0110] a target frame size determining unit, configured to determine a target frame size of a current frame to be encoded according to a target bit rate;

[0111] The encoding unit is configured to send the target QP and the target frame size to the encoder to instruct the encoder to encode the current frame to be encoded.

[0112] In some embodiments, the encoding device 200 further includes a training module, which includes:

[0113] A data set construction unit, used for constructing an image data set to be encoded;

[0114] a training encoding unit, configured to encode each image to be encoded in the image data set according to at least two QPs, to obtain at least two encoded images corresponding to the image to be encoded;

[0115] A complexity calculation unit, used to calculate the scene complexity of each image to be encoded;

[0116] a score obtaining unit, configured to obtain a score of each encoded image;

[0117] An image screening unit is configured to screen each image to be encoded according to a preset target score and the score of each encoded image corresponding to the image to be encoded, to obtain a target encoded image corresponding to the image to be encoded;

[0118] The first model training unit is used to train the first model according to the QP corresponding to each target encoded image and the scene complexity of each image to be encoded.

[0119] In some embodiments, the image screening unit includes:

[0120] a score comparison subunit, configured to compare the score of each coded image corresponding to the image to be coded with a target score to determine a candidate coded image, wherein the score of the candidate coded image is not lower than the target score;

[0121] The target coded image determination subunit is configured to determine the candidate coded image with the lowest score as the target coded image.

[0122] As can be seen from the above, the following technical effects are achieved through the embodiments of the present application: First, the encoding method combines scene complexity entropy masking and the impact of motion on bitrate, maximizing bitrate bandwidth and storage space savings while ensuring subjective visual quality. Second, the second model describes the mapping relationship between bitrate and QP, enabling the QP determined by scene complexity to be linked to the bitrate determined by motion, achieving continuous real-time dynamic adjustment of the bitrate and alleviating problems such as unstable bitrate control and image quality fluctuations. Third, through expert user ratings of each encoded image during the training of the first model, a mapping relationship between scene complexity and acceptable QP is established, which can maximize the human eye's subjective acceptability of image quality distortion and achieve a balance between image quality and bitrate bandwidth through flexible selection of target ratings. Fourth, both the second and third models are continuously trainable prediction models, and both can be continuously improved in accuracy through real-time optimization and fine-tuning during application, further ensuring the intelligence of the encoding process.

[0123] Corresponding to the encoding method provided above, the embodiment of the present application further provides an electronic device. As an example only, the electronic device can be a smart phone, a tablet computer, a surveillance camera or other types of devices, which are not limited here. Figure 3 The electronic device 3 in the embodiment of the present application includes: a memory 301, one or more processors 302 ( Figure 3 Only one is shown) and a computer program stored in memory 301 and executable on the processor. Memory 301 is used to store software programs and units. Processor 302 executes the software programs and units stored in memory 301 to perform various functional applications and data processing to obtain resources corresponding to the above-mentioned preset events. Specifically, when processor 302 executes the above-mentioned computer program stored in memory 301, it implements the following steps:

[0124] Calculate the scene complexity of the current frame to be encoded;

[0125] Determining an acceptable QP for the current frame to be encoded based on the scene complexity and a preset first model, wherein the first model describes a mapping relationship between the scene complexity and the acceptable QP;

[0126] Determining a base bitrate for the current frame to be encoded based on the acceptable QP and a preset second model, wherein the second model describes a mapping relationship between bitrate and QP;

[0127] Determine the target bit rate of the current frame to be encoded within the specified bit rate range, wherein the minimum bit rate within the specified bit rate range is the base bit rate, and the maximum bit rate within the specified bit rate range is the preset maximum bit rate;

[0128] Encode the current frame to be encoded according to the target bit rate.

[0129] Assuming that the above is the first possible implementation, in a second possible implementation provided on the basis of the first possible implementation, determining the target bit rate of the current frame to be encoded within a specified bit rate range includes:

[0130] Determine the real-time motion area of the current frame to be encoded;

[0131] The target bit rate is calculated within the specified bit rate range based on the area of the real-time motion region.

[0132] In a third possible implementation provided on the basis of the second possible implementation, determining the area of the real-time motion region of the current frame to be encoded includes:

[0133] Determine the frame difference between the current frame to be encoded and the previous encoded frame;

[0134] Get the motion area of the N frames that have been encoded before the current frame to be encoded;

[0135] The real-time moving area of the current frame to be encoded is predicted based on the frame difference information, the moving area and a preset third model, wherein the third model learns the changing rules of the moving area.

[0136] In a fourth possible implementation provided on the basis of the second possible implementation, the target bit rate is calculated within a specified bit rate range according to the area of the real-time motion region, including:

[0137] Calculate the ratio of the real-time motion area to the preset maximum motion area;

[0138] The target bit rate is calculated based on the ratio, basic bit rate and maximum bit rate.

[0139] In a fifth possible implementation provided on the basis of the first possible implementation, encoding the current frame to be encoded according to the target bit rate includes:

[0140] Determine the target QP of the current frame to be encoded according to the target bit rate and the second model;

[0141] Determine the target frame size of the current frame to be encoded according to the target bit rate;

[0142] The target QP and target frame size are sent to the encoder to instruct the encoder to encode the current frame to be encoded.

[0143] In a sixth possible implementation provided based on the first possible implementation, or the second possible implementation, or the third possible implementation, or the fourth possible implementation, or the fifth possible implementation, the training process of the first model includes:

[0144] Construct an image dataset to be encoded;

[0145] For each image to be encoded in the image data set to be encoded, encoding the image to be encoded according to at least two QPs to obtain at least two encoded images corresponding to the image to be encoded;

[0146] Calculate the scene complexity of each image to be encoded;

[0147] Get the score of each encoded image;

[0148] For each image to be encoded, screening is performed according to a preset target score and the scores of each encoded image corresponding to the image to be encoded, to obtain a target encoded image corresponding to the image to be encoded;

[0149] The first model is trained according to the QP corresponding to each target encoded image and the scene complexity of each image to be encoded.

[0150] In a seventh possible implementation provided on the basis of the six possible implementations described above, screening is performed according to a preset target score and the score of each coded image corresponding to the image to be coded to obtain a target coded image corresponding to the image to be coded, including:

[0151] Comparing the score of each coded image corresponding to the image to be coded with the target score to determine a candidate coded image, wherein the score of the candidate coded image is not lower than the target score;

[0152] The candidate coded image with the lowest score is determined as the target coded image.

[0153] It should be understood that in the embodiment of the present application, the processor 302 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0154] The memory 301 may include a read-only memory and a random access memory, and provides instructions and data to the processor 302. A portion or all of the memory 301 may also include a non-volatile random access memory. For example, the memory 301 may also store device category information.

[0155] As can be seen from the above, the following technical effects are achieved through the embodiments of the present application: First, the encoding method combines scene complexity entropy masking and the impact of motion on bitrate, maximizing bitrate bandwidth and storage space savings while ensuring subjective visual quality. Second, the second model describes the mapping relationship between bitrate and QP, enabling the QP determined by scene complexity to be linked to the bitrate determined by motion, achieving continuous real-time dynamic adjustment of the bitrate and alleviating problems such as unstable bitrate control and image quality fluctuations. Third, through expert user ratings of each encoded image during the training of the first model, a mapping relationship between scene complexity and acceptable QP is established, which can maximize the human eye's subjective acceptability of image quality distortion and achieve a balance between image quality and bitrate bandwidth through flexible selection of target ratings. Fourth, both the second and third models are continuously trainable prediction models, and both can be continuously improved in accuracy through real-time optimization and fine-tuning during application, further ensuring the intelligence of the encoding process.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0157] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0158] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of external device software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0159] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0160] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0161] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the associated hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program, when executed by the processor, can implement the steps of the above-mentioned various method embodiments. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The above-mentioned computer-readable storage medium may include: any entity or device that can carry the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer-readable memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable storage media does not include electrical carrier signals and telecommunication signals.

[0162] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A coding method, characterized in that: include: Calculate the scene complexity of the current frame to be encoded; Determining an acceptable quantization parameter QP of the current frame to be encoded according to the scene complexity and a preset first model, wherein the first model describes a mapping relationship between scene complexity and acceptable QP; Determining a base bit rate of the current frame to be encoded according to the acceptable QP and a preset second model, wherein the second model describes a mapping relationship between bit rate and QP; Determining a target bit rate for the current frame to be encoded within a specified bit rate range, wherein a minimum bit rate within the specified bit rate range is the base bit rate, and a maximum bit rate within the specified bit rate range is a preset maximum bit rate; Encoding the current frame to be encoded according to the target bit rate; Determining the target bit rate of the current frame to be encoded within the specified bit rate range includes: Determining frame difference information between the current frame to be encoded and the previous encoded frame; Obtaining the moving area of N frames of encoded frames before the current frame to be encoded; Predicting the real-time moving area of the current frame to be encoded based on the frame difference information, the moving area, and a preset third model, wherein the third model learns a changing pattern of the moving area; The target bit rate is calculated within the specified bit rate range according to the area of the real-time motion region.

2. The encoding method according to claim 1, wherein The calculating the target bit rate within the specified bit rate range according to the area of the real-time motion region includes: Calculating the ratio of the real-time motion region area to the preset maximum motion region area; The target bit rate is calculated according to the ratio, the basic bit rate and the maximum bit rate.

3. The encoding method according to claim 1, wherein The encoding of the current frame to be encoded according to the target bit rate includes: Determining a target QP for the current frame to be encoded according to the target bit rate and the second model; Determining a target frame size of the current frame to be encoded according to the target bit rate; The target QP and the target frame size are sent to an encoder to instruct the encoder to encode the current frame to be encoded.

4. The encoding method according to any one of claims 1 to 3, characterized in that The training process of the first model includes: Construct an image dataset to be encoded; For each image to be encoded in the image data set to be encoded, encoding the image to be encoded according to at least two QPs respectively to obtain at least two encoded images corresponding to the image to be encoded; Calculating the scene complexity of each of the images to be encoded; Obtaining a score for each of the encoded images; For each of the to-be-encoded images, screening is performed according to a preset target score and the score of each coded image corresponding to the to-be-encoded image to obtain a target coded image corresponding to the to-be-encoded image; The first model is trained according to the QP corresponding to each target encoded image and the scene complexity of each image to be encoded.

5. The encoding method according to claim 4, wherein: The filtering according to the preset target score and the score of each coded image corresponding to the image to be coded to obtain the target coded image corresponding to the image to be coded includes: comparing the score of each coded image corresponding to the image to be coded with the target score to determine a candidate coded image, wherein the score of the candidate coded image is not lower than the target score; The candidate coded image with the lowest score is determined as the target coded image.

6. A coding device, characterized in that include: A calculation module, used to calculate the scene complexity of the current frame to be encoded; a first determining module, configured to determine an acceptable quantization parameter QP of the current frame to be encoded based on the scene complexity and a preset first model, wherein the first model describes a mapping relationship between scene complexity and an acceptable QP; a second determining module, configured to determine a base bit rate of the current frame to be encoded based on the acceptable QP and a preset second model, wherein the second model describes a mapping relationship between bit rate and QP; a third determining module, configured to determine a target bit rate for the current frame to be encoded within a specified bit rate range, wherein a minimum bit rate within the specified bit rate range is the base bit rate, and a maximum bit rate within the specified bit rate range is a preset maximum bit rate; An encoding module, configured to encode the current frame to be encoded according to the target bit rate; Wherein, the third determining module includes: A real-time motion region area determination unit, configured to determine the real-time motion region area of the current frame to be encoded; A target bit rate calculation unit, configured to calculate the target bit rate within the specified bit rate range according to the area of the real-time motion region; The real-time motion region area determination unit includes: a frame difference information determining subunit, configured to determine frame difference information between the current frame to be encoded and the previous encoded frame; A historical motion region area acquisition subunit is used to acquire the motion region areas of N frames that have been encoded before the current frame to be encoded; The real-time motion region area prediction subunit is used to predict the real-time motion region area of the current frame to be encoded based on the frame difference information, the motion region area and a preset third model, wherein the third model learns the change law of the motion region.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images