Image coding method and related device

Abstract

The embodiment of the invention discloses an image coding method and a related device, and the method comprises the steps: obtaining coding period reference information which comprises at least one of reference network information and reference picture information; according to the coding period reference information, determining a target interval corresponding to a target period IDR frame to be coded, the target interval being an interval which should be satisfied between the target period IDR frame and a previous IDR frame; and when the interval between the IDR frame and the previous IDR frame reaches the target interval, encoding the target period IDR frame, and inserting the target period IDR frame into an image frame sequence transmitted to a video decoding end. According to the method, the IDR frame can be flexibly inserted into the image frame sequence, and the video playing quality of a video decoding end is prevented from being influenced by insertion of the IDR frame.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image encoding method and related devices. Background Art

[0002] I frame, also known as intra-coded frame, is an independent frame with all its own image information, which can be decoded independently without referring to other image information. Instantaneous Decoding Refresh (IDR) frame is a special I frame, which is used to trigger an immediate refresh so that errors do not continue to propagate, that is, a new image frame sequence is recalculated from the IDR frame for encoding and decoding.

[0003] For an ordinary I frame, the image frames that follow it (such as P frame and B frame) can reference the image frame information before the I frame as a motion reference; while for an IDR frame, the image frames that follow it cannot reference the image frame information before the IDR frame as a motion reference.

[0004] In the related art, IDR frames are generally inserted into the encoded image frame sequence at a fixed frame interval, such as inserting an IDR frame every 2000 encoded frames. However, this IDR frame insertion method is too rigid. In some cases (such as poor network environment), the video playback quality at the video decoding end may be affected by the insertion of IDR frames, resulting in blurred and flickering video images. Summary of the invention

[0005] The embodiments of the present application provide an image encoding method and related devices, which can flexibly insert IDR frames into an image frame sequence to avoid affecting the video playback quality of a video decoding end due to the insertion of IDR frames.

[0006] In view of this, the first aspect of the present application provides an image encoding method, the method comprising:

[0007] Acquire coding cycle reference information; the coding cycle reference information includes at least one of reference network information and reference picture information, the reference network information is used to characterize the network transmission condition from the video encoding end to the video decoding end within the reference time period, the reference picture information is used to characterize the picture complexity of the image frame encoded within the reference time period, and the reference time period is the time period between the encoding time of the previous IDR frame and the current time;

[0008] Determine, according to the coding cycle reference information, a target interval corresponding to the target cycle IDR frame to be encoded; the target interval is an interval that should be satisfied between the target cycle IDR frame and the previous IDR frame;

[0009] When the interval between the previous IDR frame and the previous IDR frame reaches the target interval, the target period IDR frame is encoded and inserted into the image frame sequence transmitted to the video decoding end.

[0010] A second aspect of the present application provides an image encoding device, the device comprising:

[0011] An information acquisition module, used to acquire coding cycle reference information; the coding cycle reference information includes at least one of reference network information and reference picture information, the reference network information is used to characterize the network transmission condition from the video encoding end to the video decoding end within the reference time period, and the reference picture information is used to characterize the picture complexity of the image frame encoded within the reference time period, and the reference time period is the time period between the encoding time of the previous IDR frame and the current time;

[0012] An interval determination module, used to determine a target interval corresponding to a target period IDR frame to be encoded according to the coding period reference information; the target interval is an interval that should be satisfied between the target period IDR frame and the previous IDR frame;

[0013] The encoding module is used to encode the target period IDR frame when the interval between the previous IDR frame and the previous IDR frame reaches the target interval, and insert the target period IDR frame into the image frame sequence transmitted to the video decoding end.

[0014] A third aspect of the present application provides a computer device, the device comprising a processor and a memory:

[0015] The memory is used to store computer programs;

[0016] The processor is used to execute the steps of the image encoding method as described in the first aspect according to the computer program.

[0017] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the steps of the image encoding method described in the first aspect above.

[0018] In a fifth aspect, the present application provides a computer program product or a computer program, the computer program product or the computer program includes a computer instruction, the computer instruction is stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device performs the steps of the image encoding method described in the first aspect above.

[0019] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0020] An embodiment of the present application provides an image encoding method, in which encoding period reference information is first obtained, and the encoding period reference information includes at least one of reference network information and reference picture information, wherein the reference network information is used to characterize the network transmission conditions from the video encoding end to the video decoding end within the reference time period, and the reference picture information is used to characterize the picture complexity of the image frame encoded within the reference time period, and the reference time period here is the time period between the encoding time of the previous IDR frame and the current time; then, according to the encoding period reference information, the target interval corresponding to the target period IDR frame to be encoded is determined, and the target interval is the interval that should be satisfied between the target period IDR frame and the previous IDR frame; when the interval between the target period IDR frame and the previous IDR frame reaches the target interval, the target period IDR frame is encoded and inserted into the image frame sequence transmitted to the video decoding end. In this way, referring to the network transmission conditions from the video encoding end to the video decoding end, and at least one of the information of the picture complexity of the encoded image frame, the target interval that should be met between the current target period IDR frame to be encoded and the previous IDR frame is flexibly and adaptively determined, and when the target interval is reached between the previous IDR frame and the target period IDR frame, the target period IDR frame is encoded and inserted into the image frame sequence; compared with the method of inserting IDR frames into the image frame sequence according to a fixed frame interval in the related art, the embodiment of the present application can flexibly control the insertion interval of the IDR frame, so that the insertion interval of the IDR frame is adapted to the current network transmission conditions and / or the picture complexity of the currently encoded image frame, thereby avoiding the video playback quality of the video decoding end being affected by the insertion of the IDR frame, such as avoiding frequent insertion of IDR frames when the network transmission conditions are poor or the picture complexity of the encoded image frame is high, and accordingly, avoiding the situation of network freeze and the situation of blurred and flickering video played at the video decoding end, thereby ensuring the video playback quality of the video decoding end. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of an application scenario of the image encoding method provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a flow chart of an image encoding method provided in an embodiment of the present application;

[0023] Figure 3 A refresh cycle in an exemplary intra-frame refresh technology provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a local area that customizes the use of an intra-frame prediction mode provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of a flow chart of another image encoding method provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of the structure of an image encoding device provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] The image encoding method provided in the embodiment of the present application can be executed by a computer device, which can be a terminal device or a server. The terminal device includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, an intelligent home appliance, a vehicle terminal, an aircraft, etc. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server.

[0032] It should be noted that the information (including but not limited to network information, image information, etc.), data (including but not limited to network data, image data, etc.), and signals involved in the embodiments of the present application are all authorized by the relevant objects or fully authorized by all parties, and the collection, use and processing of relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0033] To facilitate understanding of the image encoding method provided in the embodiment of the present application, the following uses the execution subject of the image encoding method as a server as an example to exemplarily introduce the application scenario of the image encoding method.

[0034] See also Figure 1 , Figure 1 Schematic diagram of application scenarios of the image encoding method provided in the embodiment of the present application. Figure 1 As shown, the application scenario includes a server 110 and a terminal device 120, and the server 110 and the terminal device 120 can communicate with each other through a network. The server 110 is used as a video encoding end to execute the image encoding method provided in the embodiment of the present application, and encode the image frame sequence transmitted to the video decoding end; the terminal device 120 is used as a video decoding end to receive the image frame sequence encoded by the video encoding end, and decode and play the image frames therein to realize the playback of the video stream.

[0035] In practical applications, the server 110 can obtain coding cycle reference information during the process of encoding the image frame sequence, and the coding cycle reference information includes at least one of reference network information and reference picture information. The reference network information is used to characterize the network transmission conditions from the video encoding end to the video decoding end within the reference period, that is, the network transmission conditions from the server 110 to the terminal device 120; the reference picture information is used to characterize the picture complexity of the image frame encoded by the server 110 within the reference period; the reference period refers to the time period between the encoding time of the previous IDR frame in the image frame sequence and the current time.

[0036] Then, the server 110 can determine the target interval corresponding to the target periodic IDR frame to be encoded currently based on the above-mentioned coding period reference information. The target periodic IDR frame is the periodic IDR frame to be encoded currently. The periodic IDR frame refers to an IDR frame inserted into the encoded image frame sequence at a specific periodic interval. In an embodiment of the present application, the periodic interval based on which such periodic IDR frames are inserted can be determined based on the above-mentioned coding period reference information; usually, such periodic IDR frames are inserted to avoid the continued propagation of errors caused by unknown problems at the video encoding end or the video decoding end. The target interval refers to the interval that should be satisfied between the target periodic IDR frame and the previous IDR frame. The target interval can specifically be a time interval or a frame number interval.

[0037] When the interval between the last IDR frame and the target interval is reached, the server 110 may encode the target period IDR frame and insert the target period IDR frame into the image frame sequence transmitted to the terminal device 120. Accordingly, after receiving the target period IDR frame, the terminal device 120 may clear the current waiting queue of frames to be decoded, that is, clear the image frames that have been received but not yet decoded, and directly start decoding from the target period IDR frame to play the corresponding video picture.

[0038] In this way, the server 110 refers to the network transmission conditions from itself to the terminal device 120, and at least one of the information of the picture complexity of the image frame it encodes, and flexibly and adaptively determines the target interval that should be met between the current target period IDR frame to be encoded and the previous IDR frame, and when the target interval is reached between the previous IDR frame, encodes the target period IDR frame and inserts it into the image frame sequence; in this way, the insertion interval of the IDR frame is flexibly controlled so that the insertion interval of the IDR frame is adapted to the current network transmission conditions and / or the picture complexity of the currently encoded image frame, thereby avoiding the video playback quality of the video decoding end being affected by the insertion of IDR frames, such as avoiding frequent insertion of IDR frames when the network transmission conditions are poor or the picture complexity of the encoded image frame is high, and accordingly, avoiding the situation where the network is stuck and the video picture played by the terminal device 120 is blurred and flickering, thereby ensuring the video playback quality of the terminal device 120.

[0039] It should be understood that Figure 1 The application scenarios shown are only examples. In actual applications, the image encoding method provided in the embodiments of the present application can also be applied to other scenarios, such as cloud gaming scenarios, live broadcast scenarios, etc. No limitation is made to the application scenarios of the image encoding method provided in the embodiments of the present application.

[0040] The image encoding method provided by the present application is described in detail below through a method embodiment.

[0041] See also Figure 2 , Figure 2 The following is a flow chart of the image encoding method provided in the embodiment of the present application. For the convenience of description, the following embodiment is still introduced by taking the execution subject of the image encoding method as a server as an example. Figure 2 As shown, the image encoding method comprises the following steps:

[0042] Step 201: Obtain coding cycle reference information; the coding cycle reference information includes at least one of reference network information and reference picture information, the reference network information is used to characterize the network transmission conditions from the video encoding end to the video decoding end within a reference time period, and the reference picture information is used to characterize the picture complexity of the image frame encoded within the reference time period, and the reference time period is the time period between the encoding time of the previous IDR frame and the current time.

[0043] In practical applications, the server needs to apply for IDR frames regularly to reduce the risk of errors that continue to propagate. The errors here are usually caused by unknown problems at the video encoding or decoding end. Specifically, if some unknown and imperceptible problems occur during the video encoding or decoding process (such problems are more likely to occur in hardware encoding and decoding scenarios), and there are no IDR frames in the image frame sequence, the errors caused by the above problems will continue to affect the encoding and decoding of subsequent image frames, and in severe cases, the video screen may even be black, green, distorted, or stuck; at this time, if an IDR frame is applied, the IDR frame can be used to complete intra-frame encoding for all pixels in the entire image. Intra-frame encoding can eliminate previously existing errors and stop the error from continuing to propagate. Based on this, IDR frames can be regularly inserted into the encoded image frame sequence to eliminate possible errors in a timely manner through regularly inserted IDR frames to prevent errors from continuing to propagate.

[0044] In the related art, the server usually inserts IDR frames at a fixed frame interval. This method of inserting IDR frames is too rigid. In some cases, the insertion of IDR frames may affect the video playback quality at the video decoding end. For example, in a poor network environment, inserting IDR frames may cause the normally played video screen to suddenly become blurred and flickering, which is particularly obvious in still picture scenes; in addition, since the files of IDR frames are usually larger (several times that of P frames), inserting IDR frames can easily cause network jams. In order to solve the above problems, the embodiments of the present application will adapt to the network conditions and / or the picture complexity of the image frame, and adaptively and flexibly determine the insertion interval of the IDR frame.

[0045] In an embodiment of the present application, after encoding each IDR frame, the server can obtain the encoding period reference information corresponding to the next periodic IDR frame to be encoded. The encoding period reference information is used to determine the interval that should be satisfied between the current periodic IDR frame to be encoded and the IDR frame just encoded.

[0046] It should be noted that the obtained coding period reference information may include at least one of reference network information and reference picture information. Among them, the reference network information is information that can characterize the network transmission conditions from the video encoding end to the video decoding end within the reference time period, such as the network bandwidth from the video encoding end to the video decoding end within the reference time period, etc. The reference picture information is information that can characterize the picture complexity of the image frame encoded within the reference time period; exemplarily, the server can obtain each image frame encoded within the reference time period, and analyze the picture complexity (such as motion complexity and texture complexity, etc.) of each image frame through a related picture complexity analysis algorithm, and then determine the reference picture information according to the picture complexity of each image frame. The above-mentioned reference time period is the time period between the encoding time of the previous IDR frame and the current time, that is, the time period between the encoding time of the IDR frame just inserted into the image frame sequence transmitted to the video decoding end and the current time.

[0047] Step 202: Determine a target interval corresponding to a target period IDR frame to be encoded according to the coding period reference information; the target interval is an interval that should be satisfied between the target period IDR frame and the previous IDR frame.

[0048] After the server obtains the coding period reference information, it can determine the target interval corresponding to the target period IDR frame to be encoded currently according to the obtained coding period reference information.

[0049] It should be noted that the target periodic IDR frame refers to the periodic IDR frame to be encoded currently. The periodic IDR frame refers to the IDR frame that needs to be applied for regularly and inserted into the image frame sequence transmitted to the video decoding end. As described above, the purpose of inserting such periodic IDR frames is to avoid the continuous propagation of errors caused by unknown problems at the video encoding end or the video decoding end. The target interval corresponding to the target periodic IDR frame refers to the interval that should be satisfied between the target periodic IDR frame and the previous IDR frame. The target interval can be a time interval (such as 30s, 50s, etc.) or a frame number interval (such as 2000 frames, 5000 frames, etc.).

[0050] Exemplarily, in the case where the coding period reference information includes reference network information, the server can determine the target interval corresponding to the target period IDR frame according to the network transmission conditions represented by the reference network information; it should be understood that in order to ensure the smoothness of the video picture played by the video decoding end, the worse the network transmission conditions are, the larger the determined target interval is, and the better the network transmission conditions are, the smaller the determined target interval is. In the case where the coding period reference information includes reference picture information, the server can determine the target interval corresponding to the target period IDR frame according to the picture complexity represented by the reference picture information; similarly, in order to ensure the smoothness of the video picture played by the video decoding end, the higher the picture complexity is, the larger the determined target interval is, and the lower the picture complexity is, the smaller the determined target interval is.

[0051] Step 203: When the interval between the previous IDR frame and the previous IDR frame reaches the target interval, encode the target period IDR frame, and insert the target period IDR frame into the image frame sequence transmitted to the video decoding end.

[0052] When it is determined that the interval between the current and previous IDR frames reaches the target interval, the server may encode a target period IDR frame, and insert the encoded target period IDR frame into the image frame sequence transmitted to the video decoding end.

[0053] Exemplarily, if the target interval is a time interval, it can be determined whether the interval between the current moment and the encoding moment of the previous IDR frame reaches the time interval, and if it has reached it, the server can encode the target period IDR frame. If the target interval is a frame number interval, it can be determined whether the difference between the number of the current image frame to be encoded and the number of the previous IDR frame reaches the frame number interval, and if it has reached it, the server can encode the target period IDR frame.

[0054] The image encoding method provided in the embodiment of the present application can refer to the network transmission conditions from the video encoding end to the video decoding end, and at least one of the information of the picture complexity of the encoded image frame, flexibly and adaptively determine the target interval that should be met between the current target period IDR frame to be encoded and the previous IDR frame, and when the target interval is reached between the previous IDR frame, encode the target period IDR frame and insert it into the image frame sequence; compared with the method of inserting IDR frames into the image frame sequence according to a fixed frame interval in the related art, the embodiment of the present application can flexibly control the insertion interval of the IDR frame, so that the insertion interval of the IDR frame is adapted to the current network transmission conditions and / or the picture complexity of the currently encoded image frame, thereby avoiding the video playback quality of the video decoding end being affected by the insertion of IDR frames, such as avoiding frequent insertion of IDR frames when the network transmission conditions are poor or the picture complexity of the encoded image frame is high, and accordingly, avoiding the situation of network freeze and the situation of blurred and flickering video played at the video decoding end, thereby ensuring the video playback quality of the video decoding end.

[0055] In one possible implementation, when the server receives an IDR frame application request, the server can respond to the IDR frame application request, encode the target temporary IDR frame, and insert the target temporary IDR frame into the image frame sequence; if the IDR frame application request is not revoked, or if the IDR frame application request is received again, when the interval between the encoded target temporary IDR frame reaches a preset interval, the server encodes the next target temporary IDR frame and inserts the next target temporary IDR frame into the image frame sequence.

[0056] In practical applications, it is often necessary to temporarily apply for IDR frames to solve current problems. For example, in the case of network freezes or problems with the decoding speed of the video decoding end, it is often necessary to apply for IDR frames to solve the problem.

[0057] Specifically, when network jam occurs, a large number of image frames will fail to be transmitted due to network reasons. If all these image frames are retransmitted, the network jam will be aggravated. At this time, if an IDR frame is applied, all the image frames that failed to be transmitted before can be discarded, and only the IDR frame will be transmitted. Accordingly, the subsequent image frames will no longer consider the image frames before the IDR frame, and the video screen can be decoded and displayed normally based on the IDR frame, reducing the network transmission pressure. In addition, when there is a problem with the decoding speed of the video decoding end, such as when the decoding speed of the video decoding end cannot keep up with the encoding speed of the video encoding end, more and more image frames will be accumulated in the decoding waiting queue of the video decoding end, affecting the decoding ability of the video decoding end. At this time, if an IDR frame is applied, the decoding waiting queue of the video decoding end can be cleared, and decoding can be started from the IDR frame. The decoding of subsequent image frames no longer refers to the previous image frames, and the video screen can be decoded and displayed normally, reducing the decoding pressure of the video decoding end.

[0058] When the above situation occurs, the upper layer (such as the control layer of the video codec) can send an IDR frame application request to the server to request the server to encode the target temporary IDR frame and solve the related problems through the target temporary IDR frame. Accordingly, the server can encode and generate the target temporary IDR frame and insert it into the image frame sequence transmitted to the video decoding end.

[0059] It should be understood that the IDR frame application request here is a request for temporarily applying for an IDR frame. Normally, the upper layer can generate the IDR frame application request and send it to the server as the video encoding end when it detects that a situation needs to apply for an IDR frame (such as network freezes, or problems with the decoding speed of the video decoding end, etc.). The target temporary IDR frame is the temporary IDR frame to be encoded at present. The temporary IDR frame refers to an IDR frame that is temporarily applied in response to an IDR frame application request. As described above, the purpose of inserting such a temporary IDR frame is to solve related temporary problems, such as network freezes, or problems with the decoding speed of the video decoding end, etc.

[0060] In actual applications, the target temporary IDR frame encoded by the server may not be able to take effect on the problem to be solved in time. For example, due to network jams and other reasons, the target temporary IDR frame cannot reach the video decoding end in time, and thus cannot solve the problem existing in the video encoding end in time. In response to this situation, the upper layer may think that the IDR frame application request it sent previously has not been responded to, so it does not cancel the IDR frame application request, or sends the IDR frame application request to the server again; at this time, if the server responds to the IDR frame application request and continuously applies for multiple target temporary IDR frames in a short period of time, it is actually not friendly to its encoding efficiency and the actual problem to be solved, that is, continuously encoding multiple IDR frames will have a greater impact on the encoding efficiency of the server, reducing the encoding efficiency of the server, and the file size of the IDR frame is generally large, and the continuous transmission of multiple IDR frames is likely to cause or aggravate network jams.

[0061] In view of the above situation, the embodiment of the present application further sets a preset interval t0 that should be satisfied between adjacent temporary IDR frames, and the preset interval t0 can be a time interval (such as 100ms) or a frame number interval (such as 5 frames). When the IDR frame application request sent by the upper layer is not revoked, or the IDR frame application request is received again, the server can detect whether the interval between the current and the just generated target temporary IDR frame reaches the above preset interval t0; if not, the next target temporary IDR frame cannot be encoded; if it is reached, and the upper layer's request for applying for an IDR frame has not been responded to, the next target temporary IDR frame can be encoded and inserted into the image frame sequence.

[0062] In this way, by limiting the preset interval between adjacent temporary IDR frames, the server can be effectively prevented from continuously applying for multiple temporary IDR frames in a short period of time, thereby avoiding problems caused by continuously applying for multiple temporary IDR frames, such as reducing the encoding efficiency of the video encoding end, causing or aggravating network freezes, etc., and ensuring that the applied temporary IDR frames can reliably play their role.

[0063] In a possible implementation, if the server receives an IDR frame application request before the interval between the previous IDR frame and the previous IDR frame reaches the target interval, the server responds to the IDR frame application request, encodes the target temporary IDR frame, inserts the target temporary IDR frame into the image frame sequence, and abandons encoding the target period IDR frame. Further, the target temporary IDR frame is used as the new previous IDR frame, and based on the encoding period reference information after the encoding time of the new previous IDR frame, the target interval corresponding to the next target period IDR frame to be encoded is determined.

[0064] Specifically, in some cases, the server may receive an IDR frame application request in the target interval corresponding to the target periodic IDR frame to be encoded. At this time, the server will respond to the IDR frame application request, encode the target temporary IDR frame, and insert it into the image frame sequence transmitted to the video decoding end; because the temporary IDR frame can also play a role in controlling the continued propagation of errors, that is, the temporary IDR frame can also control the continued propagation of errors caused by unknown problems at the video encoding end or the video decoding end, therefore, the encoding of the target periodic IDR frame can be abandoned, and the target temporary IDR frame can be used to replace the target periodic IDR frame to achieve the corresponding purpose.

[0065] Accordingly, the server can use the generated target temporary IDR frame as the new previous IDR frame, and determine the target interval corresponding to the next target period IDR frame to be encoded based on the coding period reference information after the target temporary IDR frame (such as the network transmission conditions from the video encoding end to the video decoding end after the target temporary IDR frame, and the picture complexity of the image frame encoded after the target temporary IDR frame); and when it is determined that the target interval is reached between the target temporary IDR frame (no IDR frame application request is received during this period), encode the next IDR frame application request and insert it into the image frame sequence transmitted to the video decoding end.

[0066] It should be understood that when the IDR frame application request received by the server is not revoked, or when the server continues to receive IDR frame application requests, the server will generate multiple target temporary IDR frames at the above-mentioned preset intervals. In this case, the server can use the last target temporary IDR frame generated (i.e., the last temporary IDR frame generated before the IDR frame application request is revoked, or no IDR frame application request is received after the temporary IDR frame is generated) as the above-mentioned new previous IDR frame, so as to determine the target interval corresponding to the next target period IDR frame accordingly.

[0067] In this way, the encoded target temporary IDR frame is used to replace the target periodic IDR frame that has not yet been encoded. Based on the target temporary IDR frame, the needs of related application IDR frames and the purpose of controlling error propagation can be simultaneously achieved, thereby reducing the IDR frames that need to be generated and saving resources consumed by encoding, transmitting and decoding IDR frames.

[0068] In a possible implementation, the coding period reference information may include reference network information and reference picture information at the same time. The reference network information may be a reference network bandwidth from a video encoding end to a video decoding end within a reference period, and the reference picture information may be a reference picture complexity determined according to the picture complexity of each image frame encoded within the reference period. Accordingly, the server may determine the target interval corresponding to the target period IDR frame to be encoded according to the coding period reference information in the following manner:

[0069] When the reference network bandwidth belongs to the low bandwidth interval, or the reference picture complexity belongs to the high complexity level, the target interval is determined to be the first interval; when the reference network bandwidth belongs to the high bandwidth interval and the reference picture complexity belongs to the low complexity level, the target interval is determined to be the second interval; the second interval is smaller than the first interval.

[0070] Exemplarily, the server may pre-set a bandwidth threshold for distinguishing between a high bandwidth interval and a low bandwidth interval; for example, the bandwidth threshold is set to 15M, a bandwidth below 15M will be considered to belong to the low bandwidth interval, and a bandwidth above or equal to 15M will be considered to belong to the high bandwidth interval.

[0071] In addition, the server may also pre-set the complexity level based on the adopted picture complexity algorithm. For example, when the adopted picture complexity algorithm is used to classify the picture complexity of the image frame (such as classified into high complexity, medium complexity, and low complexity), the server may use the picture complexity algorithm to determine the picture complexity category to which each image frame in the reference time period belongs, and then take the picture complexity category with the most included image frames as the reference picture complexity; when the reference picture complexity is a high complexity category, it can be regarded as belonging to a high complexity level, and when the reference picture complexity belongs to a medium complexity category or a low complexity category, it can be regarded as belonging to a low complexity level. For another example, when the adopted picture complexity algorithm is used to calculate the picture complexity value of an image frame, the server can use the picture complexity algorithm to calculate the picture complexity value corresponding to each image frame in a reference time period, and then calculate the average of the picture complexity values ​​corresponding to each image frame as the reference picture complexity; when the reference picture complexity belongs to the complexity value interval corresponding to the high complexity level, it is determined that the reference picture complexity belongs to the high complexity level; when the reference picture complexity does not belong to the complexity value interval corresponding to the high complexity level, it is determined that the reference picture complexity belongs to the low complexity level.

[0072] When the server determines that the reference network bandwidth belongs to the low bandwidth interval, or the reference picture complexity belongs to the high complexity level according to the above-mentioned measurement criteria, it can be determined that the target interval that should be satisfied between the target period IDR frame and the previous IDR frame is the first interval t1; conversely, when the server determines that the reference network bandwidth belongs to the high bandwidth interval, and the reference picture complexity belongs to the low complexity level according to the above-mentioned measurement criteria, it can be determined that the target interval that should be satisfied between the target period IDR frame and the previous IDR frame is the second interval t2.

[0073] It should be noted that the above-mentioned first interval t1 is greater than the second interval t2; when the first interval t1 and the second interval t2 are both time intervals, the first interval t1 can be, for example, 60s, and the second interval t2 can be, for example, 30s; when the first interval t1 and the second interval t2 are both frame number intervals, the first interval t1 can be, for example, 5000 frames, and the second interval t2 can be, for example, 2000 frames.

[0074] In this way, the reference network bandwidth and the reference picture complexity are comprehensively considered in the above manner to determine the target interval that should be met between the target periodic IDR frame and the previous IDR frame. When the network bandwidth is low or the picture complexity is high, periodic IDR frames can be inserted based on a larger interval, thereby avoiding network freezes or blurred video images due to the insertion of IDR frames. When the network bandwidth is high and the picture complexity is low, periodic IDR frames can be inserted based on a smaller interval, so as to resolve related errors more promptly and ensure that no negative impacts such as network freezes or blurred video images are caused.

[0075] In a possible implementation, the coding period reference information may only include reference network information, which may be a reference network bandwidth from the video encoding end to the video decoding end within the reference period. Accordingly, the server may determine the target interval corresponding to the target period IDR frame to be encoded according to the coding period reference information in the following manner:

[0076] Among multiple first candidate bandwidth intervals, the first candidate bandwidth interval to which the reference network bandwidth belongs is determined as the first target bandwidth interval, and then the IDR frame coding interval corresponding to the first target bandwidth interval is determined as the target interval. Among them, there is no overlapping area among the multiple first candidate bandwidth intervals, and the multiple first candidate bandwidth intervals correspond to different IDR frame coding intervals, respectively, and the bandwidth value in the first candidate bandwidth interval is negatively correlated with the IDR frame coding interval corresponding to the first candidate bandwidth interval.

[0077] Exemplarily, the server may pre-set multiple first candidate bandwidth intervals, which are candidate bandwidth intervals used to determine the target interval between the target period IDR frame and the previous IDR frame. There is no overlapping area between these multiple first candidate bandwidth intervals. For example, assuming that a total of four first candidate bandwidth intervals are set, they are less than or equal to 5M, greater than 5M and less than or equal to 10M, greater than 10M and less than or equal to 15M, and greater than 15M.

[0078] Multiple first candidate bandwidth intervals correspond to different IDR frame coding intervals, and there is a negative correlation between the bandwidth value in the first candidate bandwidth interval and the IDR frame coding interval corresponding to the first candidate bandwidth, that is, the smaller the bandwidth value in the first candidate bandwidth interval, the larger the IDR frame coding interval corresponding to the first candidate bandwidth interval, and the larger the bandwidth value in the first candidate bandwidth interval, the smaller the IDR frame coding interval corresponding to the first candidate bandwidth interval. Still taking the above four first candidate bandwidth intervals as an example, the IDR frame coding interval corresponding to the first candidate bandwidth interval less than or equal to 5M can be 70s or 6000 frames, the IDR frame coding interval corresponding to the first candidate bandwidth interval greater than 5M and less than or equal to 10M can be 60s or 5000 frames, the IDR frame coding interval corresponding to the first candidate bandwidth interval greater than 10M and less than or equal to 15M can be 50s or 4000 frames, and the IDR frame coding interval corresponding to the first candidate bandwidth interval greater than 15M can be 40s or 3000 frames.

[0079] Assume that the reference network bandwidth obtained by the server is 13M, which belongs to the first candidate bandwidth interval greater than 10M and less than or equal to 15M. At this time, the first candidate bandwidth interval greater than 10M and less than or equal to 15M can be used as the first target bandwidth interval; then, determine the IDR frame interval of 50s or 4000 frames corresponding to the first target bandwidth interval, which is the target interval that should be met between the target period IDR frame and the previous IDR frame.

[0080] It should be understood that the above-mentioned first candidate bandwidth interval and its corresponding IDR frame interval are only examples. In actual applications, the server may pre-set more or fewer first candidate bandwidth intervals. The set first candidate bandwidth intervals may correspond to different bandwidth values, and the IDR frame coding interval corresponding to the set first candidate bandwidth interval may also be other frame intervals. The embodiments of the present application do not impose any limitations on this.

[0081] In this way, by considering the reference network bandwidth in the above manner to determine the target interval that should be met between the target periodic IDR frame and the previous IDR frame, when the network bandwidth is low, periodic IDR frames can be inserted based on a larger interval to avoid network freezes or blurred video images due to the insertion of IDR frames. When the network bandwidth is high, periodic IDR frames can be inserted based on a smaller interval to resolve related errors more promptly and ensure that no negative impacts such as network freezes or blurred video images are caused.

[0082] In a possible implementation, the coding period reference information may only include reference picture information, and the reference picture information may be a reference picture complexity determined according to the picture complexity of each image frame encoded in the reference period. Accordingly, the server may determine the target interval corresponding to the target period IDR frame to be encoded according to the coding period reference information in the following manner:

[0083] Among multiple first candidate complexity levels, the first candidate complexity level to which the reference picture complexity belongs is determined as the first target complexity level, and the IDR frame coding interval corresponding to the first target complexity level is determined as the target interval. Among them, multiple first candidate complexity levels correspond to different IDR frame coding intervals, respectively, and the picture complexity indicated by the first candidate complexity level is positively correlated with the IDR frame coding interval corresponding to the first candidate complexity level.

[0084] Exemplarily, the server may pre-set multiple first candidate complexity gears, and the first candidate complexity gears are candidate complexity gears used to decide the target interval that should be met between the target period IDR frame and the previous IDR frame. These multiple first candidate complexity gears indicate different picture complexities. For example, assuming that three first candidate complexity gears are set, respectively specifying high picture complexity, medium picture complexity and low picture complexity, the three picture complexity categories of high picture complexity, medium picture complexity and low picture complexity can be distinguished by the picture complexity classification results, and can also be distinguished by the complexity interval to which the picture complexity value belongs.

[0085] Multiple first candidate complexity levels correspond to different IDR frame coding intervals, respectively, and there is a positive correlation between the picture complexity indicated by the first candidate complexity level and the IDR frame coding interval corresponding to the first candidate complexity level, that is, the higher the picture complexity indicated by the first candidate complexity level, the larger the IDR frame coding interval corresponding to the first candidate complexity level, and the lower the picture complexity indicated by the first candidate complexity level, the smaller the IDR frame coding interval corresponding to the first candidate complexity level. Still taking the above three first candidate complexity levels as an example, the IDR frame coding interval corresponding to the first candidate complexity level indicating high picture complexity can be 60s or 5000 frames, the IDR frame coding interval corresponding to the first candidate complexity level indicating medium picture complexity can be 50s or 4000 frames, and the IDR frame coding interval corresponding to the first candidate complexity level indicating low picture complexity can be 40s or 3000 frames.

[0086] Assuming that the reference picture complexity obtained by the server is high picture complexity, which belongs to the first candidate complexity level indicating high picture complexity, the first candidate complexity level indicating high picture complexity can be used as the first target complexity level; then, the IDR frame encoding interval of 60s or 5000 frames corresponding to the first target complexity level is determined, which is the target interval that should be met between the target period IDR frame and the previous IDR frame.

[0087] It should be understood that the above-mentioned first candidate complexity level and its corresponding IDR frame interval are only examples. In actual applications, the server can pre-set more or fewer first candidate complexity levels. The set first candidate complexity levels can indicate different picture complexities, and the IDR frame encoding interval corresponding to the set first candidate complexity level can also be other frame intervals. The embodiments of the present application do not impose any limitations on this.

[0088] In this way, by considering the complexity of the reference picture in the above manner to determine the target interval that should be met between the target periodic IDR frame and the previous IDR frame, when the picture complexity is high, periodic IDR frames can be inserted based on a larger interval to avoid network freezes or blurred video images due to the insertion of IDR frames. When the picture complexity is low, periodic IDR frames can be inserted based on a smaller interval to resolve related errors more promptly and ensure that no negative impacts such as network freezes or blurred video images are caused.

[0089] In a possible implementation, the server may also use intra refresh technology to replace the above periodic IDR frames, that is, use intra refresh technology to replace the above target periodic IDR frames. When using intra refresh technology, multiple adjacent P frames are selected to form a group and used as a refresh period. Each P frame in the group of P frames has a part of the area, which is forced to use the intra prediction mode during encoding, and the other parts can use rate distortion optimization to select the prediction mode. Figure 3 is a refresh cycle in an exemplary intra-frame refresh technology, such as Figure 3 As shown, the refresh cycle includes four P frames, wherein the first vertical area 301 in the first P frame forces the use of intra-frame prediction mode, the second vertical area 302 in the second P frame forces the use of intra-frame prediction mode, the third vertical area 303 in the third P frame forces the use of intra-frame prediction mode, and the fourth vertical area 304 in the fourth P frame forces the use of intra-frame prediction mode. After completing the encoding of the four P frames in the refresh cycle, it is equivalent to completing the encoding refresh of the entire image.

[0090] In the case where the obtained coding period reference information includes both the reference network bandwidth and the reference picture complexity, the server can use the above-mentioned intra-frame refresh technology when it is determined that the reference network bandwidth belongs to the low bandwidth interval, or the reference picture complexity belongs to the high complexity gear. At this time, when it is determined that the interval between the previous IDR frame and the previous IDR frame reaches the target interval, the intra-frame refresh technology is used to encode n frames (n is an integer greater than 1) of local intra-frame refresh frames, and insert the n frames of local intra-frame refresh frames into the image frame sequence transmitted to the video decoding end.

[0091] In the case where the acquired coding period reference information only includes the reference network bandwidth, the server may use the above-mentioned intra-frame refresh technology when it is determined that the reference network bandwidth belongs to a low bandwidth interval (at least one first candidate bandwidth interval with a lower bandwidth value included in multiple first candidate bandwidth intervals). At this time, when it is determined that the interval between the previous IDR frame and the previous IDR frame reaches the target interval, the intra-frame refresh technology is used to encode n frames (n is an integer greater than 1) of local intra-frame refresh frames, and insert the n frames of local intra-frame refresh frames into the image frame sequence transmitted to the video decoding end.

[0092] In the case where the obtained coding period reference information only includes the reference picture complexity, the server can use the above-mentioned intra-frame refresh technology when it is determined that the reference picture complexity belongs to a high complexity level (at least one of the multiple first candidate complexity levels with a higher picture complexity). At this time, when it is determined that the interval between the previous IDR frame and the previous IDR frame reaches the target interval, the intra-frame refresh technology is used to encode n frames (n is an integer greater than 1) of local intra-frame refresh frames, and insert the n frames of local intra-frame refresh frames into the image frame sequence transmitted to the video decoding end.

[0093] It should be understood that the above n partial intra-frame refresh frames are the P frames in the refresh cycle of the intra-frame refresh technology, and each partial intra-frame refresh frame has a local area that requires the use of the intra-frame prediction mode, and the local areas that the n partial intra-frame refresh frames require to use the intra-frame prediction mode are different, and the local areas that the n partial intra-frame refresh frames require to use the intra-frame prediction mode can cover the complete image frame. After completing the encoding of these n partial intra-frame refresh frames, it is equivalent to completing the refresh of the entire image frame, that is, the n partial intra-frame refresh frames can play the same role as an IDR frame.

[0094] When the interval between the last IDR frame and the target interval is reached, the server can use the intra-frame refresh technology to encode n consecutive partial intra-frame refresh frames, and use these n partial intra-frame refresh frames to replace the target period IDR frames, so as to achieve the purpose of controlling the continued propagation of errors. It should be understood that the last IDR frame here can also be a multi-frame partial intra-frame refresh frame encoded using the intra-frame refresh technology, and the encoding time of the last partial intra-frame refresh frame can be used as the basis for determining whether the interval reaches the target interval.

[0095] In this way, when the network bandwidth is low or the picture complexity is high, the intra-frame refresh technology is used to replace the periodic IDR frame, which can avoid the network freeze or video blur caused by the large file size of the IDR frame, and ensure that there is no large fluctuation in the transmission process of the image frame. At the same time, the entire image frame can be refreshed through n consecutive local intra-frame refresh frames, achieving the purpose of controlling the continued propagation of related errors.

[0096] It should be noted that when the network bandwidth is high and the picture complexity is low, periodic IDR frames can be used directly to achieve the purpose of controlling the propagation of related errors. Since the network transmission conditions and picture complexity are both good at this time, there will be no network freezes or video blurs due to the large IDR frame files, and it is more efficient to directly use IDR frames to control error propagation. In addition, for the temporary IDR frames introduced above, since they need to completely discard the previous image frames, they are not suitable for replacement by intra-frame refresh technology.

[0097] In a possible implementation, when the intra-frame refresh technology is adopted, when the coding period reference information includes the reference network bandwidth, the number of local intra-frame refresh frames included in one refresh period (ie, n in the above text) can be determined by the following method:

[0098] Among multiple second candidate bandwidth intervals, the second candidate bandwidth interval to which the reference network bandwidth belongs is determined as the second target bandwidth interval, and the number of intra-frame refresh frames corresponding to the second target bandwidth interval is determined to be n. Among them, there is no overlapping area among the multiple second candidate bandwidth intervals, the multiple second candidate bandwidth intervals correspond to different numbers of intra-frame refresh frames, and the bandwidth value in the second candidate bandwidth interval is negatively correlated with the number of intra-frame refresh frames corresponding to the second candidate bandwidth interval.

[0099] Exemplarily, the server may pre-set multiple second candidate bandwidth intervals, which are candidate bandwidth intervals used to determine the number of local intra-frame refresh frames in a refresh cycle when using the intra-frame refresh technology. There is no overlapping area among these multiple second candidate bandwidth intervals. For example, assuming that three second candidate bandwidth intervals are set, they are less than or equal to 5M, greater than 5M and less than or equal to 10M, and greater than 10M and less than or equal to 15M.

[0100] Multiple second candidate bandwidth intervals correspond to different numbers of intra-frame refresh frames, and there is a negative correlation between the bandwidth value in the second candidate bandwidth interval and the number of intra-frame refresh frames corresponding to the second candidate bandwidth interval, that is, the smaller the bandwidth value in the second candidate bandwidth interval, the larger the number of intra-frame refresh frames corresponding to the second candidate bandwidth interval, and the larger the bandwidth value in the second candidate bandwidth interval, the smaller the number of intra-frame refresh frames corresponding to the second candidate bandwidth interval. Still taking the above three second candidate bandwidth intervals as an example, the number of intra-frame refresh frames corresponding to the second candidate bandwidth interval less than or equal to 5M can be 4, the number of intra-frame refresh frames corresponding to the second candidate bandwidth interval greater than 5M and less than or equal to 10M can be 3, and the number of intra-frame refresh frames corresponding to the second candidate bandwidth interval greater than 10M and less than or equal to 15M can be 2.

[0101] Assuming that the reference network bandwidth obtained by the server is 13M, the reference network bandwidth belongs to the above-mentioned second candidate bandwidth interval greater than 10M and less than or equal to 15M. At this time, the second candidate bandwidth interval greater than 10M and less than or equal to 15M can be used as the second target bandwidth interval; then, the intra-frame refresh frame number 2 corresponding to the second target bandwidth interval is determined, which is the number n of local intra-frame refresh frames included in a refresh cycle.

[0102] It should be understood that the above-mentioned second candidate bandwidth interval and its corresponding intra-frame refresh frame number are only examples. In actual applications, the server may pre-set more or fewer second candidate bandwidth intervals. The set second candidate bandwidth intervals may correspond to different bandwidth values, and the set second candidate bandwidth interval may correspond to other intra-frame refresh frame numbers. The embodiments of the present application do not impose any limitations on this.

[0103] In this way, by considering the reference network bandwidth in the above manner to determine the number n of local intra-frame refresh frames included in a refresh cycle, the refresh of the entire image frame can be completed based on more local intra-frame refresh frames when the network bandwidth is low, avoiding large fluctuations in the network due to the transmission of larger local intra-frame refresh frames, thereby ensuring the smoothness of video playback. When the network bandwidth is high, the refresh of the entire image frame can be completed based on fewer local intra-frame refresh frames, thereby completing the refresh of the entire image frame more efficiently and resolving related errors more promptly.

[0104] In a possible implementation, when the intra refresh technology is adopted, in the case where the coding period reference information includes the reference picture complexity, the number of partial intra refresh frames included in one refresh period (ie, n in the above text) can be determined by the following method:

[0105] Among multiple second candidate complexity levels, the second candidate complexity level to which the reference picture complexity belongs is determined as the second target complexity level, and the number of intra-frame refresh frames corresponding to the second target complexity level is determined to be n. Among them, multiple second candidate complexity levels correspond to different numbers of intra-frame refresh frames, respectively, and the picture complexity indicated by the second candidate complexity level is positively correlated with the number of intra-frame refresh frames corresponding to the second candidate complexity level.

[0106] Exemplarily, the server may pre-set multiple second candidate complexity levels, which are candidate complexity levels used to decide the number of local intra-frame refresh frames in a refresh cycle when using the intra-frame refresh technology. These multiple second candidate complexity levels indicate different picture complexities. For example, assuming that three second candidate complexity levels are set, respectively specifying high picture complexity, medium picture complexity and low picture complexity, the three picture complexity categories of high picture complexity, medium picture complexity and low picture complexity can be distinguished by the picture complexity classification results, or by the complexity interval to which the picture complexity value belongs.

[0107] Multiple second candidate complexity gears correspond to different intra-frame refresh frame numbers, and there is a positive correlation between the picture complexity indicated by the second candidate complexity gear and the intra-frame refresh frame number corresponding to the second candidate complexity gear, that is, the higher the picture complexity indicated by the second candidate complexity gear, the more intra-frame refresh frames the second candidate complexity gear corresponds to, and the lower the picture complexity indicated by the second candidate complexity gear, the fewer intra-frame refresh frames the first candidate complexity gear corresponds to. Still taking the above three second candidate complexity gears as an example, the intra-frame refresh frame number corresponding to the second candidate complexity gear indicating high picture complexity can be 4, the intra-frame refresh frame number corresponding to the second candidate complexity gear indicating medium picture complexity can be 3, and the intra-frame refresh frame number corresponding to the second candidate complexity gear indicating low picture complexity can be 2.

[0108] Assuming that the reference picture complexity obtained by the server is medium picture complexity, which belongs to the second candidate complexity level indicating medium picture complexity, the second candidate complexity level indicating medium picture complexity can be used as the second target complexity level at this time; then, the intra-frame refresh frame number 3 corresponding to the second target complexity level is determined, which is the number n of local intra-frame refresh frames included in a refresh cycle.

[0109] It should be understood that the above-mentioned second candidate complexity level and its corresponding intra-frame refresh frame number are only examples. In actual applications, the server may pre-set more or fewer second candidate complexity levels. The set second candidate complexity levels may indicate different picture complexities, and the intra-frame refresh frame number corresponding to the set second candidate complexity level may also be other intra-frame refresh frame numbers. The embodiments of the present application do not impose any limitations on this.

[0110] In this way, by considering the complexity of the reference picture in the above manner, the number n of local intra-frame refresh frames included in a refresh cycle is determined. When the picture complexity is high, the refresh of the entire image frame can be completed based on more local intra-frame refresh frames, thereby avoiding large fluctuations in the network due to the transmission of large local intra-frame refresh frames, thereby ensuring the smoothness of video picture playback. When the picture complexity is low, the refresh of the entire image frame can be completed based on fewer local intra-frame refresh frames, thereby completing the refresh of the entire image frame more efficiently and resolving related errors more promptly.

[0111] It should be noted that, in actual applications, each local intra-frame refresh frame in a refresh cycle includes a local area that supports the use of the intra-frame prediction mode. Usually, the local area that supports the use of the intra-frame prediction mode in each local intra-frame refresh frame in a refresh cycle is automatically determined by the server. For example, the server can evenly allocate the local area that needs to use the intra-frame prediction mode to each local intra-frame refresh frame according to the number of local intra-frame refresh frames included in the refresh cycle; assuming that the refresh cycle includes 4 local intra-frame refresh frames, the server can divide the entire image frame horizontally into four local areas, and allocate one local area to each local intra-frame refresh frame in the refresh cycle as its local area using the intra-frame prediction mode.

[0112] In an embodiment of the present application, the local area in the local intra refresh frame that supports the use of the intra prediction mode supports custom specification. For example, the local area in the local intra refresh frame that supports the use of the intra prediction mode can be manually specified by the relevant object, as long as the local areas of each local intra refresh frame that supports the use of the intra prediction mode within the refresh cycle are combined into a complete image frame.

[0113] Exemplarily, the local area using the intra prediction mode may be allocated to each local intra refresh frame in the refresh cycle according to the picture complexity of the local area in the image frame; Figure 4 A schematic diagram of a local area that customizes the use of an intra-frame prediction mode provided in an embodiment of the present application, such as Figure 4 As shown, in the case where four local intra refresh frames are included in the refresh cycle, the local areas of the four local intra refresh frames that support the use of the intra prediction mode are 401, 402, 403 and 404 respectively, wherein the local areas 402 and 403 corresponding to more complex pictures have smaller areas, while the local areas 401 and 404 corresponding to simpler pictures have larger areas.

[0114] Of course, in actual applications, the local area of ​​each local intra refresh frame using the intra prediction mode in the refresh cycle can also be customized according to other factors, and the embodiment of the present application does not impose any limitation on this. In addition, the shape of the local area using the intra prediction mode in the local intra refresh frame can be a regular shape or an irregular shape, and the embodiment of the present application does not impose any limitation on this.

[0115] In this way, supporting customization of local areas in local intra-frame refresh frames that use intra-frame prediction modes can improve the flexibility of local intra-frame refresh frames in intra-frame refresh technology, and by customizing local areas in local intra-frame refresh frames that use intra-frame prediction modes, it helps to ensure that the definition mode of the local area is more in line with relevant business needs. In many cases, it can reduce network fluctuations and ensure the smoothness of the played video images.

[0116] In order to facilitate further understanding of the image encoding method provided in the embodiment of the present application, the following takes the image encoding method applied to the video stream of the cloud game as an example. Figure 5 The flowchart shown provides an overall exemplary introduction to the image encoding method.

[0117] It should be noted that cloud gaming is an online gaming technology based on cloud computing technology. In a cloud gaming scenario, the game is not run on the player's game terminal, but on a cloud server. The cloud server renders the game scene into a video and audio stream and transmits it to the player's game terminal through the network.

[0118] like Figure 5As shown, in the process of transmitting the video stream of the cloud game, the cloud server can obtain the network bandwidth after the encoding time of the previous IDR frame, and the picture complexity of each game image frame encoded after the previous IDR frame. Then, according to the obtained network bandwidth and picture complexity, the target interval corresponding to the next target period IDR frame to be encoded is determined; specifically, if the network bandwidth belongs to the low bandwidth interval, or the picture complexity belongs to the high picture complexity gear, the target interval can be determined as the preset first interval t1, and the first interval t1 can be, for example, 60s or 5000 frames; conversely, if the network bandwidth belongs to the high bandwidth interval, and the picture complexity belongs to the low picture complexity gear, the target interval can be determined as the preset second interval t2, and the second interval t2 can be, for example, 30s or 2000 frames.

[0119] Before the interval between the previous IDR frame reaches the above target interval, the cloud server can detect whether there is an IDR frame application request from the upper layer. If not, the P frame is encoded normally and inserted into the game image frame sequence transmitted to the player's game terminal. If it exists, the IDR frame application request is responded to, the target temporary IDR frame is encoded, and it is inserted into the game image frame sequence transmitted to the player's game terminal. In the case that the received IDR frame application request is not withdrawn, or the IDR frame application request is received again, the cloud server can detect whether the interval between the current and the just-encoded target temporary IDR frame reaches the preset interval t0, which can be 100ms or 5 frames, for example; if not, the encoded P frame is inserted into the game image frame sequence transmitted to the player's game terminal; if it has been reached, the next target temporary IDR frame is encoded and inserted into the game image frame sequence transmitted to the player's game terminal. This is done until the IDR frame application request received by the cloud server is withdrawn, or no IDR frame application request is received within a specific time.

[0120] If the cloud server does not detect an IDR frame application request before the interval between the previous IDR frame and the previous IDR frame reaches the above target interval, the cloud server can encode the target period IDR frame and insert it into the game image frame sequence transmitted to the player's game terminal when the interval between the previous IDR frame and the previous IDR frame reaches the above target interval. If the cloud server detects an IDR frame application request before the interval between the previous IDR frame and the previous IDR frame reaches the above target interval, the cloud server can respond to the IDR frame application request to encode the target temporary IDR frame as the new previous IDR frame, and re-determine the target interval that should be met between the next target period IDR frame and the new previous IDR frame.

[0121] It should be understood that the image encoding method provided in the embodiment of the present application can be applied not only to the above-mentioned cloud gaming scenario, but also to any scenario of real-time transmission of video streams, such as live broadcast scenarios, etc. No limitation is made here to the scenarios in which the image encoding method provided in the embodiment of the present application is applied.

[0122] With respect to the image encoding method described above, the present application also provides a corresponding image encoding device so that the above image encoding method can be applied and implemented in practice.

[0123] See also Figure 6 , Figure 6 It is the same as above Figure 2 The structural diagram of the image encoding method corresponding to the image encoding device 600 is shown in FIG. Figure 6 As shown, the image encoding device 600 includes:

[0124] The information acquisition module 601 is used to acquire coding cycle reference information; the coding cycle reference information includes at least one of reference network information and reference picture information, the reference network information is used to characterize the network transmission condition from the video encoding end to the video decoding end within the reference period, and the reference picture information is used to characterize the picture complexity of the image frame encoded within the reference period, and the reference period is the time period between the encoding time of the last instant decoding refresh IDR frame and the current time;

[0125] The interval determination module 602 is used to determine the target interval corresponding to the target period IDR frame to be encoded according to the coding period reference information; the target interval is the interval that should be satisfied between the target period IDR frame and the previous IDR frame;

[0126] The encoding module 603 is used to encode the target period IDR frame when the interval between the previous IDR frame and the previous IDR frame reaches the target interval, and insert the target period IDR frame into the image frame sequence transmitted to the video decoding end.

[0127] Optionally, the encoding module 603 is further used for:

[0128] When an IDR frame application request is received, responding to the IDR frame application request, encoding a target temporary IDR frame, and inserting the target temporary IDR frame into the image frame sequence;

[0129] When the IDR frame application request is not revoked or the IDR frame application request is received again, when the interval between the target temporary IDR frame and the target temporary IDR frame reaches the preset interval, the next target temporary IDR frame is encoded and inserted into the image frame sequence.

[0130] Optionally, the encoding module 603 is further used for:

[0131] If an IDR frame application request is received before the interval between the previous IDR frame and the previous IDR frame reaches the target interval, respond to the IDR frame application request, encode a target temporary IDR frame, insert the target temporary IDR frame into the image frame sequence, and abandon encoding the target periodic IDR frame;

[0132] The information acquisition module 601 is also used for:

[0133] The target temporary IDR frame is used as a new previous IDR frame, and based on the coding period reference information after the coding moment of the new previous IDR frame, a target interval corresponding to the next target period IDR frame to be coded is determined.

[0134] Optionally, the coding period reference information includes the reference network information and the reference picture information, the reference network information is the reference network bandwidth within the reference time period, and the reference picture information is the reference picture complexity determined according to the picture complexity of each image frame encoded within the reference time period; then the interval determination module 602 is specifically used to:

[0135] When the reference network bandwidth belongs to a low bandwidth interval, or the reference picture complexity belongs to a high complexity level, determining the target interval to be a first interval;

[0136] When the reference network bandwidth belongs to a high bandwidth interval and the reference picture complexity belongs to a low complexity level, the target interval is determined to be a second interval; and the second interval is smaller than the first interval.

[0137] Optionally, the coding period reference information is the reference network information, and the reference network information is the reference network bandwidth within the reference time period; then the interval determination module 602 is specifically used to:

[0138] Among multiple first candidate bandwidth intervals, determine a first candidate bandwidth interval to which the reference network bandwidth belongs as a first target bandwidth interval, and determine an IDR frame coding interval corresponding to the first target bandwidth interval as the target interval;

[0139] There is no overlapping area among the multiple first candidate bandwidth intervals, the multiple first candidate bandwidth intervals correspond to different IDR frame coding intervals respectively, and the bandwidth value in the first candidate bandwidth interval is negatively correlated with the IDR frame coding interval corresponding to the first candidate bandwidth interval.

[0140] Optionally, the coding period reference information is the reference picture information, and the reference picture information is a reference picture complexity determined according to the picture complexity of each image frame encoded in the reference time period; then the interval determination module 602 is specifically used to:

[0141] Among a plurality of first candidate complexity levels, determining a first candidate complexity level to which the reference picture complexity belongs as a first target complexity level, and determining an IDR frame coding interval corresponding to the first target complexity level as the target interval;

[0142] Among them, the multiple first candidate complexity levels correspond to different IDR frame coding intervals respectively, and the picture complexity indicated by the first candidate complexity level is positively correlated with the IDR frame coding interval corresponding to the first candidate complexity level.

[0143] Optionally, the encoding module 603 is further used for:

[0144] When the reference network bandwidth belongs to a low bandwidth interval, or the reference picture complexity belongs to a high complexity level, determining to use the intra-frame refresh technology;

[0145] When the interval between the previous IDR frame and the previous IDR frame reaches the target interval, the intra-frame refresh technology is used to encode n local intra-frame refresh frames, and the n local intra-frame refresh frames are inserted into the image frame sequence; n is an integer greater than 1.

[0146] Optionally, the device further includes a first frame number determination module, wherein the first frame number determination module is configured to:

[0147] Among multiple second candidate bandwidth intervals, determine the second candidate bandwidth interval to which the reference network bandwidth belongs as the second target bandwidth interval, and determine the number of intra-frame refresh frames corresponding to the second target bandwidth interval as n;

[0148] There is no overlapping area among the multiple second candidate bandwidth intervals, the multiple second candidate bandwidth intervals correspond to different intra-frame refresh frame numbers respectively, and the bandwidth value in the second candidate bandwidth interval is negatively correlated with the intra-frame refresh frame number corresponding to the second candidate bandwidth interval.

[0149] Optionally, the device further includes a second frame number determination module, wherein the second frame number determination module is configured to:

[0150] Among a plurality of second candidate complexity levels, determining the second candidate complexity level to which the reference picture complexity belongs as the second target complexity level, and determining the number of intra-frame refresh frames corresponding to the second target complexity level as the n;

[0151] Among them, the multiple second candidate complexity levels correspond to different intra-frame refresh frame numbers respectively, and the picture complexity indicated by the second candidate complexity level is positively correlated with the intra-frame refresh frame number corresponding to the second candidate complexity level.

[0152] Optionally, the local area in the partial intra refresh frame supports the use of an intra prediction mode, and the local area in the partial intra refresh frame that supports the use of an intra prediction mode supports custom designation;

[0153] The n local intra refresh frames each support a local area using an intra prediction mode and are combined into a complete image frame.

[0154] The image encoding device provided in the embodiment of the present application can refer to the network transmission conditions from the video encoding end to the video decoding end, and at least one of the information of the picture complexity of the encoded image frame, flexibly and adaptively determine the target interval that should be met between the current target period IDR frame to be encoded and the previous IDR frame, and when the target interval is reached between the previous IDR frame, encode the target period IDR frame and insert it into the image frame sequence; compared with the method of inserting IDR frames into the image frame sequence according to a fixed frame interval in the related art, the embodiment of the present application can flexibly control the insertion interval of the IDR frame, so that the insertion interval of the IDR frame is adapted to the current network transmission conditions and / or the picture complexity of the currently encoded image frame, thereby avoiding the video playback quality of the video decoding end being affected by the insertion of the IDR frame, such as avoiding frequent insertion of IDR frames when the network transmission conditions are poor or the picture complexity of the encoded image frame is high, and accordingly, avoiding the situation of network freeze and the situation of blurred and flickering video played at the video decoding end, thereby ensuring the video playback quality of the video decoding end.

[0155] The embodiment of the present application also provides a computer device for image encoding, which may specifically be a terminal device or a server. The terminal device and server provided in the embodiment of the present application will be introduced below from the perspective of hardware instantiation.

[0156] See also Figure 7 , Figure 7 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Figure 7 For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc., taking the terminal as a computer as an example:

[0157] Figure 7 FIG. 1 is a block diagram showing a partial structure of a computer related to a terminal provided in an embodiment of the present application. Figure 7 The computer includes: a radio frequency (RF) circuit 710, a memory 720, an input unit 730 (including a touch panel 731 and other input devices 732), a display unit 740 (including a display panel 741), a sensor 750, an audio circuit 760 (which can be connected to a speaker 761 and a microphone 762), a wireless fidelity (WiFi) module 770, a processor 780, and a power supply 790. Those skilled in the art will understand that Figure 7 The computer structure shown in the figure does not constitute a limitation of the computer, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0158] The memory 720 can be used to store software programs and modules. The processor 780 executes various functional applications and data processing of the computer by running the software programs and modules stored in the memory 720. The memory 720 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer (such as audio data, a phone book, etc.), etc. In addition, the memory 720 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0159] The processor 780 is the control center of the computer. It uses various interfaces and lines to connect various parts of the entire computer. It executes various functions of the computer and processes data by running or executing software programs and / or modules stored in the memory 720 and calling data stored in the memory 720. Optionally, the processor 780 may include one or more processing units; preferably, the processor 780 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 780.

[0160] In the embodiment of the present application, the processor 780 included in the terminal is also used to execute the steps of any implementation method of the image encoding method provided in the embodiment of the present application.

[0161] See also Figure 8 , Figure 8A schematic diagram of the structure of a server 800 provided for an embodiment of the present application. The server 800 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 822 (for example, one or more processors) and a memory 832, and one or more storage media 830 (for example, one or more mass storage devices) storing application programs 842 or data 844. Among them, the memory 832 and the storage medium 830 may be temporary storage or permanent storage. The program stored in the storage medium 830 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 822 may be configured to communicate with the storage medium 830 to execute a series of instruction operations in the storage medium 830 on the server 800.

[0162] The server 800 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input and output interfaces 858, and / or one or more operating systems, such as Windows Server 2000. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0163] The steps performed by the server in the above embodiment can be based on the Figure 8 The server structure shown.

[0164] Among them, the CPU 822 can also be used to execute the steps of any implementation method of the image encoding method provided in the embodiments of the present application.

[0165] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program, wherein the computer program is used to execute any one of the implementation methods of an image encoding method described in the aforementioned embodiments.

[0166] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes any one of the implementation methods of an image encoding method described in the above embodiments.

[0167] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0168] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0169] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0170] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store computer programs.

[0172] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0173] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image coding method, characterized in that: The method comprises: Acquire coding cycle reference information; the coding cycle reference information includes at least one of reference network information and reference picture information, the reference network information is used to characterize the network transmission condition from the video encoding end to the video decoding end within the reference time period, the reference picture information is used to characterize the picture complexity of the image frame encoded within the reference time period, and the reference time period is the time period between the encoding time of the last instant decoding refresh IDR frame and the current time; Determine, according to the coding cycle reference information, a target interval corresponding to the target cycle IDR frame to be encoded; the target interval is an interval that should be satisfied between the target cycle IDR frame and the previous IDR frame; When the interval between the previous IDR frame and the previous IDR frame reaches the target interval, the target period IDR frame is encoded and inserted into the image frame sequence transmitted to the video decoding end.

2. The method according to claim 1, characterized in that The method further comprises: When an IDR frame application request is received, responding to the IDR frame application request, encoding a target temporary IDR frame, and inserting the target temporary IDR frame into the image frame sequence; When the IDR frame application request is not revoked or the IDR frame application request is received again, when the interval between the target temporary IDR frame and the target temporary IDR frame reaches the preset interval, the next target temporary IDR frame is encoded and inserted into the image frame sequence.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: If an IDR frame application request is received before the interval between the previous IDR frame and the previous IDR frame reaches the target interval, respond to the IDR frame application request, encode a target temporary IDR frame, insert the target temporary IDR frame into the image frame sequence, and abandon encoding the target periodic IDR frame; The target temporary IDR frame is used as a new previous IDR frame, and based on the coding period reference information after the coding moment of the new previous IDR frame, a target interval corresponding to the next target period IDR frame to be coded is determined.

4. The method according to claim 1, characterized in that: The coding period reference information includes the reference network information and the reference picture information, the reference network information is the reference network bandwidth within the reference period, and the reference picture information is the reference picture complexity determined according to the picture complexity of each image frame encoded within the reference period; The step of determining, according to the coding period reference information, a target interval corresponding to a target period IDR frame to be encoded includes: When the reference network bandwidth belongs to a low bandwidth interval, or the reference picture complexity belongs to a high complexity level, determining the target interval to be a first interval; When the reference network bandwidth belongs to a high bandwidth interval and the reference picture complexity belongs to a low complexity level, the target interval is determined to be a second interval; and the second interval is smaller than the first interval.

5. The method according to claim 1, characterized in that: The coding period reference information is the reference network information, and the reference network information is the reference network bandwidth within the reference period; The step of determining, according to the coding period reference information, a target interval corresponding to a target period IDR frame to be encoded includes: Among multiple first candidate bandwidth intervals, determine a first candidate bandwidth interval to which the reference network bandwidth belongs as a first target bandwidth interval, and determine an IDR frame coding interval corresponding to the first target bandwidth interval as the target interval; There is no overlapping area among the multiple first candidate bandwidth intervals, the multiple first candidate bandwidth intervals correspond to different IDR frame coding intervals respectively, and the bandwidth value in the first candidate bandwidth interval is negatively correlated with the IDR frame coding interval corresponding to the first candidate bandwidth interval.

6. The method according to claim 1, characterized in that The coding period reference information is the reference picture information, and the reference picture information is a reference picture complexity determined according to the picture complexity of each image frame encoded in the reference period; The step of determining, according to the coding period reference information, a target interval corresponding to a target period IDR frame to be encoded includes: Among a plurality of first candidate complexity levels, determining a first candidate complexity level to which the reference picture complexity belongs as a first target complexity level, and determining an IDR frame coding interval corresponding to the first target complexity level as the target interval; Among them, the multiple first candidate complexity levels correspond to different IDR frame coding intervals respectively, and the picture complexity indicated by the first candidate complexity level is positively correlated with the IDR frame coding interval corresponding to the first candidate complexity level.

7. The method according to claim 4, characterized in that The method further comprises: When the reference network bandwidth belongs to a low bandwidth interval, or the reference picture complexity belongs to a high complexity level, determining to use the intra-frame refresh technology; When the interval between the previous IDR frame and the previous IDR frame reaches the target interval, the intra-frame refresh technology is used to encode n local intra-frame refresh frames, and the n local intra-frame refresh frames are inserted into the image frame sequence; n is an integer greater than 1.

8. The method according to claim 7, characterized in that The number n of the partial intra-frame refresh frames is determined by: Among multiple second candidate bandwidth intervals, determine the second candidate bandwidth interval to which the reference network bandwidth belongs as the second target bandwidth interval, and determine the number of intra-frame refresh frames corresponding to the second target bandwidth interval as n; There is no overlapping area among the multiple second candidate bandwidth intervals, the multiple second candidate bandwidth intervals correspond to different intra-frame refresh frame numbers respectively, and the bandwidth value in the second candidate bandwidth interval is negatively correlated with the intra-frame refresh frame number corresponding to the second candidate bandwidth interval.

9. The method according to claim 7, characterized in that: The number n of the partial intra-frame refresh frames is determined by: Among a plurality of second candidate complexity levels, determining the second candidate complexity level to which the reference picture complexity belongs as the second target complexity level, and determining the number of intra-frame refresh frames corresponding to the second target complexity level as the n; Among them, the multiple second candidate complexity levels correspond to different intra-frame refresh frame numbers respectively, and the picture complexity indicated by the second candidate complexity level is positively correlated with the intra-frame refresh frame number corresponding to the second candidate complexity level.

10. The method according to any one of claims 7 to 9, characterized in that: The local area in the local intra refresh frame supports the use of the intra prediction mode, and the local area in the local intra refresh frame that supports the use of the intra prediction mode supports custom designation; The n local intra refresh frames each support a local area using an intra prediction mode and are combined into a complete image frame.

11. An image encoding device, characterized in that: The device comprises: An information acquisition module, used to acquire coding cycle reference information; the coding cycle reference information includes at least one of reference network information and reference picture information, the reference network information is used to characterize the network transmission condition from the video encoding end to the video decoding end within the reference period, and the reference picture information is used to characterize the picture complexity of the image frame encoded within the reference period, and the reference period is the time period between the encoding time of the last instant decoding refresh IDR frame and the current time; An interval determination module, used to determine a target interval corresponding to a target period IDR frame to be encoded according to the coding period reference information; the target interval is an interval that should be satisfied between the target period IDR frame and the previous IDR frame; The encoding module is used to encode the target period IDR frame when the interval between the previous IDR frame and the previous IDR frame reaches the target interval, and insert the target period IDR frame into the image frame sequence transmitted to the video decoding end.

12. A computer device, characterized in that: The device includes a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the image encoding method according to any one of claims 1 to 10 according to the computer program.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the image encoding method according to any one of claims 1 to 10.

14. A computer program product, comprising a computer program or instructions, characterized in that: When the computer program or the instruction is executed by a processor, the image encoding method according to any one of claims 1 to 10 is implemented.

Images