Image encoding method and apparatus

Abstract

The application relates to the field of video coding, and particularly provides an image coding method and device. The image coding method of the application comprises the following steps: acquiring a first screen content image; if the first screen content image is a first frame type, performing first coding on the first screen content image to obtain a first code stream, the first code stream comprising coding data of the first screen content image, and the first frame type being used for representing a key frame; and if the first screen content image is a second frame type, inputting the first screen content image into a coding chip, the coding chip being used for performing second coding on the first screen content image to obtain a second code stream, the second code stream comprising coding data of the first screen content image, and the second frame type being used for representing a non-key frame. The application is beneficial to solving the problems of large CPU power consumption and large delay caused by screen content coding.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to video coding technology, and in particular, to an image coding method and device. BACKGROUND

[0002] With the development of the times, screen content mainly generated by computer images, such as game live broadcast, shared documents, etc., occupies an increasingly important position in video coding. And computer-generated images have the characteristics of sharp boundaries, complex textures, and noise-free pure color areas, which are different from traditional video content recorded by lenses. If the main (main profile) level of the High Efficiency Video Coding (HEVC) standard is used to encode computer-generated content, under the same code rate, sharp edges are more likely to cause compression distortion and produce artifacts. In order to obtain the same image quality, more code rate needs to be consumed, increasing the bandwidth burden. Therefore, in order to adapt to the characteristics of computer-generated images, Screen Content Coding (SCC) technology is proposed.

[0003] SCC technology uses palette and intra block copy (IBO) coding techniques to save code rate while ensuring image quality, thereby ensuring certain video coding efficiency. However, this also brings higher computational complexity to the video coding process. Moreover, the video coding process of the SCC technology is usually executed by a central processing unit (CPU), and the high computational complexity leads to high CPU coding power consumption, which causes obvious delay and lag in high-performance scenarios. SUMMARY

[0004] Embodiments of the present application provide an image coding method and device, which can solve the problems of high CPU power consumption and large delay caused by screen content coding.

[0005] In a first aspect, the embodiments of the present application provide an image coding method, which can include: obtaining a first screen content image. If the first screen content image is a first frame type, the first screen content image is first encoded to obtain a first code stream, the first code stream including the encoded data of the first screen content image, and the first frame type is used to represent a key frame. If the first screen content image is a second frame type, the first screen content image is input into an encoding chip, the encoding chip is used to second encode the first screen content image to obtain a second code stream, the second code stream including the encoded data of the first screen content image, and the second frame type is used to represent a non-key frame.

[0006] Thus, in the first aspect, by first encoding the first screen content image of the first frame type, the first screen content image of the second frame type is provided to the encoding chip (e.g., SoC) to make the encoding chip (e.g., SoC) second encode the first screen content image. In this way, the image quality of the key frame is guaranteed by the first encoding, and the encoding power consumption of the non-key frame is reduced by the second encoding, so that the image quality of the screen content image is guaranteed while the encoding power consumption and delay can be reduced.

[0007] In a possible design, the method can further include:

[0008] If M≤L-2, it is determined whether the first screen content image has a picture mutation according to a second screen content image; M represents the number of frames between the first screen content image and a previous key frame, L represents the length of a group of pictures (GOP), and the second screen content image is a previous frame image of the first screen content image.

[0009] If the first screen content image has the picture mutation, the first screen content image is determined as the first frame type.

[0010] If the first screen content image does not have the picture mutation, the first screen content image is determined as the second frame type.

[0011] In this way, by comparing the size relationship between M and L-2, it can be determined whether the first screen content image and the previous key frame belong to the same GOP. If it is determined that the first screen content image does not belong to the same GOP from the length of the GOP, the first screen content image is determined as the first frame type, and the first screen content image of the first frame type is first encoded. If it is determined that the first screen content image should belong to the same GOP from the length of the GOP, it is further determined whether the first screen content image has the picture mutation according to the second screen content image. If the first screen content image has the picture mutation, the first screen content image is determined as the first frame type, and the first screen content image of the first frame type is first encoded. If the first screen content image does not have the picture mutation, the first screen content image is determined as the second frame type, and the first screen content image of the second frame type is provided to the encoding chip (e.g., SoC) to make the encoding chip (e.g., SoC) second encode the first screen content image. In this way, the image quality of the key frame is guaranteed by the first encoding, and the encoding power consumption of the non-key frame is reduced by the second encoding, so that the image quality of the screen content image is guaranteed while the encoding power consumption and delay can be reduced. Moreover, by detecting whether the first screen content image has the picture mutation, the first encoding is used when the picture mutation occurs to guarantee the image quality of the picture mutation.

[0012] In a possible design, if the first screen content image is of the first frame type, the method can further include:

[0013] obtaining a third screen content image, the third screen content image being a frame image after the first screen content image, the third screen content image and the first screen content image belonging to a same image group, and the third screen content image being the second frame type.

[0014] inputting the third screen content image and the reconstructed frame into the encoding chip, the encoding chip being further configured to encode the third screen content image as the second encoding by taking the reconstructed frame as a reference frame, and the second code stream further comprising encoded data of the third screen content image.

[0015] wherein the reconstructed frame is a reconstructed image obtained by decoding the encoded data of the first screen content image.

[0016] Thus, the first screen image of the first frame type is encoded as the first encoding to ensure the image quality of the first screen image, and the third screen content image belonging to the same image group as the first screen content image is provided to the encoding chip (e.g., SoC) to enable the encoding chip (e.g., SoC) to encode the third screen content image as the second encoding, thereby reducing the encoding power consumption and delay. The reconstructed frame of the first screen image can provide a better quality reference frame for the third screen content image of the same image group, thereby improving the picture quality.

[0017] In a possible design, the method can further include:

[0018] obtaining a first image, performing scene recognition on the first image, and determining, according to a scene recognition result of the first image, whether the first image is the first screen content image or a first natural scene image.

[0019] In a possible design, if the first image is the first natural scene image, the method can further include:

[0020] inputting the first natural scene image into the encoding chip, the encoding chip being further configured to encode the first natural scene image as the second encoding.

[0021] In a possible design, the first frame type includes an I frame, and the second frame type includes a P frame.

[0022] In a second aspect, an embodiment of the present application provides an image encoding apparatus. The apparatus can be an electronic device or a processing module in an electronic device, for example, a functional module in an electronic device for implementing the first aspect or any possible implementation manner of the first aspect. For example, the image encoding apparatus can include an obtaining module and an encoding module.

[0023] The obtaining module is configured to obtain a first screen content image.

[0024] The encoding module is configured to: if the first screen content image is a first frame type, encode the first screen content image to obtain a first bitstream, the first bitstream comprising encoded data of the first screen content image, the first frame type being used to represent a key frame; and if the first screen content image is a second frame type, input the first screen content image to an encoding chip, the encoding chip being configured to encode the first screen content image to obtain a second bitstream, the second bitstream comprising encoded data of the first screen content image, the second frame type being used to represent a non-key frame.

[0025] In a possible design, the encoding module is further configured to: if M≤L-2, determine whether the first screen content image has a picture mutation according to a second screen content image, M representing a number of frames between the first screen content image and a previous key frame, L representing a length of a group of pictures, and the second screen content image being a previous frame image of the first screen content image; if the first screen content image has the picture mutation, determine that the first screen content image is of the first frame type; and if the first screen content image does not have the picture mutation, determine that the first screen content image is of the second frame type.

[0026] In a possible design, if the first screen content image is of the first frame type, the obtaining module is further configured to obtain a third screen content image, the third screen content image being a frame image after the first screen content image and belonging to a same group of pictures as the first screen content image, and the third screen content image being of the second frame type.

[0027] The encoding module is further configured to input the third screen content image and the reconstructed frame to the encoding chip, and the encoding chip is further configured to encode the third screen content image by taking the reconstructed frame as a reference frame, and the second bitstream further comprises encoded data of the third screen content image.

[0028] The reconstructed frame is a reconstructed image obtained by decoding the encoded data of the first screen content image.

[0029] In a possible design, the obtaining module is further configured to: obtain a first image; and perform scene recognition on the first image, and determine, according to a scene recognition result of the first image, whether the first image is the first screen content image or a first natural scene image.

[0030] In a possible design, if the first image is the first natural scene image, the encoding module is further configured to: input the first natural scene image to the encoding chip, and the encoding chip is further configured to encode the first natural scene image by using the second encoding manner.

[0031] In one possible design, the first frame type includes I-frames, and the second frame type includes P-frames.

[0032] In a third aspect, an embodiment of the present application provides an image encoding apparatus, including: one or more processors. A memory, configured to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method in the first aspect or any one of the first aspect.

[0033] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, including a bitstream obtained according to the method in the first aspect or any one of the first aspect.

[0034] In a fifth aspect, an embodiment of the present application provides a computer program product, when the computer program product is run on a computer, causing the computer to execute the method in the first aspect or any one of the first aspect.

[0035] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, including computer instructions, when the computer instructions are run on a computer, causing the computer to execute the method in the first aspect or any one of the first aspect.

[0036] It should be understood that the second to sixth aspects of the present application are consistent with the technical solution of the first aspect of the present application, and the beneficial effects obtained by each aspect and the corresponding feasible implementation manner are similar, which will not be repeated. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A schematic block diagram of a video encoding and decoding system 10 to which embodiments of the present application are applied;

[0038] Figure 2 A schematic diagram of a GOP provided by an embodiment of the present application;

[0039] Figure 3 A schematic diagram of an application scenario of an embodiment of the present application;

[0040] Figure 4 A flowchart of an image encoding method provided by an embodiment of the present application;

[0041] Figure 5 A schematic diagram of an image encoding method provided by an embodiment of the present application;

[0042] Figure 6 A flowchart of an image encoding method provided by an embodiment of the present application;

[0043] Figure 7 A schematic diagram of an image encoding method provided by an embodiment of the present application;

[0044] Figure 8 A schematic diagram of an image coding method provided by an embodiment of the present application is shown in FIG. 1.

[0045] Figure 9 A schematic diagram of an image coding device provided by an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0046] An image coding method and device provided by an embodiment of the present application are described in detail below with reference to the accompanying drawings.

[0047] The terms "first" and "second" and the like in the description of the present application and the accompanying drawings are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of the objects.

[0048] In addition, the terms "comprising" and "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include other steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0049] It should be noted that in the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Rather, the use of "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0050] In the description of the present application, unless otherwise specified, "a plurality of" means two or more. "And / or" in this document is merely a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone.

[0051] Video coding generally refers to processing a sequence of pictures forming a video (also referred to as a video stream) or video sequence. In the field of video processing, the terms "picture", "frame" or "image" can be used as synonyms. Video coding is performed at the sending side, typically including processing (e.g., by re-encoding) low-quality video and high-quality video to reduce the amount of data required to obtain high-quality video at the receiving side, thus more efficiently storing and / or transmitting. Video decoding is performed at the receiving side, typically including inverse processing relative to coding to reconstruct video frames. The combination of the coding part and the decoding part is also referred to as coding and decoding (codec).

[0052] A system architecture to which embodiments of the present application apply is described below. Referring to Figure 1 , Figure 1 An exemplary block diagram of a video encoding and decoding system 10 to which embodiments of the present application apply is shown. As shown in Figure 1 , video encoding and decoding system 10 can include a source device 12 that generates encoded video data and, therefore, can be referred to as a video encoding apparatus. A destination device 14 can decode the encoded video data generated by the source device 12 and, therefore, can be referred to as a video decoding apparatus. Source device 12 and destination device 14 can comprise various devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" telephones, televisions, cameras, display devices, digital media players, video gaming consoles, car computers, wireless communication devices, or the like.

[0053] Although Figure 1 Source device 12 and destination device 14 are depicted as separate devices, device embodiments can also include source device 12 and destination device 14 simultaneously, or functionality of both, i.e., source device 12 or corresponding functionality and destination device 14 or corresponding functionality. In such embodiments, source device 12 or corresponding functionality and destination device 14 or corresponding functionality can be implemented using the same hardware and / or software, or separate hardware and / or software, or any combination thereof.

[0054] Source device 12 and destination device 14 can communicate via a link 13 over which the destination device 14 can receive encoded video data from the source device 12. Link 13 can comprise one or more media or devices capable of conveying encoded video data from source device 12 to destination device 14. In one example, link 13 can comprise one or more communication media that enable source device 12 to transmit encoded video data directly to destination device 14 in real-time. In this example, source device 12 can modulate encoded video data according to a communication standard, such as a wireless communication protocol, and can transmit the modulated video data to destination device 14. The one or more communication media can include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media can form part of a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet. The one or more communication media can include routers, switches, base stations, or other equipment that facilitates communication from source device 12 to destination device 14.

[0055] Source device 12 includes an encoder 20, and optionally, source device 12 can also include a picture source 16, a picture pre-processor 18, and a communication interface 22. In implementations, the encoder 20, the picture source 16, the picture pre-processor 18, and the communication interface 22 can be hardware components in source device 12, or software programs in source device 12. They are described as follows, respectively:

[0056] Picture source 16 can include or be any kind of picture capturing device for capturing, for example, real-world pictures, and / or any kind of picture or comment (for screen content coding, some text on the screen is also considered as part of the picture or image to be coded) generating device, for example, a computer graphics processor for generating computer-animated pictures, or any kind of device for acquiring and / or providing real-world pictures, computer-animated pictures (e.g., screen content, virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). Picture source 16 can be a camera for capturing pictures or a memory for storing pictures, and picture source 16 can also include any kind of (internal or external) interface for storing previously captured or generated pictures and / or for acquiring or receiving pictures. When picture source 16 is a camera, picture source 16 can be, for example, a local or integrated camera integrated in source device 12; when picture source 16 is a memory, picture source 16 can be a local or integrated memory, for example, integrated in source device 12. When picture source 16 includes an interface, the interface can be, for example, an external interface for receiving pictures from an external video source, for example, an external picture capturing device such as a camera, an external memory, or an external picture generating device, for example, an external computer graphics processor, a computer, or a server. The interface can be any kind of interface according to any proprietary or standardized interface protocol, for example, a wired or wireless interface, an optical interface.

[0057] A picture can be considered as a two-dimensional array or matrix of picture elements. The picture elements in the array can also be referred to as samples. The number of picture elements in the array or picture in horizontal and vertical direction (or axis) defines the size and / or resolution of the picture. For representing colors, typically three color components are used, i.e. the picture can be represented or comprise three sample arrays. For example in the RBG format or color space, the picture comprises corresponding arrays of red, green and blue samples. However, in video coding, typically each pixel is represented in a luma / chroma format or color space, e.g. for a picture in YUV format, comprising a luma component Y (sometimes also indicated by L) and two chroma components U and V. The luma component Y represents the luminance or gray level intensity (e.g. both in a gray scale picture), while the two chroma components U and V represent the chrominance or color information components. Accordingly, a picture in YUV format comprises a luma sample array of luma sample values (Y) and two chroma sample arrays of chroma values (U and V). A picture in RGB format can be converted or transformed into YUV format and vice versa, which process is also referred to as color conversion or transformation. If the picture is black and white, the picture can comprise only a luma sample array. In the embodiments of the present application, the picture transmitted by the picture source 16 to the picture processor can also be referred to as original picture data 17.

[0058] The picture pre-processor 18 is configured to receive the original picture data 17 and to perform a pre-processing on the original picture data 17 to obtain a pre-processed picture 19 or pre-processed picture data 19. The pre-processing performed by the picture pre-processor 18 can comprise, for example, trimming, color format conversion (e.g. from RGB to YUV format), toning or de-noising.

[0059] The encoder 20 (or video encoder 20) is configured to receive the pre-processed picture data 19 and to process the pre-processed picture data 19 using a relevant encoding scheme to provide encoded picture data 21.

[0060] The communication interface 22 is configured to receive the encoded picture data 21 and to transmit the encoded picture data 21 to the destination device 14 or any other device (e.g. a storage) via the link 13 for storage or direct reconstruction. The communication interface 22 can be configured to, for example, encapsulate the encoded picture data 21 into a suitable format, e.g. packets, for transmission over the link 13.

[0061] The destination device 14 comprises a decoder 30 and, optionally, a communication interface 28, a picture post-processor 32 and a display device 34. These are described separately as follows:

[0062] The communication interface 28 can be configured to receive the encoded picture data 21 from the source device 12 or any other source, such as a storage device, e.g. an encoded picture data storage device. The communication interface 28 can be configured to transmit or receive the encoded picture data 21 via a link 13 between the source device 12 and the destination device 14, such as a direct wired or wireless connection, or via any kind of network, such as a wired or wireless network or any combination thereof, or any kind of private and public network, or any combination thereof. The communication interface 28 can be configured to, for example, de-encapsulate data packets transmitted by the communication interface 22 to obtain the encoded picture data 21.

[0063] Both the communication interface 28 and the communication interface 22 can be configured as unidirectional or bidirectional communication interfaces and can be configured to, for example, send and receive messages to establish a connection, confirm and exchange any other information related to the communication link and / or the transmission of data, such as the encoded picture data.

[0064] The decoder 30 (or decoder 30) is configured to receive the encoded picture data 21 and provide decoded picture data 31 or decoded pictures 31.

[0065] The picture post-processor 32 is configured to perform post-processing on the decoded picture data 31 (also referred to as reconstructed picture data) to obtain post-processed picture data 33. The post-processing performed by the picture post-processor 32 can include color format conversion (e.g. from YUV to RGB format), toning, retouching or resampling, or any other processing, and can further be configured to transmit the post-processed picture data 33 to the display device 34.

[0066] The display device 34 is configured to receive the post-processed picture data 33 to display pictures, e.g. to a user or viewer. The display device 34 can be or can include any kind of display for presenting reconstructed pictures, such as an integrated or external display or monitor. For example, the display can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other kind of display.

[0067] Although, Figure 1Source device 12 and destination device 14 are illustrated as separate devices, but device embodiments can also include both source device 12 and destination device 14 or functionality of both, i.e., source device 12 or corresponding functionality and destination device 14 or corresponding functionality, at the same time. In such embodiments, source device 12 or corresponding functionality and destination device 14 or corresponding functionality can be implemented using the same hardware and / or software, or separate hardware and / or software, or any combination thereof.

[0068] The functionality of the different units or Figure 1 The presence and (precise) division of the functionality of source device 12 and / or destination device 14 as illustrated can vary from actual devices and applications. Source device 12 and destination device 14 can comprise any of a variety of devices, including handheld or stationary devices, of any category, e.g., notebook or laptop computers, mobile phones, smartphones, tablets or tablet computers, camcorders, desktop computers, set-top boxes, televisions, cameras, car devices, display devices, digital media players, video gaming consoles, video streaming devices, e.g., content service servers or content distribution servers, broadcast receiver devices, broadcast transmitter devices, etc., and can use or not use any category of operating system.

[0069] Both encoder 20 and decoder 30 can be implemented as any of a variety of suitable circuitry, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combinations thereof. If the techniques are implemented partially in software, a device can store instructions for the software in a suitable, non-transitory computer-readable storage medium and can execute the instructions with one or more processors to perform the techniques of this disclosure. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) can be considered processing circuitry.

[0070] In order to reduce the power consumption of the encoder 20 and the decoder 30, the embodiments of the present application adopt a combination of hardware and software to code and decode the video image. The specific implementation manner can be referred to the specific explanation and description of the following embodiments. The hardware part is exemplified by a system on chip (SoC), and the software part is exemplified by one or more CPUs executing the instructions of the software in hardware. The SoC has the characteristics of small area, low power consumption and good performance. The one or more CPUs executing the instructions of the software in hardware has the characteristic of high flexibility. Of course, it can be understood that the SoC can be replaced by other ASICs, and the embodiments of the present application are not limited thereto. The one or more CPUs executing the instructions of the software can also be replaced by other processors, and the embodiments of the present application are not limited thereto.

[0071] In some cases, Figure 1 The video coding and decoding system 10 shown in FIG. 1 is merely an example, and the techniques of the present application can be applied to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between the encoding and decoding devices. In other examples, data can be retrieved from local memory, streamed over a network, etc. The video encoding device can encode data and store the data to memory, and / or the video decoding device can retrieve data from memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to memory and / or retrieve data from memory and decode the data.

[0072] The technical solutions of the embodiments of the present application are described in detail as follows:

[0073] First, some technical terms or technical concepts applicable to the embodiments of the present application are introduced.

[0074] I frame: In the field of video coding, a frame that can be decoded independently of other frames is generally referred to as an "I frame". The coding prediction mode of all blocks of the I frame is intra prediction.

[0075] P frame: In the field of video coding, a frame that refers to a forward frame and is marked as a P frame type in the code stream is generally referred to as a "P frame".

[0076] B frame: In the field of video coding, a frame that can refer to both a forward frame and a backward frame is generally referred to as a "B frame". The embodiments of the present application mainly aim at low latency scenarios, and therefore generally encode as "P frames" in order to reduce latency.

[0077] Group of Pictures (GOP) Stream: For ease of description, this application embodiment refers to a video stream in which at least one P-frame is present between adjacent I-frames, or a video stream that allows inter-frame prediction modes, as a "GOP stream". It should be noted that a GOP stream can also be called a first bitstream and / or a second bitstream, etc., and its name is not limited thereto. In this application embodiment, the encoding process for generating a GOP stream includes first encoding and / or second encoding.

[0078] For example, Figure 2 This is a schematic diagram of a GOP provided in an embodiment of this application, such as... Figure 2 As shown, taking a GOP of length 30 as an example, a GOP can consist of one I-frame and 29 P-frames. I-frames are represented by striped rectangles, while P-frames are represented by untextured rectangles. P-frame encoding references the previous frame. For example, P-frame number 1 references I-frame number 0, P-frame number 2 references P-frame number 1, and so on, until the start of the next GOP. The next GOP uses the same method, which will not be elaborated further here.

[0079] Software encoding: This involves storing software instructions in a suitable non-transitory computer-readable storage medium, and allowing one or more processors (e.g., CPUs) to execute the instructions in hardware to encode the image. Software encoding supports SCC (Simplified Character Classification). In this application embodiment, software encoding can also be referred to as first encoding. The decoding process corresponding to software encoding is called software decoding.

[0080] Hard coding: This involves initiating an encoding chip (e.g., a System-on-a-Chip) to encode the image. Hard coding supports the main profile level of HEVC, such as the main profiles of protocols H.264 and H.265. In this embodiment, hard coding can also be referred to as secondary coding. The decoding process corresponding to soft coding is called hard decoding.

[0081] Hardware encoding offers advantages such as high performance and low power consumption, but its flexibility is a disadvantage due to the customized hardware. Software encoding, on the other hand, allows for on-the-fly modification of instructions and supports almost all encoding specifications, but CPU encoding consumes a lot of power and can cause noticeable stuttering in high-performance scenarios. This application's embodiments combine the advantages of both hardware and software encoding to improve encoding and decoding performance, effectively addressing the problems of high power consumption, high latency, and limited performance associated with software encoding.

[0082] Reconstructed Frame: During the encoding process, the encoding result of the previous frame needs to be decoded and saved for reference by subsequent frames. The decoded frame is generally called a "reconstructed frame." This reconstructed frame is consistent with the frame decoded by the decoder, which is called encoding-decoding consistency. For ease of description, the frames decoded by the decoder in this application embodiment are also collectively referred to as "reconstructed frames."

[0083] Screen content images refer to images captured from the image display units of various devices (such as computers, mobile terminals, etc.). Screen content images include, but are not limited to, computer graphics and text images, images that combine natural video with graphics / text, and computer-generated animated images. Screen content images are ubiquitous in various scenarios such as document sharing, game streaming, and video conferencing.

[0084] Natural scene images: refers to images captured by a camera.

[0085] Using the CPU to perform SCC (Simultaneous Computation and Encoding) on ​​screen content images can lead to high CPU power consumption, high latency, and limited performance in certain scenarios (e.g., video conferencing, shared desktops, and other high-performance interactive scenarios). This is because in these scenarios, continuous encoding of screen content images by the CPU results in high computational complexity and increased CPU power consumption. Sudden changes in the screen content image also increase encoding and decoding latency, causing noticeable stuttering. To address these issues, this application implements differentiated encoding for screen content images of different frame types to solve the problems of high power consumption, high latency, and limited performance. Specifically, the first frame type of screen content image is software encoded, while the second frame type is hard encoded. This reduces encoding power consumption and latency while maintaining the image quality of the screen content image.

[0086] For example, Figure 3 This is a schematic diagram illustrating an application scenario of an embodiment of this application, such as... Figure 3 As shown, this application scenario can include multiple electronic devices. Here, we use three electronic devices (electronic device 20, electronic device 30, and electronic device 40) as an example. Electronic device 20 initiates desktop sharing with electronic devices 30 and 40. Electronic device 20 acts as an encoding end; it can encode its own screen content image using the image encoding method of this embodiment to generate a bitstream, and then send the bitstream to electronic devices 30 and 40 respectively. Electronic devices 30 and 40 each receive the bitstream. Electronic devices 30 and 40 each act as decoding ends, decoding the received bitstream to obtain a reconstructed image, thereby achieving desktop sharing between different electronic devices.

[0087] It should be noted that, Figure 2The embodiments shown take the electronic device 20 as a notebook computer, and the electronic devices 30 and 40 as mobile phones as exemplary examples, and can also be other product forms. For example, at present, some electronic devices are exemplified as: a mobile phone, a tablet computer, a notebook computer, a palm computer, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with a wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a 5G network, or a terminal device in an evolved public land mobile network (PLMN), and the like, and the embodiments of the present application are not limited thereto.

[0088] By way of example and not limitation, in the embodiments of the present application, the wearable device can also be referred to as a wearable smart device, which is a general term for devices that are designed and developed by applying wearable technology to daily wear, such as glasses, gloves, watches, clothing, and shoes. The wearable device is a portable device that is directly worn on the body or integrated into the clothes or accessories of the user. The wearable device is not only a hardware device, but also a device that realizes powerful functions through software support and data interaction and cloud interaction. The general wearable smart device includes a device with full functions and large size, which can realize complete or partial functions without relying on a smart phone, such as a smart watch or smart glasses, and a device that focuses on a certain application function and needs to be used in cooperation with other devices, such as a smart phone, such as various smart wristbands and smart jewelry for monitoring vital signs.

[0089] In addition, in the embodiments of the present application, the electronic device can also be an electronic device in an Internet of Things (IoT) system. The IoT is an important component of future information technology development, and its main technical feature is to connect objects through communication technology and a network, thereby realizing an intelligent network of human-machine interconnection and object-object interconnection.

[0090] Various electronic devices as described above can be considered as on-board electronic devices if they are located on a vehicle (for example, placed in or installed in a vehicle), for example, also referred to as on-board units (OBU).

[0091] Please refer to Figure 4 , Figure 4 is a flowchart of an image encoding method provided by the embodiments of the present application. The method of the embodiments of the present application can be executed by an image encoding device. The image encoding device can be one or more processors (such as a CPU) to execute instructions stored in a storage medium in hardware to encode a screen content image. The image encoding device can be applied to the source device 12 in the above-mentioned embodiments, for example, the electronic device 20 in the embodiments as shown in Figure 3 It should be understood that a series of steps or operations involved in the embodiments of the present application can be executed in various orders and / or simultaneously, and the execution order is not limited by the size of the step number as shown in Figure 4 The method as shown in Figure 4 may include the following implementation steps:

[0092] S401, obtaining a first screen content image.

[0093] For example, the first screen content image can be captured from an image display unit of the electronic device 20. The first screen content image includes but is not limited to computer graphics and text images, natural video and image mixed with graphics / text, and computer-generated animation images, etc.

[0094] Before S401, color conversion and other processing can also be performed to obtain the first screen content image. Then, according to the frame type of the first screen content image, S402 or S403 can be selectively executed.

[0095] S402, if the first screen content image is of a first frame type, performing first encoding on the first screen content image to obtain a first code stream.

[0096] The first code stream can include encoding data of the first screen content image, and the first frame type is used to represent a key frame. For example, the first frame type can be an I frame.

[0097] As described above, the first encoding can also be referred to as soft encoding, which supports SCC. For example, the first screen content image can be encoded using a palette encoding framework. The first encoding can improve the distortion caused by screen content image encoding and improve the obtained image quality. The first encoding can reduce the code rate while obtaining equivalent image quality, thereby benefiting the reduction of bandwidth burden for transmitting the first code stream.

[0098] In some embodiments, the first code stream can further include first encoding indication information, which can be used to indicate that the first code stream is a code stream obtained through the first encoding. In this way, the decoding end can perform first decoding on the first code stream based on the first encoding indication information to obtain the reconstructed image.

[0099] Optionally, if the first screen content image is of the first frame type, one or more second screen content images of a second frame type following the first screen content image can be encoded using the first screen content image as a reference frame to improve image quality while reducing power consumption.

[0100] For example, the method described above can further include: obtaining a third screen content image, the third screen content image being a frame image following the first screen content image and belonging to the same group of pictures as the first screen content image, and the third screen content image being of the second frame type; and inputting the third screen content image and the reconstructed frame into an encoding chip (e.g., SoC) to enable the encoding chip (e.g., SoC) to encode the third screen content image using the reconstructed frame as a reference frame to obtain a second code stream, the second code stream including encoded data of the third screen content image.

[0101] The reconstructed frame is a reconstructed image obtained by decoding the encoded data of the first screen content image.

[0102] S403, if the first screen content image is of the second frame type, inputting the first screen content image into an encoding chip, the encoding chip being configured to encode the first screen content image to obtain a second code stream.

[0103] The second code stream can include encoded data of the first screen content image, and the second frame type can be used to represent a non-key frame. For example, the second frame type can be a P frame or a B frame.

[0104] The execution subject of the embodiment can provide the first screen content image of the second frame type to the encoding chip (for example, the SoC) to make the encoding chip (for example, the SoC) perform the second encoding on the first screen content image. As described above, the second encoding can also be referred to as the hard encoding, which supports the main profile of the HEVC. For example, the first screen content image can be encoded by using the hybrid encoding framework of the HEVC. Compared with the first encoding, the second encoding has the advantage of low power consumption.

[0105] In some embodiments, the second code stream can further include second encoding indication information, which can be used to indicate that the second code stream is a code stream obtained by the second encoding. In this way, the decoding end can perform the second decoding on the second code stream based on the second encoding indication information to obtain the reconstructed image.

[0106] For example, the first encoding indication information or the second encoding indication information can be carried in the video parameter set (VPS) of the first code stream or the second code stream.

[0107] For example, the length of the GOP is 30, the first frame type is the I frame, and the second frame type is the P frame. Figure 5 For example, the length of the GOP is 30, the first frame type is the I frame, and the second frame type is the P frame. Figure 5 As shown in the schematic diagram of the image encoding method provided by the embodiment of the present application, the encoding end performs the first encoding (that is, the SCC profile) on the I frame numbered 0 by using the CPU and performs the second encoding (that is, the main profile) on the P frame numbered 1 by using the SoC in the first GOP, the P frame numbered 1 refers to the I frame numbered 0, the P frames numbered 2 to 29 are encoded by using the SoC, and each P frame refers to the previous P frame. In the second GOP, the encoding manner is similar to that of the first GOP, thereby obtaining the first code stream and the second code stream. The first code stream includes the encoding data of the I frame numbered 0 in each GOP, and the second code stream includes the encoding data of the P frames in each GOP. Correspondingly, the decoding end performs the first decoding on the first code stream and the second decoding on the second code stream, and the reconstructed image of each screen content image can be obtained.

[0108] The above embodiment explains and illustrates the screen content image encoding. Alternatively, the image encoding method of the embodiment of the present application can also perform second encoding on a natural scene image to reduce image encoding power consumption and improve encoding performance. For example, scene recognition can be performed on an image to be encoded (e.g., a first image), and according to the scene recognition result, the first image is determined to be a first screen content image or a first natural scene image. If the first image is a first natural scene image, the first natural scene image is input to an encoding chip (e.g., a SoC), and the encoding chip is used to perform second encoding on the first natural scene image.

[0109] In the embodiment, the first screen content image of the first frame type is first encoded, and the first screen content image of the second frame type is second encoded to obtain a code stream. In this way, the image quality of the key frame is guaranteed by the first encoding, and the encoding power consumption of the non-key frame is reduced by the second encoding, so that the image quality of the screen content image is guaranteed while the encoding power consumption and the delay are reduced.

[0110] Please refer to Figure 6 , Figure 6 is a flowchart of an image encoding method provided by the embodiment of the present application. The method of the embodiment of the present application can be executed by an image encoding device. The image encoding device can be one or more processors (e.g., a CPU) to execute instructions stored in a storage medium in hardware to encode a screen content image. The image encoding device can be applied to the source device 12 in the above embodiment, for example, the electronic device 20 in the embodiment as shown in Figure 3 It should be understood that a series of steps or operations involved in the embodiment of the present application can be executed in various orders and / or simultaneously, and the execution order is not limited by the size of the step sequence number as shown in Figure 6 The method as shown in Figure 6 may include the following implementation steps:

[0111] S601, a first screen content image is acquired.

[0112] The explanation of S601 can be referred to the specific explanation of S401, which will not be repeated here.

[0113] S602, it is judged whether M≤L-2. If yes, S603 is executed, and if no, S604 is executed.

[0114] M represents the number of frames between the first screen content image and the previous key frame, and L represents the length of the group of pictures. By comparing the size relationship between M and L-2, whether the first screen content image and the previous key frame belong to the same GOP can be determined.

[0115] For example, Figure 7The schematic diagram of the image encoding method provided by the embodiment of the present application is shown in Figure 7 For example, L=30, when the first screen content image is the image frame with code 30, the previous key frame is the I frame with code 0, and the interval between them is 29 frames, then M=29, thus M>L-2, that is, the image frame with code 30 and the I frame with code 0 do not belong to the same GOP, then S604 is executed. When the first screen content image is the image frame with code 32, the previous key frame is the I frame with code 30, and the interval between them is 1 frame, then M=1, thus M<L-2, that is, the image frame with code 32 and the I frame with code 30 should belong to the same GOP according to the length of the GOP, then S603 is executed. Whether a new GOP needs to be started is determined through S603.

[0116] S603, determining whether the first screen content image has a picture mutation according to the second screen content image, if the first screen content image has a picture mutation, executing S604, and if the first screen content image does not have a picture mutation, executing S606.

[0117] The second screen content image is the previous frame image of the first screen content image. The first screen content image is taken as an example. Figure 7 For example, the image frame with code 31 and the image frame with code 32 can be PPT pictures with different contents (the specific contents are not limited), that is, a picture mutation occurs, as shown in FIG. 4. Figure 7 As shown in FIG. 4, the image frame with code 32 is first encoded (that is, SCC profile) by using the CPU, and the previous GOP is terminated and a new GOP is started. The GOP is changed from the previous code 30 to code 59, code 60 to code 89 to code 32 to code 61, code 62 to code 91, etc. The image frame with code 32 uses SCC, can fully utilize the coding gain of SCC for screen content images, save the code rate, and provide better reference frames for P frames in the GOP and improve the picture quality.

[0118] The picture mutation can occur in application scenarios such as video conferencing or desktop sharing, such as PPT page turning, document scrolling, etc.

[0119] S604, determining that the first screen content image is the first frame type.

[0120] After S604 is executed, S605 is executed to obtain the first code stream.

[0121] S605, first encoding the first screen content image to obtain the first code stream.

[0122] The explanation of S605 can refer to the specific explanation of S402 above, and will not be repeated here.

[0123] S606, determining that the first screen content image is of a second frame type.

[0124] After S606 is performed, S607 below is performed to obtain a second code stream.

[0125] S607, inputting the first screen content image into an encoding chip, the encoding chip being configured to encode the first screen content image to obtain a second code stream.

[0126] The explanation of S607 can refer to the specific explanation of S403 above, and will not be repeated here.

[0127] In this embodiment, by comparing the size relationship between M and L-2, it can be determined whether the first screen content image and the previous key frame belong to the same GOP. If they do not belong to the same GOP from the length of the GOP, the first screen content image is determined to be of a first frame type, and the first screen content image of the first frame type is encoded. If they should belong to the same GOP from the length of the GOP, it is further determined whether the first screen content image has a picture mutation according to the second screen content image. If the first screen content image has a picture mutation, the first screen content image is determined to be of the first frame type, and the first screen content image of the first frame type is encoded. If the first screen content image does not have a picture mutation, the first screen content image is determined to be of a second frame type, and the first screen content image of the second frame type is encoded. In this way, the image quality of the key frame is ensured by the first encoding, and the coding power consumption of the non-key frame is reduced by the second encoding, so that the image quality of the screen content image can be ensured while the coding power consumption and the delay are reduced. Moreover, by detecting whether the first screen content image has a picture mutation, the first encoding is used when the picture mutation occurs to ensure the image quality of the picture mutation.

[0128] Figure 8 A schematic diagram of the image encoding and decoding method provided by the embodiment of the present application is shown in FIG. 1. Figure 8As shown, the encoding end can be a sending end of the code stream, and the decoding end can be a receiving end of the code stream, and the code stream is transmitted between the two through a network. The original image is collected by a camera or captured from an image display unit at the encoding end. The original image can be an RGB image. Color conversion is performed on the original image to obtain a YUV image. Then, content detection is performed by the CPU of the encoding end to determine whether to encode through way 1, way 2 or way 3. First, the content detection can include scene recognition, for example, by recognizing the image source to determine whether the YUV image is a screen content image or a natural scene image. When the YUV image is collected by the camera, the YUV image is a natural scene image, and is encoded through way 2. Specifically, hard coding is performed by the SoC of the encoding end to obtain a second code stream. For example, the second code stream can be a protocol H.265 code stream. When the YUV image is captured from the image display unit, the YUV image is a screen content image, and is encoded through way 1 or way 3. Specifically, the content detection can further include picture mutation detection. According to the length of the GOP and whether the picture mutation occurs, the frame type of the screen content image is determined, and the screen content images of different frame types are encoded through different ways. When the screen content image is an I frame, it is encoded through way 1. Specifically, soft coding is performed by the CPU of the encoding end to obtain a first code stream. The first code stream can also be referred to as an SCC code stream. When the screen content image is a P frame, it is encoded through way 3. Specifically, hard coding is performed by the SoC of the encoding end to obtain a second code stream.

[0129] The I frame described above includes but is not limited to a picture mutation frame (for example, a page turning frame), and the P frame includes a relatively static frame. The encoding end transmits the SCC code stream and / or the protocol H.265 code stream to the decoding end through the network. The decoding end can perform soft decoding on the SCC code stream to obtain a reconstructed image. The decoding end can perform hard decoding on the protocol H.265 code stream to obtain a reconstructed image. The reconstructed image can be a YUV image, and then color correction is performed to convert it into an RGB image for display.

[0130] Thus, the image quality of the key frame is guaranteed by SCC soft coding, and the coding power consumption of the non-key frame is reduced by SoC hard coding, so that the image quality of the screen content image can be guaranteed while the coding power consumption and delay are reduced. Moreover, whether the picture mutation of the screen content image occurs is detected, and when the picture mutation occurs, SCC soft coding is adopted to improve the picture quality of the SCC.

[0131] The image encoding method of the embodiment of the present application is described in detail above in combination with the drawings. The image decoding method of the embodiment of the present application is described in detail below in combination with the drawings. Figure 9The image coding apparatus of the embodiments of the present application is introduced. It should be understood that the image coding apparatus can perform the image coding method of the embodiments of the present application. In order to avoid unnecessary repetition, the repeated description is appropriately omitted when the image coding apparatus of the embodiments of the present application is introduced below.

[0132] Reference is made to Figure 9 , Figure 9 The structure schematic diagram of an image coding apparatus provided by the embodiments of the present application is shown. As shown in Figure 9 , the image coding apparatus 900 can include an acquisition module 901 and an encoding module 902.

[0133] The acquisition module 901 is configured to acquire a first screen content image.

[0134] The encoding module 902 is configured to perform first encoding on the first screen content image to obtain a first code stream if the first screen content image is a first frame type, the first code stream including encoded data of the first screen content image, the first frame type being used to represent a key frame. The first screen content image is input into an encoding chip if the first screen content image is a second frame type, the encoding chip being configured to perform second encoding on the first screen content image to obtain a second code stream, the second code stream including encoded data of the first screen content image, the second frame type being used to represent a non-key frame.

[0135] In some embodiments, the encoding module 902 is further configured to determine whether the first screen content image has a picture mutation according to a second screen content image if M≤L-2, M representing a frame number of intervals between the first screen content image and a previous key frame, L representing a length of a group of pictures, the second screen content image being a previous frame image of the first screen content image; determine the first screen content image as the first frame type if the first screen content image has the picture mutation; and determine the first screen content image as the second frame type if the first screen content image does not have the picture mutation.

[0136] In some embodiments, if the first screen content image is the first frame type, the acquisition module 901 is further configured to acquire a third screen content image, the third screen content image being a frame image after the first screen content image, and the third screen content image and the first screen content image belonging to a same group of pictures, the third screen content image being the second frame type.

[0137] The encoding module 902 is further configured to input the third screen content image and a reconstructed frame into the encoding chip, the encoding chip being further configured to perform the second encoding on the third screen content image by taking the reconstructed frame as a reference frame, the second code stream further including encoded data of the third screen content image.

[0138] The reconstructed frame is a reconstructed image obtained by decoding the encoded data of the first screen content image.

[0139] In some embodiments, the acquisition module 901 is further configured to acquire a first image, perform scene recognition on the first image, and determine, according to a scene recognition result of the first image, whether the first image is the first screen content image or a first natural scene image.

[0140] In some embodiments, if the first image is the first natural scene image, the encoding module 902 is further configured to input the first natural scene image into the encoding chip, and the encoding chip is further configured to perform the second encoding on the first natural scene image.

[0141] In some embodiments, the first frame type includes an I frame, and the second frame type includes a P frame.

[0142] It should be noted that the image encoding apparatus 900 can perform the encoding method of any of the embodiments shown in the above description. Figure 4 or Figure 6 or Figure 8 The specific implementation principles and technical effects can be referred to the specific explanation and description of the above method embodiments, which will not be described here.

[0143] Based on the same inventive concept as the above method, the embodiments of the present application provide an image encoding apparatus, comprising: one or more processors; a memory configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement part or all of the steps of the image encoding method described in one or more of the above embodiments.

[0144] Based on the same inventive concept as the above method, the embodiments of the present application provide a computer readable storage medium, which stores program codes, wherein the program codes comprise instructions for executing part or all of the steps of the image encoding method described in one or more of the above embodiments.

[0145] Based on the same inventive concept as the above method, the embodiments of the present application provide a computer program product, when the computer program product runs on a computer, the computer program product makes the computer execute part or all of the steps of the image encoding method described in one or more of the above embodiments.

[0146] The image encoding apparatus, the computer readable storage medium, and the computer program product provided by the embodiments of the present application are used to execute the corresponding method provided above, and thus the beneficial effects achieved thereby can be referred to the beneficial effects of the corresponding method provided above, which will not be described here.

[0147] Through the description of the above embodiments, those skilled in the art can understand that, for the convenience and brevity of description, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0148] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0149] The units described as separate components can or can not be physically separated, and the components shown as units can be one physical unit or multiple physical units, that is, can be located in one place or can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0150] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0151] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or the parts that make contributions to the prior art or all or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method of the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage medium that can store program codes.

[0152] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image encoding method, characterized in that, include: Get the image of the first screen content; If the first screen content image is of the first frame type, then the first screen content image is first encoded to obtain a first bitstream. The first bitstream includes the encoded data of the first screen content image, and the first frame type is used to represent a keyframe. If the first screen content image is of the second frame type, the first screen content image is input into the encoding chip. The encoding chip is used to perform a second encoding on the first screen content image to obtain a second bitstream. The second bitstream includes the encoded data of the first screen content image. The second frame type is used to represent a non-keyframe. If the first screen content image is of the first frame type, the method further includes: Acquire a third screen content image, wherein the third screen content image is a frame image following the first screen content image, and the third screen content image belongs to the same image group as the first screen content image, and the third screen content image is of the second frame type; The third screen content image and the reconstructed frame are input into the encoding chip. The encoding chip is also used to use the reconstructed frame as a reference frame to perform the second encoding on the third screen content image. The second bitstream also includes the encoded data of the third screen content image. The reconstructed frame is a reconstructed image obtained by decoding the encoded data of the first screen content image.

2. The method according to claim 1, characterized in that, The method further includes: If M≤L-2, then based on the second screen content image, determine whether the first screen content image has undergone a sudden change in image; M represents the number of frames between the first screen content image and the previous keyframe, L represents the length of the image group, and the second screen content image is the previous frame image of the first screen content image. If the first screen content image undergoes a sudden change in appearance, then the first screen content image is determined to be of the first frame type; If the first screen content image does not undergo a sudden change in image quality, then the first screen content image is determined to be of the second frame type.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Get the first image; Scene recognition is performed on the first image, and based on the scene recognition result of the first image, the first image is determined to be either the first screen content image or the first natural scene image.

4. The method according to claim 3, characterized in that, If the first image is the first natural scene image, the method further includes: The first natural scene image is input into the encoding chip, and the encoding chip is also used to perform the second encoding on the first natural scene image.

5. The method according to claim 1, characterized in that, The first frame type includes I-frames, and the second frame type includes P-frames.

6. An image encoding device, characterized in that, include: The acquisition module is used to acquire the image of the content on the first screen; The encoding module is used to perform a first encoding on the first screen content image if the first screen content image is of the first frame type, to obtain a first bitstream, wherein the first bitstream includes the encoded data of the first screen content image, and the first frame type is used to represent a keyframe. If the first screen content image is of the second frame type, the first screen content image is input into the encoding chip. The encoding chip is used to perform a second encoding on the first screen content image to obtain a second bitstream. The second bitstream includes the encoded data of the first screen content image. The second frame type is used to represent a non-keyframe. If the first screen content image is of the first frame type, the acquisition module is further used to acquire a third screen content image, wherein the third screen content image is a frame image following the first screen content image, and the third screen content image belongs to the same image group as the first screen content image, and the third screen content image is of the second frame type. The encoding module is further configured to input the third screen content image and the reconstructed frame into the encoding chip, and the encoding chip is further configured to use the reconstructed frame as a reference frame to perform the second encoding on the third screen content image, and the second bitstream also includes the encoded data of the third screen content image; The reconstructed frame is a reconstructed image obtained by decoding the encoded data of the first screen content image.

7. The apparatus according to claim 6, characterized in that, The encoding module is also used for: If M≤L-2, then based on the second screen content image, determine whether the first screen content image has undergone a sudden change in image; M represents the number of frames between the first screen content image and the previous keyframe, L represents the length of the image group, and the second screen content image is the previous frame image of the first screen content image. If the first screen content image undergoes a sudden change in appearance, then the first screen content image is determined to be of the first frame type; If the first screen content image does not undergo a sudden change in image quality, then the first screen content image is determined to be of the second frame type.

8. The apparatus according to claim 6 or 7, characterized in that, The acquisition module is also used for: Get the first image; Scene recognition is performed on the first image, and based on the scene recognition result of the first image, the first image is determined to be either the first screen content image or the first natural scene image.

9. The apparatus according to claim 8, characterized in that, If the first image is the first natural scene image, the encoding module is further configured to: input the first natural scene image into the encoding chip, and the encoding chip is further configured to perform the second encoding on the first natural scene image.

10. The apparatus according to claim 6, characterized in that, The first frame type includes I-frames, and the second frame type includes P-frames.

11. An image encoding device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

12. A computer program product, characterized in that, Includes a computer program that, when run on a computer, causes the computer to perform the image encoding method as described in any one of claims 1-5.

13. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on a computer, cause the computer to perform the image encoding method as described in any one of claims 1-5 to obtain a bitstream.

Images