Image processing method, device, electronic device and storage medium

Abstract

The present application provides an image processing method, apparatus, device and storage medium, wherein the method comprises: obtaining change data, wherein the change data is display data obtained by processing a first data stream by an image renderer, and the change data is used to characterize the difference from a previous frame of image; wherein the first data stream comprises graphic data of multiple frames of images; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output; based on a target encoding strategy, the change data is processed to obtain encoded data to form a second data stream, wherein the second data stream comprises encoded data of multiple frames; and the second data stream is transmitted, wherein the second data stream is at least used for decoding and displaying by a terminal device receiving the second data stream.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] When two or more terminals are sharing the screens of displayed content, the shared display content is encoded and transmitted from one terminal to the other terminals, taking transmission security into consideration. Taking images as an example, since the displayed image is composed of multiple pixels, encoding the displayed image means encoding each pixel individually, which is time-consuming. Therefore, the scheme of encoding and transmitting each frame of the displayed image one by one in the related art undoubtedly increases the transmission latency of the shared content. Summary of the Invention

[0003] The present application provides an image processing method, apparatus, device and storage medium to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present application, there is provided an image processing method, comprising:

[0005] Obtaining change data, wherein the change data is display data obtained by processing the first data stream by the image renderer, and the change data is used to represent a difference from a previous frame of image; wherein the first data stream includes graphic data of multiple frames of image; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output;

[0006] Based on a target encoding strategy, the changed data is processed to obtain encoded data to form a second data stream, wherein the second data stream includes encoded data of multiple frames;

[0007] The second data stream is transmitted, and the second data stream is at least used for decoding and displaying by a terminal device that receives the second data stream.

[0008] In one possible implementation, a target encoding strategy is determined from preset encoding strategies based on parameter information of the change data; wherein the parameter information represents the data size of the change data and / or the size of the position occupied by the change data.

[0009] In one possible implementation manner, determining a target encoding strategy from preset encoding strategies based on the parameter information of the change data includes:

[0010] If the parameter information satisfies the first condition, the first coding strategy in the preset coding strategy is determined to be the target coding strategy; if the parameter information satisfies the second condition, the second coding strategy in the preset coding strategy is determined to be the target coding strategy.

[0011] In one possible implementation manner, the first encoding strategy in the preset encoding strategy is to encode the change data through the first channel of the video codec and to encode the position of the change data through a second channel different from the first channel; the second encoding strategy in the preset encoding strategy is to encode the change data through the first channel of the video codec.

[0012] In one possible implementation, the change data is obtained when a refresh operation is performed on adjacent frames in a multi-frame image; wherein the refresh operation is an operation for different data in the adjacent frames.

[0013] In one embodiment, the second data stream includes multiple frames of encoded data, and at least two frames have different encoding strategies.

[0014] In one possible implementation manner, the first frame image in the first data stream is encoded using a second encoding strategy among preset encoding strategies, wherein the first frame image is obtained by processing the graphic data of the first frame image through an image renderer.

[0015] According to a second aspect of the present application, there is provided an image processing apparatus, comprising:

[0016] a first obtaining unit, configured to obtain change data, wherein the change data is display data obtained by processing the first data stream by the image renderer, and the change data is used to represent a difference from a previous frame of image; wherein the first data stream includes graphic data of multiple frames of image; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output;

[0017] a second obtaining unit, configured to process the changed data based on a target encoding strategy to obtain encoded data to form a second data stream, wherein the second data stream includes encoded data of multiple frames;

[0018] The transmission unit is used to transmit the second data stream, where the second data stream is at least used for decoding and displaying by a terminal device that receives the second data stream.

[0019] According to a third aspect of the present application, an electronic device is provided, including:

[0020] at least one processor; and

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0023] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application.

[0024] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an illustrative and non-limiting manner, in which:

[0026] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0027] Figure 1 The following is a schematic diagram of the implementation process of the image processing method of the embodiment of the present application Figure 1 ;

[0028] Figure 2 The following is a schematic diagram of the implementation process of the image processing method of the embodiment of the present application Figure 2 ;

[0029] Figure 3 A schematic diagram showing the corresponding video codec and encoding strategy of an embodiment of the present application is shown;

[0030] Figure 4 A schematic diagram of a channel of a video codec according to an embodiment of the present application is shown;

[0031] Figure 5 A schematic diagram of an application scenario of the first embodiment of the present application is shown;

[0032] Figure 6 A schematic diagram of the internal structure of a terminal according to an embodiment of the present application is shown;

[0033] Figure 7 A processing diagram of an image renderer according to an embodiment of the present application is shown;

[0034] Figure 8 A schematic diagram showing the structure of an image processing device according to an embodiment of the present application is shown;

[0035] Figure 9 A schematic diagram of the structure of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0037] The processing logic of the image processing method of the embodiments of the present application can be deployed in any reasonable electronic device. The electronic device can be a terminal or a server. Terminals include, but are not limited to, tablet computers, all-in-one computers, desktop computers, mobile phones, smart speakers, smart watches, etc. Servers include ordinary servers and cloud servers, etc.

[0038] In the image processing method of the present application, the change data is obtained based on the processing of the first data stream by the image renderer, which can achieve rapid acquisition of the change data, and only encode and transmit the change data in the first data stream. Based on this, it can effectively reduce the encoding time, reduce the transmission delay, and achieve efficient transmission.

[0039] This application provides a first embodiment of an image processing method, such as Figure 1 As shown, the method includes:

[0040] S101: Obtaining change data, wherein the change data is display data obtained by processing a first data stream through an image renderer, and the change data is used to represent a difference from a previous frame of image; wherein the first data stream includes graphic data of multiple frames of image; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output;

[0041] In the embodiment of the present application, if the terminal that shares the content to be displayed with other terminals is regarded as the first terminal and the other terminals are regarded as the second terminal, then the first data stream can be regarded as each frame of pictures or images that the first terminal needs to share with the second terminal.

[0042] The first data stream is a stream data including graphic data of a first frame image, graphic data of a second frame image, ..., graphic data of an Nth frame image, where N is a positive integer greater than or equal to 2.

[0043] In this application, an image renderer is provided in the first terminal. The image renderer can process the graphic data of each frame of an image to obtain each frame of an image that needs to be shared between the first terminal and the second terminal. In a specific implementation, the image renderer can be a module located in a central processing unit (CPU), a module located in a graphics processing unit (GPU), or a module located in other suitable processors or chips.

[0044] The graphic data of each frame image can be the patterns of various graphics (such as points, surfaces, lines, etc.) that constitute each frame image, as well as information such as the position, size, and color of each pattern. For the concept of graphic data, please refer to the relevant description and will not be repeated here.

[0045] The image renderer's processing may include: the image renderer may draw each frame of image according to the graphic data of each frame of image, thereby obtaining each frame of image presented to the user via the display screens of the first terminal and the second terminal. Alternatively, the graphic data in each frame of image may be divided into layers, each layer including one or more graphic data of the same frame of image, and the layers may be synthesized according to information such as the position and size of the graphic data in each layer, thereby obtaining each frame of image.

[0046] In implementation, the change data may be obtained by using at least one of the following methods:

[0047] Method 1: The image renderer compares the differences between two adjacent frames to obtain the change data of the i+1th frame (the current frame or the next frame) relative to the ith frame (the previous frame). Where i is a positive integer greater than or equal to 1 and less than or equal to (N-1).

[0048] Method 2: The image renderer can obtain the difference between two adjacent frames, such as the difference between the i+1th frame and the ith frame, from an external source. The image renderer can then draw or synthesize the image based on the externally obtained graphical data of the difference, thereby obtaining the change data.

[0049] Method 3: The different graphic data are drawn or synthesized outside the image renderer, and the image renderer reads or intercepts the changed data.

[0050] Method 4: The image renderer draws or synthesizes an image based on the graphic data of the i+1th frame image to obtain the i+1th frame image. The image renderer externally specifies the position of the i+1th frame image where the i+1th frame image differs from the ith frame image. Based on the position of the i+1th frame image where the i+1th frame image differs, data is intercepted from the i+1th frame image obtained by the image renderer. The intercepted data is the change data.

[0051] In methods 2 to 4, the external part of the image renderer can be any other device or application (APP) in the first terminal that is independent of the image renderer and can know the difference between two adjacent frames of images. For example, the external part of the image renderer is an APP, and the multiple frames of images in the first data stream are data that need to be displayed in the APP. The APP can know the difference between two adjacent frames of images displayed by itself. The APP can be an application such as a conference, social, or screen projection application.

[0052] In method one, the difference between two adjacent image frames is determined by the image renderer through comparison. In methods two to four, the difference between two adjacent image frames is externally determined, and the image renderer can obtain the change data without comparison, making the acquisition of change data more direct and simple. Compared with methods that require comparison to obtain change data, this direct and simple method can greatly shorten the image sharing process from the first terminal to the second terminal by directly obtaining change data, thereby reducing the transmission delay of the shared image and achieving efficient transmission.

[0053] It should be noted that in Methods 1 to 4, the change data between adjacent frames in the first data stream in this application are all obtained based on the image renderer. Since the image renderer itself has the characteristics of fast processing speed and high accuracy, the change data obtained based on the image renderer is not only fast, effectively simplifies the process of obtaining the change data, but also can ensure the accuracy of the change data.

[0054] In the first to fourth methods, it is preferred to adopt one of the second to fourth methods to obtain the change data. Among the second to fourth methods, the fourth method is preferred as the implementation method for obtaining the change data.

[0055] S102: Processing the changed data based on a target encoding strategy to obtain encoded data to form a second data stream, where the second data stream includes encoded data of multiple frames;

[0056] In S102, the changed data may be processed by an encoder of the electronic device. For example, the encoder may be a video codec.

[0057] In the embodiments of the present application, two or more encoding strategies are provided for processing, such as encoding, the changing data. The target encoding strategy is one of the preset encoding strategies. The target encoding strategy is used to encode the changing data based on a preset codec algorithm to obtain encoded data. The preset codec algorithm can be any reasonable image codec algorithm, such as H.264, H.263, MPEG, etc.

[0058] For the first, second, ..., Nth frames of image in the first data stream, in practical applications, two adjacent frames of image may be different or the same. Based on this, if at least some of the two adjacent frames starting from the second frame of image are different, the second data stream includes the encoded data of the first frame of image and the encoded data of the differences between the two adjacent frames of image starting from the second frame of image.

[0059] For example, if the first and second frames are identical, the third frame is different from the second frame, and the fourth frame is different from the third frame, the first data stream includes the encoded data of the first frame, the encoded data of the changes in the third frame relative to the second frame, and the encoded data of the changes in the fourth frame relative to the third frame. The encoded data of each frame constitutes the second data stream.

[0060] S103: Transmit the second data stream, where the second data stream is at least used for decoding and displaying by a terminal device that receives the second data stream.

[0061] In this step, the first terminal transmits the second data stream to the second terminal. The second terminal, as a terminal device receiving the second data stream, uses a preset encoding and decoding algorithm to decode the received second data stream and display each decoded frame image.

[0062] In a specific implementation, all frames in the first data stream can be encoded and then transmitted from the first terminal to the second terminal, which then decodes all frames and displays each decoded frame one by one. Alternatively, the first frame of the first data stream can be encoded and transmitted from the first terminal to the second terminal, which then decodes and displays the first frame. Starting from the second frame, if there are differences between two adjacent images, the encoded data of the differences is transmitted to the second terminal, which decodes and displays the decoded differences together with the data that has not changed compared to the previous frame.

[0063] In S101-S103, the first terminal obtains the change data of two adjacent frames in the first data stream based on the image renderer. This simplifies the process of obtaining the change data, allowing for faster acquisition and ensuring a certain degree of acceleration in the transmission process. Furthermore, a target encoding strategy is employed to encode the change data, and a second data stream including the encoded data of the change data is transmitted to the second terminal. Compared to related technologies, this method only requires transmitting the encoded data of the change data, eliminating the need to transmit the encoded data of each frame. This effectively reduces encoding time, significantly speeds up the transmission process, reduces transmission latency, and achieves efficient transmission.

[0064] In some embodiments, there are two or more preset coding strategies. The coding strategy of the two or more preset coding data used to encode the change data between each two adjacent frames can be implemented according to the following scheme, see Figure 2 After S101 and before S102, the method further includes:

[0065] S104: Determine a target encoding strategy from preset encoding strategies based on parameter information of the changed data; wherein the parameter information represents the data size of the changed data and / or the size of the position occupied by the changed data.

[0066] Furthermore, the parameter information may be the size of the amount of the change data of the (i+1th frame image) relative to the i-th frame image as a percentage of the total data amount of the displayed image (of the (i+1th frame or the i-th frame), e.g., the amount of the change data accounts for 5% or 10% of the total data amount. The parameter information may also be the size of the area on the display screen used to display each frame image where the change data of the (i+1th frame image) relative to the i-th frame image is located as a percentage of the entire display screen (or the area on the display screen where the (i+1th frame image) is located, or the area on the display screen where the i-th frame image is located), e.g., the area on the display screen where the change data is located accounts for 10% or 15% of the area of ​​the entire display screen.

[0067] The above is an explanation of the parameter information of the change data from two perspectives: the size of the change data and / or the size of the position occupied by the change data. Any other reasonable parameter that can be used to evaluate the size of the change data can be used as the parameter information of this application.

[0068] It can be understood that the parameter information of the change data can reflect the actual change between two adjacent frames of images, and which encoding strategy among the preset encoding strategies to use as the target encoding strategy is determined according to the actual change.

[0069] In practical applications, the actual changes between two adjacent frames of images can be either large or small. In the case of small changes, only the change data can be encoded. In the case of large changes, the entire image is used as the change data and encoded, thereby achieving adaptive encoding based on the actual change situation.

[0070] If encoding only the changed data when the change is small is considered as the first encoding strategy in the preset encoding strategy, encoding the entire image when the change is large can be considered as the second encoding strategy in the preset encoding strategy.

[0071] In a specific implementation, the preset first encoding strategy may specifically be: encoding the change data when the ratio of the amount of the change data to the total amount of data does not reach the first threshold (or the amount of the change data does not reach the third threshold), and / or when the ratio of the area of ​​the area where the change data is located on the display screen to the area of ​​the entire display screen does not reach the second threshold (or the area of ​​the area where the change data is located on the display screen does not reach the fourth threshold). The second encoding strategy may specifically be: encoding the entire image when the ratio of the amount of the change data to the total amount of data reaches the first threshold (or the amount of the change data reaches the third threshold), and / or when the ratio of the area of ​​the area where the change data is located on the display screen to the area of ​​the entire display screen reaches the second threshold (or the area of ​​the area where the change data is located on the display screen reaches the fourth threshold). The first to fourth thresholds are flexibly set according to actual conditions.

[0072] When applied, the parameter information of the changed data is calculated to determine whether the parameter information of the changed data meets the first condition or the second condition. If the parameter information meets the first condition, the first coding strategy in the preset coding strategy is determined to be the target coding strategy; if the parameter information meets the second condition, the second coding strategy in the preset coding strategy is determined to be the target coding strategy.

[0073] Among them, the first condition may be a condition that the parameter information of the changing data must meet when the first encoding strategy is used to encode the changing data. For example, the first condition is: the ratio of the amount of the changing data to the total amount of data does not reach the first threshold, the amount of the changing data does not reach the third threshold, the area ratio of the area where the changing data is located on the display screen to the entire display screen does not reach the second threshold, and the area of ​​the area where the changing data is located on the display screen does not reach the fourth threshold. The second condition may be a condition that the parameter information of the changing data must meet when the second encoding strategy is used to encode the changing data. For example, the second condition is: the ratio of the amount of the changing data to the total amount of data reaches the first threshold, the amount of the changing data reaches the third threshold, the area ratio of the area where the changing data is located on the display screen to the entire display screen reaches the second threshold, and the area of ​​the area where the changing data is located on the display screen reaches the fourth threshold. The first and second conditions may also be any other reasonable circumstances.

[0074] Exemplarily, if the ratio of the change data of the i+1th frame image relative to the i-th frame image to the total data volume does not reach a first threshold, only the change data is encoded, and the encoded data of the change data is transmitted to the second terminal, so that after decoding, the second terminal displays the image together with the data of the i+1th frame image that has not changed relative to the i-th frame image. If the ratio of the area on the display screen where the change data of the i+1th frame image relative to the i-th frame image is located to the area of ​​the entire display screen reaches a second threshold, the i+1th frame image is used as the change data, the i+1th frame image is encoded, and the encoded i+1th frame image is transmitted to the second terminal, which decodes the encoded image and displays the i+1th frame image.

[0075] This method determines the target encoding strategy for the changed data based on its parameter information, enabling flexible encoding based on the actual changes in the data. Compared to related art approaches that encode the entire image, this flexible selection of encoding strategies based on the actual changes in the data simplifies the encoding process, shortens encoding time, and enables efficient transmission of the encoded data from the first terminal to the second terminal.

[0076] In the embodiment of the present application, a video codec can be used to implement encoding and decoding of images. Considering that there are two or more preset encoding strategies, targeted encoding can be implemented according to the actual changes in the change data. Taking the preset encoding strategies as two as an example, in the embodiment of the present application, M video codecs can be set, where the number of M is consistent with the number of preset encoding strategies, such as Figure 3 M is a positive integer greater than or equal to 2. Each of the M video codecs encodes the change data using a corresponding encoding strategy among the M preset encoding strategies.

[0077] For example, if the preset encoding strategies include the aforementioned first and second encoding strategies, then the number of video codecs may be set to two: codec 1 and codec 2. If the first encoding strategy is determined as the target encoding strategy based on the parameter information of the change data, one of the two video codecs may be used to encode the change data. If the second encoding strategy is determined as the target encoding strategy based on the parameter information of the change data, the other of the two video codecs may be used to encode the entire image.

[0078] In addition to encoding the image using M video codecs consistent with the preset encoding strategy, the same video codec can also be used to encode the change data or the entire image. Considering that if the change data is small, the position of the change data on the display screen needs to be transmitted from the first terminal to the second terminal so that the first and second terminals know the change position. The change data needs to be encoded before transmission, and the position of the change data on the display screen (the change position) can be encoded before transmission or transmitted directly without encoding.

[0079] Taking the position of the changed data on the display screen as an example, which needs to be encoded before transmission, the encoding of the changed data and the changed position is realized. Figure 4 As shown, the same video codec can be configured to include two channels: a first channel and a second channel. One of the two channels can be configured to encode the change data, while the other channel can be configured to encode the change position (position information). When the change data is small, that is, when encoding is performed using the first encoding strategy, the change data can be encoded via the first channel of the video codec, and the position of the change data can be encoded via a second channel different from the first channel. When the change data is large, that is, when encoding is performed using the first encoding strategy, the entire image serving as the change data can be encoded via the first channel of the video codec.

[0080] The aforementioned encoding process for both the small and large change scenarios using the same video codec can be implemented using preset encoding strategies. For example, a first encoding strategy in the preset encoding strategies is to encode the change data using the first channel of the video codec and to encode the location of the change data using a second channel different from the first channel; a second encoding strategy in the preset encoding strategies is to encode the change data using the first channel of the video codec.

[0081] It is understood that the first and second encoding strategies in the preset encoding strategies in the embodiments of the present application can not only indicate whether the data to be encoded is partial data (a portion of an image) or full data (e.g., the entire image), but also indicate which channel to use for encoding. Encoding the data to be encoded and the channel indicated in the encoding strategy can greatly simplify the encoding process, reduce encoding time, and thus achieve efficient transmission of the encoded data from the first terminal to the second terminal.

[0082] The independence between channels of the same video codec ensures that the change data and position information are encoded accurately, thereby achieving accurate transmission of the encoded data.

[0083] When using the same video codec to encode changing data, different channels are used to encode the changing data and the changing location, thereby achieving targeted encoding of different data (changing data and changing location). Targeted encoding can shorten the encoding process and effectively reduce encoding time.

[0084] In some embodiments, when adjacent frames in the first data stream differ (in data), the operating system (OS), specifically the engine layer, of the first terminal generates a refresh operation based on the difference between the (i+1)th frame and the (i)th frame, thereby displaying the adjacent frames one by one. From a user's perspective, this refresh operation can be considered as replacing the data displayed at the same location in the (i)th frame as the difference with the data in the (i+1)th frame relative to the (i)th frame.

[0085] Based on this, an embodiment of the image processing method of the present application further includes: when a refresh operation is performed on adjacent frames in a multi-frame image, the refresh operation is an operation on different data in the adjacent frames, and the change data is obtained using any of the aforementioned methods 1 to 4. In this case, the refresh operation provides an opportunity for obtaining the change data, and the change data is obtained when the refresh operation is triggered, thereby making it easier to obtain the change data.

[0086] Furthermore, since the refresh operation is performed on different data in adjacent frames, the different data targeted by the refresh operation can be directly used as the changed data. That is, the changed data can be directly obtained based on the refresh operation, which is more direct and simplifies the process of obtaining the changed data.

[0087] It is understood that the second data stream includes the encoded data of the first frame image and the encoded data of the change data between two adjacent frames of image that are different starting from the second frame image. The first frame image in the first data stream, as the first image displayed or presented by the first terminal and / or the second terminal, is encoded using the second encoding strategy among the preset encoding strategies, that is, the entire first frame image is encoded to obtain the encoded data of the first frame image. The first frame image is obtained by processing the graphic data of the first frame image through the image renderer. For details, please refer to the relevant description above.

[0088] In practical applications, if the parameter information of the change data of two adjacent frames starting from the second frame may meet the first condition or the second condition, the first encoding strategy should be used to encode the change data if the first condition is met. If the second condition is met, the second encoding strategy should be used to encode the change data.

[0089] Exemplarily, if the parameter information of the change data of the third frame relative to the second frame meets the first condition, the first encoding strategy is adopted to encode the change data. If the parameter information of the change data of the fourth frame relative to the third frame meets the second condition, the second encoding strategy is adopted to encode the change data. Based on this, the second data stream is a kind of stream data, and the second data stream includes multiple frames of encoded data, and the encoding strategies of at least two frames are different. Which encoding strategy is adopted for the two adjacent frames of data in the 1st to Nth frames of this first data stream needs to be determined according to the parameter information of the change data. Determining the encoding strategy of the change data of the two adjacent frames in the 2nd to Nth frames according to the parameter information of the change data can make the encoding strategies of at least two frames in the encoded data of multiple frames different, which has strong practical significance. Determining the target encoding strategy based on the parameter information of the change data is easy to be promoted in engineering and has high credibility.

[0090] The following combination Figure 5-Figure 7 The following further illustrates the technical solution of the embodiment of the present application.

[0091] exist Figure 5 The application scenario shown uses video sharing between two terminals as an example. Video sharing can be achieved through screen projection applications, conferencing applications, or chat applications. During video sharing, the images on both terminals remain consistent. The first of the two terminals acts as the sender of the shared video, and the second terminal acts as the receiver of the shared video.

[0092] From the inside of the terminal, Figure 6 As shown, the first terminal includes an application layer, an engine layer, and a bottom layer. The application layer is used to provide screen projection applications, conference applications, or chat applications that can share videos. The engine layer is used to implement the image processing method of the embodiment of the present application. The bottom layer is the hardware layer, which provides hardware support for the image processing method of the embodiment of the present application.

[0093] Taking the example of the first terminal starting a screen projection application and sharing a video through the screen projection application, for the upper layer of the first terminal, such as the application layer, for the first frame image of the shared video, a second encoding strategy can be adopted, and a preset codec algorithm is used to encode the entire image through the first channel of the video codec to obtain the encoded data of the first frame image. The encoded data of the first frame image is sent from the first terminal to the second terminal. The second terminal receives the encoded data of the first frame image, decodes the encoded data using the preset codec algorithm, and displays it on its own display screen, such as Figure 5 As shown in the display screen, two terminals display the same screen.

[0094] Starting from the second frame image of the shared video, the application layer, such as the activated screen projection application, will know whether the two adjacent frames in the shared video are the same picture (image) or different pictures. If the application layer learns that the i+1th frame is different from the i-th frame, a refresh request is generated and the refresh request is sent to the engine layer. When the engine layer receives the refresh request, it responds to the refresh request and performs a refresh operation. The execution of the refresh operation by the engine layer is equivalent to executing an update of the shared screen or image in the two terminals, such as updating the i-th frame to the i+1th frame. Specifically, this update (refresh operation) of the engine layer replaces the data in the i-th frame corresponding to the position of the aforementioned difference in the i+1th frame with the difference between the i+1th frame and the i-th frame. In other words, the refresh operation is an operation on the different data of two adjacent frames of images. In layman's terms, the refresh operation brushes the changed part of the i+1th frame relative to the i-th frame, such as the changed content.

[0095] If the difference between the i+1th frame and the ith frame is considered data refreshed by the engine layer, the engine layer can directly use the refreshed data as the change data of the i+1th frame relative to the ith frame. This method of obtaining change data is simple and easy, and can speed up the acquisition of change data.

[0096] Alternatively, the engine layer can also extract the data at position 1 from the i-th frame image processed by the image renderer based on the calculated position of the difference between the i+1-th frame image and the i-th frame image, and use the extracted data as the change data. Specifically, in this application scenario, the entire image including the refreshed data, such as the i+1-th frame image, is stored in the bottom layer. Based on the refresh content and the entire image content known from the bottom layer, the engine layer can know the area where the refresh content is located - the refresh area. Based on the refresh area and the entire image area, the coordinates of the difference between the i+1-th frame image and the i-th frame image in the entire i+1-th frame image are calculated, thereby obtaining the changed position of the difference between the i+1-th frame image and the i-th frame image in the i+1-th frame image. The engine layer extracts the data at the coordinates from the i-th frame image processed by the image renderer as the change data.

[0097] In the above solution, the refresh operation of the engine layer is equivalent to providing an opportunity for obtaining the changed data. The changed data is obtained when the refresh operation is triggered, thereby making it easier to obtain the changed data.

[0098] In this application scenario, the image renderer processes the graphic data of the i+1th frame image to obtain the i+1th frame image. It can be understood that within the first terminal, the display screen presented by the terminal at a certain moment is obtained by synthesizing several image layers such as the terminal's desktop screen, the screen of the application presenting the content, and the mouse screen. The image renderer uses the desktop screen at a certain sharing moment (such as the i+1th moment), the screen presenting the content at the sharing moment, and the mouse screen at the sharing moment as various image layers. Figure 7 As shown, layer 1 is the desktop screen layer, layer 2 is the screen layer for application presentation, and layer 3 is the mouse screen layer. The image renderer synthesizes the images of all the image layers according to the graphic data of the screen in each image layer, thereby obtaining the image (the i-th frame image or screen) that the end user sees on the display screen of the first terminal.

[0099] The image renderer can obtain the (i+1)th frame image according to the aforementioned image layer synthesis method. Based on the coordinates of the difference between the (i+1)th frame image and the (i)th frame image, the data at the coordinates in the (i+1)th frame image obtained by the image renderer is intercepted, and the intercepted data is used as the change data of the (i+1)th frame image relative to the (i)th frame image.

[0100] The image renderer's processing of the graphics data of the i+1th frame to generate the i+1th frame image is not strictly sequential with the application layer's initiation of refresh and the engine layer's refresh, and can occur simultaneously. This is sufficient as long as the changed data can be successfully extracted from the i+1th frame image synthesized by the image renderer based on the calculated coordinates.

[0101] This method of obtaining change data using an image renderer is a completely new way to obtain change data, providing a new technical support for obtaining change data. In addition, due to the high processing speed of image renderers, using an image renderer to obtain change data can accelerate the process of obtaining change data, thereby ensuring the efficient transmission of images.

[0102] It is understood that the intercepted data is a captured image, and the engine layer calculates the size of the data volume in the captured image relative to the total data volume of the (i+1)th frame image, and / or calculates the size of the screen area occupied by the captured image.

[0103] If the proportion of the data volume in the captured image to the total data volume of the i+1th frame image does not reach a first threshold, and / or the proportion of the captured image to the display screen area does not reach a second threshold, the parameter information of the change data is deemed to meet the first condition, i.e., the actual change of the i+1th frame image relative to the ith frame image is small, and the first encoding strategy among the preset encoding strategies is adopted as the target encoding strategy. The captured image is encoded using a preset codec algorithm via the first channel of the video codec, and the position information of the captured image within the i+1th frame image is encoded via the second channel. The first terminal transmits the encoded image and the encoded position to the second terminal. The second terminal decodes the encoded image and the encoded position using the preset codec algorithm, thereby obtaining the change data and the change position of the i+1th frame image relative to the ith frame image. Considering that the display screen sizes of the second terminal and the first terminal may or may not be the same, a reference coordinate system can be pre-established based on the display screen coordinate systems of both terminals. Knowing the pre-established reference coordinate system and the position information of the captured image on the first terminal, the display position of the change data on the second terminal's display screen (e.g., position 2) can be obtained. The second terminal uses the decoded changed data to replace the data at position 2 in the i-th frame image and displays it, so that the i+1-th frame image can be displayed in the second terminal, thereby achieving sharing of the i+1-th frame image in the two terminals.

[0104] This scheme of encoding and transmitting the changed data and the position of the changed data when the actual change of the i+1th frame image relative to the i-th frame image is small can effectively avoid the problem of long encoding time caused by encoding the entire image, shorten the encoding extension, and thus achieve efficient transmission of shared data.

[0105] If the ratio of the amount of data in the captured image to the total amount of data in the i+1th frame image reaches a first threshold, and / or the ratio of the area of ​​the captured image to the display screen area reaches a second threshold, then the parameter information of the change data is considered to meet the second condition, that is, the actual change of the i+1th frame image relative to the i-th frame image is large, and the second encoding strategy among the preset encoding strategies is adopted as the target encoding strategy. The entire image (i+1th frame image) is encoded using the preset codec algorithm through the first channel of the video codec, and the encoded entire image is transmitted by the first terminal to the second terminal. The second terminal decodes the encoded entire image using the preset codec algorithm and displays it on the display screen of the second terminal, thereby achieving the sharing of the i+1th frame image between the two terminals.

[0106] This approach, which encodes and transmits the entire image when the actual change in frame (i+1) relative to frame (i) is significant, ensures accurate data transmission, avoiding missing or incorrect encoding of the changed portion due to significant changes. It also aligns with actual conditions and enhances practicality. By capturing change data from the frame image generated by the image renderer during refresh opportunities provided by the engine layer, this shortens the acquisition time and speeds up the process, ensuring efficient transmission.

[0107] In actual applications, the image obtained by the image renderer is an image visible to the end user, and the image includes the position information of the input device, such as the position of the mouse on the display screen. If the content to be shared in the two adjacent frames of each frame has not changed, and only the mouse position in the subsequent image has changed relative to the mouse position in the previous image, the first terminal needs to calculate the mouse position after the change in the subsequent image and send the mouse position to the second terminal. The second terminal presents its own mouse position in the subsequent image according to the position of the mouse in the subsequent image, so that the mouse positions of the two terminals remain consistent in the same screen displayed by the two terminals. Of course, the first terminal can also obtain the movement trajectory of the mouse and send the movement trajectory to the second terminal, so that the second terminal moves from the original position of the mouse to the changed position according to the same movement trajectory.

[0108] In addition, the mouse position and / or movement trajectory after the change in the subsequent image can be encoded using a preset codec algorithm through the second channel of the video codec, and then transmitted from the first terminal to the second terminal, and the second terminal uses the preset codec algorithm to decode and use it.

[0109] In layman's terms, the encoding of the changed data by the first terminal can be considered as image compression, and the decoding of the encoded image by the second terminal can be considered as decompression of the encoded image. Thus, this solution can be generally understood as a solution that uses different encoding strategies based on the changed parts of two adjacent frames to encode and transmit the changed parts. Specifically, it can be:

[0110] When it is detected that the shared image of two adjacent frames has not changed, and there is only a user input event (for example, the user moves the mouse position during screen projection), the sending end (first terminal) may not compress the mouse position image and only transmit the input event to the receiving end (second terminal).

[0111] When it is detected that the area of ​​a changed region in a shared image between two adjacent frames is less than a specific threshold, the coordinates of the changed region and the data of the changed region (the changed data) are encoded and sent from the transmitting end to the receiving end. The coordinates of the changed region can also be transmitted directly without encoding. The specific threshold can be determined based on actual experience.

[0112] When it is detected that the changed area of ​​the shared image of two adjacent frames is greater than or equal to a specific threshold, the subsequent frame image of the two adjacent frames will be encoded as a whole image and the encoded whole image will be sent.

[0113] In layman's terms, this solution can be seen as making full use of the detection of image content, and adopting different encoding strategies (encoding using different encoding schemes) based on the parameter information of the detected change data to achieve the technical goal of shortening the video encoding time, thereby reducing the end-to-end (from sender to receiver) delay of the network transmission of (real-time) images.

[0114] The engine layer utilizes refresh timing to identify the differences and change locations between adjacent frames. Based on these change locations, the updated data (i+1 frame) is extracted from the i+1 frame processed by the image renderer. This allows for rapid acquisition of change data, simplifies the process, and ensures efficient transmission.

[0115] The present application also provides an image processing device, such as Figure 8 As shown, the device includes:

[0116] A first obtaining unit 10 is configured to obtain change data, wherein the change data is display data obtained by processing the first data stream by the image renderer, and the change data is used to indicate a difference from a previous frame of image; wherein the first data stream includes graphic data of multiple frames of image; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output;

[0117] A second obtaining unit 11 is configured to process the changed data based on a target encoding strategy to obtain encoded data to form a second data stream, where the second data stream includes encoded data of multiple frames;

[0118] In a specific implementation, the second obtaining unit 11 is an encoder, for example, the encoder may be a video codec, and may be a software functional module or a hardware functional module.

[0119] The transmission unit 12 is configured to transmit the second data stream, where the second data stream is at least used for decoding and displaying by a terminal device that receives the second data stream.

[0120] In some embodiments, the apparatus further comprises a determining unit configured to:

[0121] Determining a target encoding strategy from preset encoding strategies based on parameter information of the changed data;

[0122] The parameter information represents the size of the data volume of the changed data and / or the size of the position occupied by the changed data.

[0123] In some embodiments, the determining unit is configured to:

[0124] If the parameter information satisfies the first condition, determining the first encoding strategy in the preset encoding strategies as the target encoding strategy;

[0125] If the parameter information satisfies the second condition, the second encoding strategy in the preset encoding strategy is determined to be the target encoding strategy.

[0126] In some embodiments, a first encoding strategy in the preset encoding strategy is encoding the change data via a first channel of a video codec and encoding a location of the change data via a second channel different from the first channel;

[0127] The second encoding strategy in the preset encoding strategy is to encode the change data through the first channel of the video codec.

[0128] In some embodiments, the first obtaining unit 10 is used to

[0129] In the case where a refresh operation is performed on adjacent frames in a multi-frame image, the change data is obtained; wherein the refresh operation is an operation for different data in the adjacent frames.

[0130] In some embodiments, the second data stream includes multiple frames of encoded data, at least two of which have different encoding strategies.

[0131] In some embodiments, the first frame image in the first data stream is encoded using a second encoding strategy among preset encoding strategies, wherein the first frame image is obtained by processing the graphic data of the first frame image through an image renderer.

[0132] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0133] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to at least perform:

[0134] Obtaining change data, wherein the change data is display data obtained by processing the first data stream by the image renderer, and the change data is used to represent a difference from a previous frame of image; wherein the first data stream includes graphic data of multiple frames of image; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output;

[0135] Based on a target encoding strategy, the changed data is processed to obtain encoded data to form a second data stream, wherein the second data stream includes encoded data of multiple frames;

[0136] The second data stream is transmitted, and the second data stream is at least used for decoding and displaying by a terminal device that receives the second data stream.

[0137] In addition, in the electronic device, the instructions are executed by the at least one processor, so that the at least one processor can at least execute other contents of the aforementioned image processing method.

[0138] The storage medium stores computer instructions, and the computer instructions are used to cause the computer to at least execute:

[0139] Obtaining change data, wherein the change data is display data obtained by processing the first data stream by the image renderer, and the change data is used to represent a difference from a previous frame of image; wherein the first data stream includes graphic data of multiple frames of image; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output;

[0140] Based on a target encoding strategy, the changed data is processed to obtain encoded data to form a second data stream, wherein the second data stream includes encoded data of multiple frames;

[0141] The second data stream is transmitted, and the second data stream is at least used for decoding and displaying by a terminal device that receives the second data stream.

[0142] In addition, in the storage medium, the computer instructions are executed by the at least one processor, so that the at least one processor can at least execute other contents of the aforementioned image processing method.

[0143] Figure 9 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0144] like Figure 9 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0145] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0146] The computing unit 801 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the image processing method. For example, in some embodiments, the image processing method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the image processing method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the image processing method by any other appropriate means (e.g., by means of firmware).

[0147] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0148] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0149] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0150] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0151] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0152] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0153] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0154] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0155] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An image processing method, comprising: Obtaining change data, wherein the change data is display data obtained by processing the first data stream by the image renderer, and the change data is used to represent a difference from a previous frame of image; wherein the first data stream includes graphic data of multiple frames of image; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output; Determining a target encoding strategy from preset encoding strategies based on parameter information of the changed data; The parameter information represents at least one of the following: the amount of the changed data, the size of the position occupied by the changed data; Based on a target encoding strategy, the changed data is processed to obtain encoded data to form a second data stream, wherein the second data stream includes encoded data of multiple frames; Transmitting the second data stream, where the second data stream is at least used for decoding and displaying by a terminal device receiving the second data stream; If the parameter information of the changed data satisfies a first condition indicating that the change of the changed data is small, the target encoding strategy is a first encoding strategy for encoding the changed data; If the parameter information of the change data satisfies the second condition indicating that the change of the change data is large, the target encoding strategy is the second encoding strategy for encoding the entire image.

2. The method according to claim 1, wherein A first encoding strategy in the preset encoding strategy is encoding the change data via a first channel of a video codec and encoding a position of the change data via a second channel different from the first channel; The second encoding strategy in the preset encoding strategy is to encode the change data through the first channel of the video codec.

3. The method according to claim 1, further comprising: When a refresh operation is performed on adjacent frames in the multi-frame image, obtaining the change data; The refresh operation is an operation on different data in adjacent frames.

4. The method according to claim 1, wherein The second data stream includes multiple frames of coded data, at least two of which have different coding strategies.

5. The method according to claim 1, wherein The first frame image in the first data stream is encoded using a second encoding strategy among the preset encoding strategies, wherein the first frame image is obtained by processing the graphic data of the first frame image through an image renderer.

6. An image processing device comprising: a first obtaining unit, configured to obtain change data, wherein the change data is display data obtained by processing the first data stream by the image renderer, and the change data is used to represent a difference from a previous frame of image; wherein the first data stream includes graphic data of multiple frames of image; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output; Determining a target encoding strategy from preset encoding strategies based on parameter information of the changed data; The parameter information represents at least one of the following: the amount of the changed data, the size of the position occupied by the changed data; a second obtaining unit, configured to process the changed data based on a target encoding strategy to obtain encoded data to form a second data stream, wherein the second data stream includes encoded data of multiple frames; a transmission unit, configured to transmit the second data stream, wherein the second data stream is at least used for decoding and displaying by a terminal device receiving the second data stream; If the parameter information of the changed data satisfies a first condition indicating that the change of the changed data is small, the target encoding strategy is a first encoding strategy for encoding the changed data; If the parameter information of the change data satisfies the second condition indicating that the change of the change data is large, the target encoding strategy is the second encoding strategy for encoding the entire image.

7. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to at least perform: Obtaining change data, wherein the change data is display data obtained by processing the first data stream by the image renderer, and the change data is used to represent a difference from a previous frame of image; wherein the first data stream includes graphic data of multiple frames of image; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output; Determining a target encoding strategy from preset encoding strategies based on parameter information of the changed data; The parameter information represents at least one of the following: the amount of the changed data, the size of the position occupied by the changed data; Based on a target encoding strategy, the changed data is processed to obtain encoded data to form a second data stream, wherein the second data stream includes encoded data of multiple frames; Transmitting the second data stream, where the second data stream is at least used for decoding and displaying by a terminal device receiving the second data stream; If the parameter information of the changed data satisfies a first condition indicating that the change of the changed data is small, the target encoding strategy is a first encoding strategy for encoding the changed data; If the parameter information of the change data satisfies the second condition indicating that the change of the change data is large, the target encoding strategy is the second encoding strategy for encoding the entire image.

8. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to at least: Obtaining change data, wherein the change data is display data obtained by processing the first data stream through the image renderer, and the change data is used to represent the difference from the previous frame image; wherein, The first data stream includes graphic data of multiple frames of images; the graphic data of each frame of image is processed by the image renderer to obtain a frame of image for display output; Determining a target encoding strategy from preset encoding strategies based on parameter information of the changed data; The parameter information represents at least one of the following: the amount of the changed data, the size of the position occupied by the changed data; Based on a target encoding strategy, the changed data is processed to obtain encoded data to form a second data stream, wherein the second data stream includes encoded data of multiple frames; Transmitting the second data stream, where the second data stream is at least used for decoding and displaying by a terminal device receiving the second data stream; If the parameter information of the changed data satisfies a first condition indicating that the change of the changed data is small, the target encoding strategy is a first encoding strategy for encoding the changed data; If the parameter information of the change data satisfies the second condition indicating that the change of the change data is large, the target encoding strategy is the second encoding strategy for encoding the entire image.

Images