Image encoding method, image decoding method, terminal device and readable storage medium

Abstract

The present application is applicable to the field of terminal technology, and in particular relates to an image encoding method, an image decoding method, a terminal device, and a readable storage medium. The present application can enable a high-resolution image to be displayed on a low-resolution terminal device by only transmitting or acquiring partial image data and differential data, thereby effectively shortening the transmission time of the image data and increasing the image transmission speed, or effectively shortening the acquisition time of the image data and increasing the image acquisition speed. Moreover, the terminal device only needs to decode partial image data and differential data, that is, it can obtain a target image that meets the display requirements of the terminal device, thereby effectively reducing the computational complexity of the terminal device during image processing, reducing the processing power consumption of the terminal device, improving the image processing efficiency, increasing the image presentation speed, and improving the user experience.

Description

Technical Field

[0001] The present application belongs to the field of terminal technology, and in particular relates to an image encoding method, an image decoding method, a terminal device, and a computer-readable storage medium. Background Art

[0002] With the development of image acquisition devices, image resolution is increasing. However, the screen resolution of existing terminal devices is far lower than the image resolution. In other words, due to the limitations of the terminal device's display capabilities, the terminal device is generally unable to present images according to the image resolution. However, high-resolution images increase the image transmission time, resulting in slower image transmission speeds. They also increase the computational complexity of image processing such as image decoding and display, resulting in high processing power consumption of the terminal device, slow image rendering speeds, and a poor user experience. Summary of the Invention

[0003] The embodiments of the present application provide an image encoding method, an image decoding method, a terminal device, and a computer-readable storage medium, which can solve the problems of slow image transmission speed, high computational complexity, high processing power consumption of the terminal device, and slow image presentation speed caused by high-resolution images.

[0004] In a first aspect, an embodiment of the present application provides an image encoding method, which may include:

[0005] Obtaining a first resolution of an original image and a second resolution of a target image, and determining a target encoding round corresponding to the target image according to the first resolution and the second resolution;

[0006] Acquire an intermediate image corresponding to the original image according to the target encoding round, and downsample the intermediate image using a downsampling algorithm to obtain a predicted image;

[0007] Scaling the original image according to the second resolution to obtain a scaled image corresponding to the original image, and performing a difference process between the scaled image and the predicted image to obtain a difference image;

[0008] compressing the differential image using a compression algorithm to obtain differential data corresponding to the differential image;

[0009] A target encoding file of the original image is obtained according to the second resolution, the target encoding round, the downsampling algorithm, the compression algorithm, the differential data and the original encoding file, where the original encoding file is an encoding file obtained by encoding the original image.

[0010] Through the above-mentioned image coding method, based on the original image coding, an intermediate image of a certain coding round can be selected as the basis, down-sampled as a predicted image of the low-resolution target image, and a differential image can be obtained based on the predicted image and the scaled image corresponding to the original image. Then, the differential data corresponding to the differential image is inserted into the original encoded file, so that the specific low-resolution target image can be accurately restored by the predicted image and the differential data together, thereby realizing variable-resolution image coding compatible with the original coding method. At the same time, when sharing images encoded based on this image coding method, part of the image data can be transmitted according to the display resolution of the receiving device, which can greatly reduce the amount of data to be transmitted, increase the image transmission speed, and enhance the user experience.

[0011] In a possible implementation manner of the first aspect, the original encoded file is an encoded file obtained by encoding the original image in JPEG progressive mode.

[0012] Exemplarily, determining the target encoding round corresponding to the target image according to the first resolution and the second resolution may include:

[0013] Obtaining the number of encoded coefficients corresponding to each encoding round according to the original encoding file;

[0014] A target coding round corresponding to the target image is determined according to the first resolution, the second resolution, and the number of encoded coefficients, where the target coding round is any one of the encoding rounds.

[0015] In the image coding method provided by this solution, when the original image is encoded using the spectral-selective JPEG progressive mode, spectral selection first encodes the low-frequency coefficients in each pixel block, and then sequentially encodes the high-frequency coefficients in each pixel block. This means that multiple rounds of encoding are performed on the 64 DCT coefficients in each pixel block, with each round encoding only a portion of the DCT coefficients in each pixel block. This allows the number of encoded DCT coefficients to gradually increase with each encoding round, thereby gradually improving image clarity. Therefore, the target encoding round corresponding to the target image can be accurately determined based on the first resolution, the second resolution, and the number of encoded DCT coefficients.

[0016] Exemplarily, determining the target encoding round corresponding to the target image according to the first resolution and the second resolution may include:

[0017] Obtaining the total number of bytes of discrete cosine transform coefficients and the number of encoded bytes corresponding to each encoding round according to the original encoding file;

[0018] A target encoding round corresponding to the target image is determined according to the first resolution, the second resolution, the total number of bytes, and the number of encoded bytes, where the target encoding round is any one of the encoding rounds.

[0019] In the image encoding method provided by this solution, when the original image is encoded using the JPEG progressive mode based on successive approximation, successive approximation first encodes the high-order byte of each DCT coefficient in each pixel block, and then successively encodes the low-order byte of each DCT coefficient. This makes the value of each DCT coefficient increasingly accurate, thereby gradually improving image clarity. Therefore, the target encoding round corresponding to the target image can be accurately determined based on the first resolution, the second resolution, the total number of bytes, and the number of bytes already encoded.

[0020] In another possible implementation manner of the first aspect, the original encoded file is an encoded file obtained by encoding the original image in a PNG interlaced scanning mode.

[0021] Exemplarily, determining the target encoding round corresponding to the target image according to the first resolution and the second resolution may include:

[0022] Obtaining the number of encoded pixels corresponding to each encoding round according to the original encoding file;

[0023] A target encoding round corresponding to the target image is determined according to the first resolution, the second resolution, and the number of encoded pixels, where the target encoding round is any one of the encoding rounds.

[0024] In the image encoding method provided by this solution, when the original image is encoded using the PNG interlaced scanning mode, each scanning pass in the PNG interlaced scanning mode only scans a portion of the image's pixels. In other words, the PNG interlaced scanning mode progressively replenishes image data over seven scanning passes, allowing the image outline to be quickly displayed initially. Then, as the scanning passes increase, the image data is gradually increased, thereby gradually improving image clarity. Therefore, the target encoding pass corresponding to the target image can be accurately determined based on the first resolution, the second resolution, and the number of encoded pixels.

[0025] Specifically, obtaining the intermediate image corresponding to the original image according to the target encoding round may include:

[0026] Image data corresponding to the target encoding round is obtained from the original encoding file, and the image data is decoded to obtain an intermediate image corresponding to the original image.

[0027] It can be understood that obtaining the target encoded file of the original image according to the second resolution, the target encoding round, the downsampling algorithm, the compression algorithm, the differential data, and the original encoded file may include:

[0028] Determining a first position of the image data corresponding to the target encoding round in the original encoded file, and determining a second position of the differential data in the original encoded file based on the first position;

[0029] inserting the differential data into the second position;

[0030] Encoding the second resolution, the target encoding round, the first algorithm type of the downsampling algorithm, and the second algorithm type of the compression algorithm to obtain a variable resolution file header;

[0031] The variable resolution file header is inserted into the original file header of the original encoded file.

[0032] In the image encoding method provided by this solution, a second resolution corresponding to a target image, a target encoding round, a first algorithm type corresponding to a downsampling algorithm, and a second algorithm type corresponding to a compression algorithm can be encoded to obtain a variable-resolution file header. The variable-resolution file header and differential data can then be inserted into the original encoded file as custom extension fields to obtain the target encoded file. To conveniently and quickly determine whether the original image can be displayed at a variable resolution, the variable-resolution file header can be inserted after or before the original file header in the original encoded file to form a new file header for the target encoded file. Furthermore, to enable decoding to obtain a low-resolution target image by only acquiring partial image data from the target encoded file, differential data can be inserted before or after the image data corresponding to the target encoding round in the original encoded file. This allows, when decoding and displaying the image, after determining the target encoding round corresponding to the low-resolution target image, to acquire only the image data and differential data corresponding to the target encoding round, thereby decoding to obtain the low-resolution target image, improving image presentation speed and user experience.

[0033] In a second aspect, an embodiment of the present application provides an image decoding method, which may include:

[0034] Obtaining a display resolution of a terminal device, and determining a target resolution of a target image based on the display resolution and a target encoding file, wherein the target encoding file is an encoded file obtained by encoding according to any one of the methods of the first aspect above;

[0035] Determining a target encoding round corresponding to the target image according to the target resolution, and acquiring image data and differential data corresponding to the target image according to the target encoding round;

[0036] Decoding the image data to obtain an intermediate image, and downsampling the intermediate image using a downsampling algorithm to obtain a predicted image;

[0037] The differential data is decoded to obtain a differential image, and a target image of the target resolution is generated according to the predicted image and the differential image.

[0038] Through the above-mentioned image decoding method, when the display resolution of the terminal device is lower than the first resolution of the original image, the terminal device can obtain partial image data and differential data based on its own display resolution and the second resolution that the original image can support for display, and only needs to decode the partial image data and differential data, that is, the target image that meets the display requirements of the terminal device can be obtained, which can effectively reduce the amount of data acquired when the terminal device displays the image, reduce the computational complexity when the terminal device decodes and displays the image, reduce the processing power consumption of the terminal device, increase the speed of the terminal device in presenting the image, and enhance the user experience.

[0039] Exemplarily, downsampling the intermediate image using a downsampling algorithm to obtain a predicted image may include:

[0040] A downsampling algorithm corresponding to the intermediate image is acquired according to the target resolution, and the intermediate image is downsampled using the downsampling algorithm to obtain the predicted image.

[0041] Exemplarily, decoding the differential data to obtain a differential image may include:

[0042] A compression algorithm corresponding to the differential image is acquired according to the target resolution, and the differential data is decoded according to the compression algorithm to obtain the differential image.

[0043] Specifically, generating the target image of the target resolution according to the predicted image and the differential image may include:

[0044] The predicted image and the differential image are merged to obtain a target image of the target resolution.

[0045] In a third aspect, an embodiment of the present application provides an image encoding apparatus, which may include:

[0046] a coding round determination module, configured to obtain a first resolution of an original image and a second resolution of a target image, and determine a target coding round corresponding to the target image according to the first resolution and the second resolution;

[0047] A predicted image acquisition module, configured to acquire an intermediate image corresponding to the original image according to the target encoding round, and downsample the intermediate image using a downsampling algorithm to obtain a predicted image;

[0048] a differential image acquisition module, configured to scale the original image according to the second resolution to obtain a scaled image corresponding to the original image, and perform differential processing between the scaled image and the predicted image to obtain a differential image;

[0049] A differential data acquisition module, configured to compress the differential image using a compression algorithm to obtain differential data corresponding to the differential image;

[0050] An image encoding module is used to obtain a target encoding file of the original image based on the second resolution, the target encoding round, the downsampling algorithm, the compression algorithm, the differential data and the original encoding file, where the original encoding file is an encoding file obtained by encoding the original image.

[0051] In a possible implementation manner of the third aspect, the original encoded file is an encoded file obtained by encoding the original image in JPEG progressive mode.

[0052] Exemplarily, the encoding round determination module may include:

[0053] A coefficient number obtaining unit, configured to obtain the number of encoded coefficients corresponding to each encoding round according to the original encoding file;

[0054] A first coding round determining unit is configured to determine a target coding round corresponding to the target image according to the first resolution, the second resolution, and the number of encoded coefficients, where the target coding round is any one of the coding rounds.

[0055] Exemplarily, the encoding round determination module may include:

[0056] a byte number acquisition unit, configured to acquire the total number of bytes of discrete cosine transform coefficients and the number of encoded bytes corresponding to each encoding round according to the original encoding file;

[0057] A second encoding round determining unit is configured to determine a target encoding round corresponding to the target image according to the first resolution, the second resolution, the total number of bytes, and the number of encoded bytes, where the target encoding round is any one of the encoding rounds.

[0058] In another possible implementation manner of the third aspect, the original encoded file is an encoded file obtained by encoding the original image using a PNG interlaced scanning mode.

[0059] Exemplarily, the encoding round determination module may include:

[0060] A pixel number acquisition unit, configured to acquire the number of encoded pixels corresponding to each encoding round according to the original encoding file;

[0061] A third encoding round determining unit is configured to determine a target encoding round corresponding to the target image according to the first resolution, the second resolution, and the number of encoded pixels, where the target encoding round is any one of the encoding rounds.

[0062] Specifically, the differential image acquisition module may include:

[0063] The intermediate image acquisition unit is used to obtain image data corresponding to the target encoding round from the original encoding file, and decode the image data to obtain an intermediate image corresponding to the original image.

[0064] Optionally, the image encoding module may include:

[0065] a position determining unit, configured to determine a first position of the image data corresponding to the target encoding round in the original encoded file, and determine a second position of the differential data in the original encoded file based on the first position;

[0066] a differential data inserting unit, configured to insert the differential data into the second position;

[0067] A file header obtaining unit, configured to encode the second resolution, the target encoding round, the first algorithm type of the downsampling algorithm, and the second algorithm type of the compression algorithm to obtain a variable resolution file header;

[0068] The file header inserting unit is used to insert the variable resolution file header into the original file header of the original encoded file.

[0069] In a fourth aspect, an embodiment of the present application provides an image decoding apparatus, which may include:

[0070] a resolution determination module, configured to obtain a display resolution of a terminal device and determine a target resolution of a target image based on the display resolution and a target encoding file, wherein the target encoding file is an encoded file obtained by encoding according to any one of the methods of the first aspect above;

[0071] a coding round determination module, configured to determine a target coding round corresponding to the target image according to the target resolution, and obtain image data and differential data corresponding to the target image according to the target coding round;

[0072] A predicted image acquisition module, configured to decode the image data to obtain an intermediate image, and downsample the intermediate image using a downsampling algorithm to obtain a predicted image;

[0073] An image decoding module is used to decode the differential data to obtain a differential image, and generate a target image of the target resolution according to the predicted image and the differential image.

[0074] Exemplarily, the predicted image acquisition module is specifically configured to acquire a downsampling algorithm corresponding to the intermediate image according to the target resolution, and downsample the intermediate image using the downsampling algorithm to obtain the predicted image.

[0075] Exemplarily, the image decoding module may include:

[0076] The differential image acquisition unit is configured to acquire a compression algorithm corresponding to the differential image according to the target resolution, and decode the differential data according to the compression algorithm to obtain the differential image.

[0077] Specifically, the image decoding module may further include:

[0078] An image merging unit is used to merge the predicted image and the differential image to obtain a target image of the target resolution.

[0079] In the fifth aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the terminal device implements any one of the methods described in the first aspect above.

[0080] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a computer, the computer implements any one of the methods described in the first aspect above.

[0081] In a seventh aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any of the methods described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1is a schematic structural diagram of a terminal device to which the image encoding method or image decoding method provided in one embodiment of the present application is applicable;

[0083] Figure 2 This is a schematic diagram of a software architecture to which the image encoding method or image decoding method provided in one embodiment of the present application is applicable;

[0084] Figure 3 This is a schematic diagram of a scenario in which an image encoded in JPEG progressive mode is decoded and displayed according to an embodiment of the present application;

[0085] Figure 4 This is a flowchart of an image encoding method provided by an embodiment of the present application;

[0086] Figure 5 This is a flowchart of an image decoding method provided by an embodiment of the present application;

[0087] Figure 6 This is a scanning schematic diagram of the PNG interlaced scanning mode provided by an embodiment of the present application. DETAILED DESCRIPTION

[0088] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0089] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0090] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0091] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0092] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0093] In addition, the “plurality” mentioned in the embodiments of the present application should be interpreted as two or more.

[0094] The steps involved in the image encoding method or image decoding method provided in the embodiments of the present application are merely examples. Not all steps are required, nor are all information or messages required. These steps may be added or removed as needed during use. The same step or steps or messages with the same function in different embodiments of the present application may be referenced and learned from each other.

[0095] The business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field will know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0096] With the advancement of media experience requirements and image acquisition devices, the resolution of images, such as photos, is increasing. For example, the resolution of images captured by certain mobile phones can reach 8192×6144. However, the screen resolution of displays in existing terminal devices is far lower than the image resolution. For example, the screen resolution of certain mobile phones is approximately 2640×1200, which is about 1 / 16 of this resolution. The screen resolution of certain laptops is approximately 3000×2000, which is about 1 / 8 of this resolution. Due to the limitations of terminal display capabilities, existing terminal devices are generally unable to fully present such high-resolution images. However, the large amount of data required for high-resolution images increases the computational complexity of image processing, such as image decoding and display, and increases the central processing unit (CPU) utilization. This results in high power consumption, slow image rendering, and a poor user experience. Furthermore, when sharing images on cloud servers or across multiple connected terminal devices, the large data volume of high-resolution images consumes a considerable amount of time, resulting in slow image transmission and a poor user experience.

[0097] That is, for terminal devices that can only display a fraction or a dozen of an image, high-resolution images result in a large waste of data transmission and image processing, which greatly reduces the image transmission speed and image presentation speed, increases the processing power consumption of the terminal device, and affects the user experience.

[0098] Currently, the hierarchical mode provided by the Joint Photographic Experts Group (JPEG) can achieve variable resolution. That is, by performing variable resolution processing on high-resolution images, lower-resolution image data can be transmitted to low-resolution terminal devices, reducing data transmission volume and increasing image transmission speed. At the same time, it can also reduce the processing power consumption of terminal devices and increase the image presentation speed of terminal devices.

[0099] Specifically, the JPEG hierarchical mode compresses and encodes different resolution versions of the same image into a single image file in a multi-frame format, forming a pyramidal code. The first frame of the pyramidal code is a non-differential frame that can be decoded independently. Subsequent frames are differential frames calculated based on a prediction of the previous frame, with the resolution of each subsequent frame being doubled in both width and height, until the final frame represents the original resolution of the image. Differential frames contain little data and offer a high compression ratio. Therefore, the file size of the resulting image compressed using the JPEG hierarchical mode is generally no larger than the file size of the original resolution image compressed using a single frame. If a terminal device only needs to restore an image at a specific lower resolution, it can read only the portion of data from the compressed image file starting at the file header and decode and display this data to obtain the image at the specified lower resolution. However, due to the complex encoding and decoding methods of the JPEG hierarchical mode and its incompatibility with the widely supported JPEG progressive mode, its ecosystem adoption has been difficult. For example, mainstream commercial decoding libraries, browsers, and operating systems currently use the JPEG progressive mode, with little application or implementation of the JPEG hierarchical mode. However, the current JPEG progressive mode does not have the ability to change resolution. When performing image processing such as transmission and decoding and display of high-resolution images, it will still cause high processing power consumption of terminal devices, resulting in slow image transmission and image presentation speeds, affecting user experience.

[0100] To address the above-mentioned issues, embodiments of the present application provide an image encoding method, an image decoding method, a terminal device, and a computer-readable storage medium. This image encoding method, when encoding a high-resolution original image based on the JPEG progressive mode or the interlaced scanning mode of portable network graphics (PNG), can determine the second resolution of a low-resolution target image based on the first resolution of the original image, and can determine the target encoding pass corresponding to the target image based on the first and second resolutions. Subsequently, an intermediate image corresponding to the target encoding pass can be obtained and downsampled to obtain a predicted image corresponding to the target image. The original image can then be directly scaled according to the second resolution to obtain a scaled image, and the scaled image and the predicted image can be differentially processed to obtain a differential image. Finally, the differential image can be compression-encoded to obtain differential data corresponding to the differential image, and the differential data can be added to the original encoded file corresponding to the JPEG progressive mode or the PNG interlaced scanning mode to obtain the target encoded file corresponding to the original image. When decoding and displaying the image, the terminal device can obtain the terminal device's display resolution and determine the target resolution of the target image based on the display resolution and the target encoded file. The terminal device can then determine the target encoding pass corresponding to the target image based on the target resolution and retrieve the image data of the intermediate image corresponding to the target encoding pass and the differential data corresponding to the target encoding pass from the target encoding file. The terminal device can then decode the image data of the intermediate image to obtain the intermediate image and downsample the intermediate image to obtain the predicted image. Simultaneously, the terminal device can decode the differential data to obtain the differential image. Finally, the terminal device can obtain the target image at the target resolution based on the predicted image and the differential image.

[0101] That is, the embodiment of the present application is based on the JPEG progressive mode or the PNG interlaced scanning mode to construct an image encoding method that is compatible with the JPEG progressive mode or the PNG interlaced scanning mode. This image encoding method can enable a high-resolution image to be displayed on a low-resolution terminal device by only transmitting or acquiring partial image data and differential data, thereby effectively shortening the image data transmission time and increasing the image transmission speed, or effectively shortening the image data acquisition time and increasing the image acquisition speed. Moreover, the terminal device only needs to decode partial image data and differential data to obtain a target image that meets the display requirements of the terminal device, thereby effectively reducing the computational complexity of the terminal device during image processing, reducing the processing power consumption of the terminal device, improving image processing efficiency, increasing image presentation speed, and improving user experience, thereby having strong ease of use and practicality.

[0102] It is understandable that the terminal devices involved in the embodiments of the present application may be mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), desktop computers, cloud servers, etc. The embodiments of the present application do not impose any restrictions on the specific types of terminal devices.

[0103] The following first introduces the terminal device involved in the embodiment of this application. Figure 1 , Figure 1 A schematic structural diagram of the terminal device 100 is shown.

[0104] The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0105] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0106] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0107] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0108] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0109] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0110] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the terminal device 100.

[0111] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0112] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0113] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0114] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the terminal device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the terminal device 100.

[0115] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0116] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the terminal device 100 and to transfer data between the terminal device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect to other terminal devices, such as AR devices.

[0117] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt a different interface connection method from the above embodiments, or a combination of multiple interface connection methods.

[0118] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the terminal device 100. While charging the battery 142, the charging management module 140 can also provide power to the terminal device via the power management module 141.

[0119] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0120] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0121] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0122] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0123] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0124] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. applied to the terminal device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0125] In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0126] The terminal device 100 implements display functions through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0127] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, terminal device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0128] The terminal device 100 can realize the shooting function through the ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0129] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0130] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0131] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0132] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. This allows terminal device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0133] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in the terminal device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0134] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0135] The internal memory 121 can be used to store computer executable program code, which includes instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the terminal device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0136] The terminal device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0137] The buttons 190 include a power button, a volume button, etc. The buttons 190 may be mechanical buttons or touch buttons. The terminal device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the terminal device 100.

[0138] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0139] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0140] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and disconnected from the terminal device 100 by inserting or removing it from the SIM card interface 195. The terminal device 100 can support one or N SIM card interfaces, where N is a positive integer greater than one. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

[0141] The software system of the terminal device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the terminal device 100.

[0142] Figure 2 It is a software structure block diagram of the terminal device 100 according to an embodiment of the present application.

[0143] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0144] The application layer can include a series of application packages.

[0145] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0146] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0147] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0148] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0149] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0150] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0151] The phone manager is used to provide communication functions of the terminal device 100, such as management of call status (including answering, hanging up, etc.).

[0152] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0153] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically, without requiring user interaction. For example, the Notification Manager can be used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, vibrating the device, or flashing indicator lights.

[0154] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0155] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0156] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0157] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0158] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0159] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0160] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0161] A 2D graphics engine is a drawing engine for 2D drawings.

[0162] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0163] The following describes the workflow of the software and hardware of the terminal device 100 in combination with the capture and photo shooting scene.

[0164] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture a still image or video through the camera 193.

[0165] The following will describe in detail the image encoding method and image decoding method provided in the embodiments of the present application in conjunction with the accompanying drawings and specific application scenarios.

[0166] [Example 1]

[0167] The image encoding method provided in this embodiment expands upon the JPEG progressive mode to implement variable-resolution image encoding compatible with the JPEG progressive mode. Therefore, when decoding and displaying an image encoded using this image encoding method, a terminal device can decode to obtain a specific low-resolution image or the original high-resolution image, meeting the display requirements of different scenarios. The following details image encoding and decoding based on the JPEG progressive mode.

[0168] 1. Image Coding Based on JPEG Progressive Mode

[0169] The encoding process of JPEG progressive mode mainly includes: (1) dividing the original image into multiple 8*8 pixel blocks, each with 64 pixels; (2) performing discrete cosine transform (DCT) on each pixel block to obtain 64 DCT coefficients corresponding to the pixel block; 3. quantizing the 64 DCT coefficients of each pixel block and encoding the quantized DCT coefficients. Unlike the JPEG sequential mode, which encodes all the DCT coefficients of each pixel block from left to right and from top to bottom at once, the JPEG progressive mode encodes and transmits all the DCT coefficients of each pixel block in multiple rounds. When displaying the image, it can also be decoded and displayed round by round, so that the image outline can be displayed quickly first, and then as the decoding rounds increase, the image data is gradually increased, thereby gradually improving the image clarity. Please refer to Figure 3 , Figure 3 A schematic diagram of a scene showing an image encoded in JPEG progressive mode is shown. Figure 3 (a) to Figure 3 As shown in (e), as the number of decoding rounds increases, the image data gradually increases, and the clarity of the displayed image also gradually increases.

[0170] It is understandable that the JPEG progressive mode can be implemented in the following ways: a. spectrum selection; b. stepwise approximation; or a combination of ca and b.

[0171] Spectral selection: First, the low-frequency coefficients in each pixel block are encoded. Then, the high-frequency coefficients in each pixel block are encoded in successive rounds. This involves performing multiple rounds of encoding on the 64 DCT coefficients in each pixel block, with each round encoding only a portion of the DCT coefficients. For example, the first round might encode the DC coefficient of each pixel block. The second round might encode the first and second AC coefficients of each pixel block. The third round might encode the third, fourth, fifth, and sixth AC coefficients of each pixel block, and so on, until all 64 DCT coefficients of each pixel block are encoded. After performing the DCT transform on each 8x8 pixel block, an 8x8 DCT coefficient matrix is ​​generated. From the top left corner to the bottom right corner of the matrix, the DCT coefficients represent increasingly higher-frequency coefficients. The DC coefficient refers to the first DCT coefficient starting from the top left corner of each pixel block, while the AC coefficients refer to the remaining 63 DCT coefficients in each pixel block.

[0172] Successive approximation: First, encode the high-order byte of each DCT coefficient in each pixel block, and then encode the low-order byte of each DCT coefficient in turn, so that the value of each DCT coefficient becomes increasingly accurate. Therefore, when displaying an image, the clarity of the image can be gradually improved as the number of decoding rounds increases. However, successive approximation is for AC coefficients. That is, in each round of encoding of successive approximation, all AC coefficients except the DC coefficient need to be encoded, but only part of the bytes of each AC coefficient are encoded. For the DC coefficient, all bytes (including high-order bytes and low-order bytes) need to be encoded once. For example, the first round can encode the DC coefficient of each pixel block (that is, all bytes of the DC coefficient), the second round can continue to encode the first high-order byte of all AC coefficients, the third round can continue to encode the second high-order byte of all AC coefficients, and so on, until the last round encodes the last low-order byte of all AC coefficients.

[0173] See also Figure 4 , Figure 4 FIG. 1 shows a schematic flow chart of the image encoding method provided by this embodiment. Figure 4 As shown, the image encoding method may include:

[0174] S401 : Acquire a first resolution of an original image and a second resolution of a target image, and determine a target encoding round corresponding to the target image according to the first resolution and the second resolution.

[0175] Among them, the target image is a low-resolution version of the original image, that is, the target image has the same content as the original image, but the second resolution of the target image is lower than the first resolution of the original image. It should be understood that this embodiment does not specifically limit the number and second resolution of target images, and the number and second resolution of target images can be determined based on the first resolution of the original image and the specific application scenario. Since the JPEG progressive mode uses 8*8 pixel blocks as processing units, this embodiment can determine the second resolution of the target image based on multiples of 64 and the first resolution. For example, the second resolution of the target image can be determined as 1 / 64 of the first resolution, or the second resolution of the target image can be determined as 1 / 32 of the first resolution, or the second resolution of the target image can be determined as 1 / 16 of the first resolution, and so on. The following description takes one target image as an example. It should be understood that when there are two or more target images, you can refer to Figure 4 The method is used to process each target image separately.

[0176] It should be noted that, since the processing of each 8*8 pixel block in JPEG progressive mode is identical, the following description uses a single 8*8 pixel block as an example. That is, the number of DCT coefficients mentioned below refers to the number of DCT coefficients in an 8*8 pixel block, and the minimum number of sampling points refers to the minimum number of points required to sample an 8*8 pixel block.

[0177] In spectrally selective JPEG progressive mode, the target encoding round corresponding to the target image can be determined based on the first resolution of the original image, the second resolution of the target image, and the number of DCT coefficients corresponding to each encoding round. For example, the minimum number of sampling points required for the target image can be determined based on the first and second resolutions. Then, a number of DCT coefficients close to this minimum number of sampling points can be determined, and the encoding round corresponding to the determined number of DCT coefficients can be determined as the target encoding round corresponding to the target image. The number of DCT coefficients corresponding to each encoding round refers to the total number of DCT coefficients encoded up to that encoding round. Specifically, the number of DCT coefficients corresponding to the first encoding round can be the number of DCT coefficients encoded in the first encoding round; the number of DCT coefficients corresponding to the second encoding round can be the sum of the number of DCT coefficients encoded in the first and second encoding rounds; the number of DCT coefficients corresponding to the third encoding round can be the sum of the number of DCT coefficients encoded in the first, second, and third encoding rounds, and so on.

[0178] For example, when the first round encodes one DCT coefficient (i.e., a DC coefficient), the second round continues to encode three DCT coefficients, the third round continues to encode five DCT coefficients, and the fourth round continues to encode 12 DCT coefficients, the number of DCT coefficients corresponding to the first round of encoding may be 1, the number of DCT coefficients corresponding to the second round of encoding may be 4, the number of DCT coefficients corresponding to the third round of encoding may be 9, and the number of DCT coefficients corresponding to the fourth round of encoding may be 21. Therefore, when the minimum number of sampling points is 1, the target encoding round corresponding to the target image can be determined as the first round; when the minimum number of sampling points is 4, the target encoding round corresponding to the target image can be determined as the second round; when the minimum number of sampling points is 8, the target encoding round corresponding to the target image can be determined as the third round; when the minimum number of sampling points is 16, the target encoding round corresponding to the target image can be determined as the fourth round, and so on.

[0179] It should be noted that, to improve the accuracy of subsequent predicted images, in this embodiment, when determining the number of DCT coefficients close to the minimum number of sampling points, the number can be determined from the number of DCT coefficients greater than or equal to the minimum number of sampling points. For example, when the minimum number of sampling points is 4, the number of DCT coefficients close to the minimum number of sampling points can be determined from the number of DCT coefficients greater than or equal to 4. Assuming that the number of DCT coefficients corresponding to the second round of encoding is 3 and the number of DCT coefficients corresponding to the third round of encoding is 5, the number of DCT coefficients corresponding to the third round of encoding, 5, can be determined as the number of DCT coefficients close to the minimum number of sampling points.

[0180] The following describes how to determine the minimum number of sampling points required for the target image based on the first resolution and the second resolution.

[0181] In this embodiment, when the second resolution of the target image is 1 / 64 of the first resolution of the original image, it means that at least one sampling point needs to be determined in each pixel block of the original image (i.e., every 64 pixels) to constitute the target image. In this case, the minimum number of sampling points corresponding to the target image can be determined as 1; when the second resolution is 1 / 32 of the first resolution, it means that at least one sampling point needs to be determined in every 32 pixels of the original image to constitute the target image, i.e., at least 2 sampling points need to be determined in each pixel block to constitute the target image. In this case, the minimum number of sampling points corresponding to the target image can be determined as 2; when the second resolution is 1 / 16 of the first resolution, it means that at least one sampling point needs to be determined in every 16 pixels of the original image to constitute the target image, i.e., at least 4 sampling points need to be determined in each pixel block to constitute the target image. In this case, the minimum number of sampling points corresponding to the target image can be determined as 4; When the second resolution is 1 / 8 of the first resolution, it means that at least one sampling point needs to be determined in every 8 pixels of the original image to constitute the target image, that is, at least 8 sampling points need to be determined in each pixel block to constitute the target image. In this case, the minimum number of sampling points corresponding to the target image can be determined as 8; when the second resolution is 1 / 4 of the first resolution, it means that at least one sampling point needs to be determined in every 4 pixels of the original image to constitute the target image, that is, at least 16 sampling points need to be determined in each pixel block to constitute the target image. In this case, the minimum number of sampling points corresponding to the target image can be determined as 16; when the second resolution is 1 / 2 of the first resolution, it means that at least one sampling point needs to be determined in every 2 pixels of the original image to constitute the target image, that is, at least 32 sampling points need to be determined in each pixel block to constitute the target image. In this case, the minimum number of sampling points corresponding to the target image can be determined as 32, and so on.

[0182] Similarly, in the JPEG progressive mode based on successive approximation, the target coding round corresponding to the target image can be determined based on the first resolution of the original image, the second resolution of the target image, the total number of bytes of the DCT coefficients, and the number of bytes encoded corresponding to each coding round. For example, the minimum number of sampling points required for the target image can be first determined based on the first resolution and the second resolution. Then, the number of DCT coefficients corresponding to each coding round can be determined based on the total number of bytes of the DCT coefficients and the number of bytes encoded corresponding to each coding round. The coding round corresponding to the number of DCT coefficients closest to the minimum number of sampling points is determined as the target coding round corresponding to the target image.

[0183] Since successive approximation is performed for AC coefficients, while DC coefficients still require full encoding of all bytes in the first encoding round, the number of DCT coefficients corresponding to the first encoding round can be directly 1. The number of DCT coefficients corresponding to each subsequent encoding round (except the first encoding round) can be determined by (1 + 63 * number of encoded bytes corresponding to each encoding round / total number of bytes in the DCT coefficients). It should be understood that the number of encoded bytes corresponding to each encoding round refers to the number of bits of all bytes in the DCT coefficients that have been encoded up to that encoding round.

[0184] Exemplarily, when the total number of bytes of the DCT coefficient is 8 bits, wherein the first four bits are high-order bytes and the last four bits are low-order bytes, and the first round encodes the DC coefficient, the second round continues to encode the first high-order byte of each AC coefficient, the third round continues to encode the second high-order byte of each AC coefficient, the fourth round continues to encode the third high-order byte of each AC coefficient, the fifth round continues to encode the fourth high-order byte of each AC coefficient, the sixth round continues to encode the first low-order byte of each AC coefficient, the seventh round continues to encode the second low-order byte of each AC coefficient, the eighth round continues to encode the third low-order byte of each AC coefficient, and the ninth round continues to encode the fourth low-order byte of each AC coefficient, the The number of DCT coefficients corresponding to the first round of encoding may be 1, the number of DCT coefficients corresponding to the second round of encoding may be (1+63*1 / 8), the number of DCT coefficients corresponding to the third round of encoding may be (1+63*2 / 8), the number of DCT coefficients corresponding to the fourth round of encoding may be (1+63*3 / 8), the number of DCT coefficients corresponding to the fifth round of encoding may be (1+63*4 / 8), the number of DCT coefficients corresponding to the sixth round of encoding may be (1+63*5 / 8), the number of DCT coefficients corresponding to the seventh round of encoding may be (1+63*6 / 8), the number of DCT coefficients corresponding to the eighth round of encoding may be (1+63*7 / 8), and the number of DCT coefficients corresponding to the ninth round of encoding may be (1+63*8 / 8).

[0185] Therefore, when the minimum number of sampling points is 1, the target encoding round corresponding to the target image can be determined as the first round; when the minimum number of sampling points is 4, the target encoding round corresponding to the target image can be determined as the second round; when the minimum number of sampling points is 8, the target encoding round corresponding to the target image can be determined as the second round; when the minimum number of sampling points is 16, the target encoding round corresponding to the target image can be determined as the third round; when the minimum number of sampling points is 32, the target encoding round corresponding to the target image can be determined as the fifth round, and so on.

[0186] Similar to the JPEG progressive mode based on spectrum selection, in the JPEG progressive mode based on successive approximation, when determining the number of DCT coefficients close to the minimum number of sampling points, the number of DCT coefficients can also be determined from the number of DCT coefficients greater than or equal to the minimum number of sampling points, thereby improving the accuracy of the subsequent predicted image. Furthermore, the determination of the minimum number of sampling points can refer to the determination of the minimum number of sampling points in the JPEG progressive mode based on spectrum selection described above.

[0187] S402 : Acquire an intermediate image corresponding to the original image according to the target encoding round, and downsample the intermediate image using a downsampling algorithm to obtain a predicted image.

[0188] It is understood that after multiple rounds of encoding of the original image based on the JPEG progressive mode, the original encoded file described in Table 1 below can be obtained. This embodiment can obtain partial image data from the original encoded file based on the target encoding round, and can obtain intermediate images by decoding the partial image data. For example, when the target encoding round is the first round, the first-round image data in Table 1 can be decoded to obtain the intermediate image; when the target encoding round is the second round, the first-round image data and the second-round image data in Table 1 can be decoded to obtain the intermediate image; when the target encoding round is the third round, the first-round image data, the second-round image data, and the third-round image data in Table 1 can be decoded to obtain the intermediate image, and so on.

[0189] Table 1

[0190]

[0191] The intermediate image is an approximate version of the original image, meaning that its resolution is the same as the first resolution of the original image, but its clarity is lower than that of the original image. For example, when the intermediate image is obtained based on the first round of image data decoding, since the first round of image data only includes the DC coefficients in each pixel block, the pixel values ​​of all pixels in each pixel block of the intermediate image are the pixel values ​​corresponding to the DC coefficients in that pixel block. This results in the intermediate image having a clarity significantly lower than that of the original image, meaning that a clear, low-resolution image cannot be obtained based on the intermediate image itself. Therefore, in this embodiment, a preset downsampling algorithm can be used to downsample the intermediate image, and the downsampled image can be determined as a predicted image corresponding to the low-resolution target image, thereby obtaining a low-resolution, clear target image based on the predicted image.

[0192] It is understandable that this embodiment does not specifically limit the downsampling algorithm. For example, a linear mean algorithm can be used to downsample the intermediate image. That is, the intermediate image can be first divided into multiple pixel blocks. Then, for each pixel block, the average value of all pixel values ​​in the pixel block is calculated, the calculated average value is determined as a sampling point (i.e., a pixel corresponding to the predicted image), and the image composed of all sampling points is determined as the predicted image. Here, the size of the divided pixel blocks can be determined based on the first resolution of the original image and the second resolution of the target image. For example, when the second resolution is 1 / 64 of the first resolution, the divided pixel blocks can be 8*8 pixel blocks; when the second resolution is 1 / 16 of the first resolution, the divided pixel blocks can be 4*4 pixel blocks; when the second resolution is 1 / 32 of the first resolution, the divided pixel blocks can be 8*4 or 4*8 pixel blocks, and so on.

[0193] When the second resolution is 1 / 64 of the first resolution, the target encoding pass can be the first pass, i.e., the intermediate image can be decoded based on the first-pass image data. In this case, the pixel values ​​of all 64 pixels in any 8*8 pixel block of the intermediate image are identical. That is, the linear mean of each 8*8 pixel block is the same as the pixel value of any pixel in that pixel block. Therefore, a single pixel can be directly sampled from each pixel block to serve as a sampling point corresponding to each pixel block, and the image formed by these sampling points is then determined as the predicted image. When the second resolution is 1 / 16 of the first resolution, the target encoding pass can be the second or third pass, etc. The second-pass or third-pass image data can contain AC coefficients, i.e., richer high-frequency details. Therefore, a linear mean can be calculated on each 4*4 pixel block to obtain a sampling point corresponding to every 16 pixels, and the image formed by these sampling points is then determined as the predicted image.

[0194] S403. Scale the original image according to the second resolution to obtain a scaled image corresponding to the original image, perform differential processing on the scaled image and the predicted image to obtain a differential image, and compress the differential image using a compression algorithm to obtain differential data corresponding to the differential image.

[0195] It is understood that the scaled image refers to an image obtained by directly scaling the original image to the second resolution. In this embodiment, any existing scaling algorithm, such as a nearest neighbor interpolation algorithm or a bilinear interpolation algorithm, can be used to scale the original image to obtain a scaled image at the second resolution. The scaled image and the predicted image can then be subjected to pixel-by-pixel difference processing to obtain a differential image. This differential image can then be compressed and encoded using a preset compression algorithm to obtain differential data corresponding to the differential image. This allows, during image decoding, a low-resolution, clear target image to be obtained based on the predicted image and the differential data.

[0196] Among them, pixel-by-pixel difference processing of the scaled image and the predicted image refers to subtracting the pixel value of each pixel in the scaled image from the pixel at the same position in the preset image to obtain the difference between each pixel, and using the difference between each pixel as the pixel value of the pixel at the same position in the differential image.

[0197] For example, when the pixel value of the first pixel in the scaled image is 231, the pixel value of the second pixel is 224, the pixel value of the third pixel is 217, and the pixel value of the fourth pixel is 203, and the pixel value of the first pixel in the predicted image is 231, the pixel value of the second pixel is 214, the pixel value of the third pixel is 217, and the pixel value of the fourth pixel is 201, then it can be determined that the pixel value of the first pixel in the differential image is 0, the pixel value of the second pixel is 10, the pixel value of the third pixel is 0, and the pixel value of the fourth pixel is 2.

[0198] This embodiment does not specifically limit the compression algorithm used to compress the differential image, and any existing compression algorithm can be used to compress the differential image. However, considering internal consistency and simplicity, the existing JPEG compression algorithm can be used to compress the differential image.

[0199] S404: Obtain a target encoding file of the original image according to the second resolution, the target encoding round, the downsampling algorithm, the compression algorithm, the differential data, and the original encoding file.

[0200] Specifically, the second resolution corresponding to the target image, the target encoding round, the first algorithm type corresponding to the downsampling algorithm, and the second algorithm type corresponding to the compression algorithm can be encoded to obtain a variable resolution file header. The variable resolution file header and differential data can be inserted into the original encoded file corresponding to the JPEG progressive mode as custom extension fields to obtain a target encoded file. For example, the variable resolution file header and differential data can be inserted into the original encoded file described in Table 1 above to obtain the target encoded file described in Table 2 below.

[0201] Table 2

[0202]

[0203] It should be noted that this embodiment does not specifically limit the insertion positions of the variable resolution file header and differential data in the original encoded file. However, in order to conveniently and quickly determine whether the original image can be displayed at a variable resolution, the variable resolution file header can be inserted after or before the original file header in the original encoded file (wherein, the above Table 2 takes insertion after the original file header as an example) to form a new file header of the target encoded file.

[0204] At the same time, in order to obtain only part of the image data in the target coding file, that is, to decode and obtain a low-resolution target image, the differential data can be inserted before or after the image data corresponding to the target coding round in the original coding file (wherein, the above Table 2 is explained by inserting the image data corresponding to the target coding round as an example), so that when decoding and displaying the image, after determining the target coding round corresponding to the low-resolution target image, only the image data and differential data corresponding to the target coding round can be obtained, that is, the low-resolution target image can be decoded, which can effectively reduce the amount of data acquired by the terminal device, reduce the computational complexity of the terminal device for image decoding and display, reduce the processing power consumption of the terminal device, increase the image presentation speed, and improve the user experience. Moreover, when sharing images, it can also effectively reduce the amount of data transmitted by the cloud server or terminal device, reduce the transmission power consumption of the cloud server or terminal device, increase the image transmission speed, and improve the user experience.

[0205] It is understandable that when there are multiple target images, the variable resolution file header may also include the second resolutions, target encoding rounds, first algorithm types corresponding to the downsampling algorithm, and second algorithm types corresponding to the compression algorithm, etc., corresponding to the multiple target images. That is, the variable resolution file header at this time may be a list, and each list item in the list corresponds to a target image. In other words, each list item may include the second resolution, target encoding rounds, first algorithm types corresponding to the downsampling algorithm, and second algorithm types corresponding to the compression algorithm, etc., corresponding to a target image. At this time, the number of inserted differential data corresponds to the number of target images, that is, in this embodiment, corresponding differential data may be inserted into each round of image data, or corresponding differential data may be inserted into the image data of some rounds. For example, as described in Table 1, when there are two target images, corresponding differential data may be inserted after the first round of image data and the third round of image data, respectively.

[0206] In this embodiment, an intermediate image from a certain encoding round in the JPEG progressive mode is selected as a basis, downsampled as a predicted image of a low-resolution target image, and a differential image is obtained based on the predicted image and the scaled image corresponding to the original image. The differential data corresponding to the differential image is then inserted into the original encoded file corresponding to the JPEG progressive mode. This allows the predicted image and differential data to be used together to accurately restore the specific low-resolution target image, thereby achieving variable-resolution image encoding compatible with the JPEG progressive mode. Furthermore, when sharing images encoded using this image encoding method, partial image data can be transmitted based on the display resolution of the receiving device, significantly reducing the amount of data required for transmission, increasing image transmission speed, and enhancing the user experience.

[0207] 2. Image decoding based on JPEG progressive mode

[0208] See also Figure 5 , Figure 5 FIG1 shows a schematic flow chart of the image decoding method provided by this embodiment, which can be applied to a terminal device. Figure 5 As shown, the image decoding method may include:

[0209] S501: Obtain the display resolution of the terminal device, and determine the target resolution of the target image according to the display resolution and the target encoding file.

[0210] When decoding and displaying an original image, the terminal device can first obtain the file header of the target encoded file and, based on the file header, determine the first resolution of the original image and the second resolution supported by the original image. Subsequently, the terminal device can determine the target resolution of the target image based on its own display resolution and the second resolution supported by the original image. The target resolution is any one of the second resolutions. Specifically, the terminal device can determine the target resolution of the target image as the second resolution supported by the original image that is closest to the terminal device's display resolution.

[0211] It should be understood that the original image can be an image transmitted to the terminal device in real time from a cloud server or other terminal device, or it can be an image stored in the terminal device. The target encoded file can be an encoded file obtained by encoding the original image using the aforementioned image encoding method. The second resolution is a resolution lower than the first resolution of the original image, that is, the second resolution is a low resolution supported by the original image. The target image is a low-resolution version of the original image, and the target image is the image to be displayed on the terminal device.

[0212] For example, when the second resolution supported by the original image includes resolution A, resolution B and resolution C, and resolution A is 1 / 64 of the first resolution, resolution B is 1 / 32 of the first resolution, and resolution C is 1 / 16 of the first resolution, if the display resolution of the terminal device is 1 / 128 of the first resolution, then the target resolution of the target image can be determined to be resolution A; if the display resolution of the terminal device is 1 / 32 of the first resolution, then the target resolution of the target image can be determined to be resolution B, and so on.

[0213] S502: Determine a target encoding round corresponding to the target image according to the target resolution, and obtain image data and differential data corresponding to the target image according to the target encoding round.

[0214] It is understandable that the variable resolution file header of the target encoding file records the encoding rounds corresponding to each second resolution. Therefore, after determining the target resolution of the target image (i.e., a certain second resolution), the terminal device can determine the target encoding round corresponding to the target image based on the target resolution and the variable resolution file header in the target encoding file. Subsequently, the terminal device can obtain all image data up to the target encoding round and the differential data corresponding to the target encoding round, that is, only part of the image data and differential data need to be obtained to decode and restore the target image, without having to obtain all the image data in the target encoding file. This greatly reduces the amount of image data obtained and can effectively improve the rendering speed of the target image.

[0215] For example, when the target encoding round is the first round, the terminal device can obtain the first round image data and the differential data corresponding to the first round. When the target encoding round is the third round, the terminal device can obtain the first round image data, the second round image data, and the third round image data, as well as the differential data corresponding to the third round, and so on.

[0216] S503 : Decode the image data to obtain an intermediate image, and downsample the intermediate image using a downsampling algorithm to obtain a predicted image.

[0217] The variable resolution file header of the target encoding file records the first algorithm type of the downsampling algorithm corresponding to each second resolution. In this embodiment, the terminal device can determine the downsampling algorithm based on the first algorithm type corresponding to the target resolution in the variable resolution file header. That is, the terminal device can use the downsampling algorithm used in image encoding to downsample the intermediate image to obtain a predicted image. In other words, the downsampling algorithm used for the intermediate image in the image decoding process is the same as the downsampling algorithm used for the intermediate image in the image encoding process, so that a clear, low-resolution image can be obtained later. Among them, the process of downsampling the intermediate image using the downsampling algorithm can refer to the downsampling process in the aforementioned image encoding and will not be repeated here.

[0218] S504 : Decode the differential data to obtain a differential image, and generate a target image of a target resolution according to the predicted image and the differential image.

[0219] It should be understood that the variable-resolution file header of the target coded file also records the second algorithm type of the compression algorithm corresponding to the differential data associated with each second resolution. In this embodiment, the terminal device can determine the compression algorithm corresponding to the differential image based on the second algorithm type corresponding to the target resolution in the variable-resolution file header, and can decode and restore the differential data based on the compression algorithm to obtain the differential image, thereby obtaining the target image of the second resolution based on the differential image and the predicted image.

[0220] Specifically, the terminal device can merge the predicted image and the differential image, for example, it can add the pixel values ​​of the pixels at the same position in the predicted image and the differential image, and can use the added pixel value as the pixel value of the pixel at the same position in the target image, thereby obtaining a low-resolution, clear target image.

[0221] For example, when the pixel value of the first pixel in the differential image is 0, the pixel value of the second pixel is 10, the pixel value of the third pixel is 0, and the pixel value of the fourth pixel is 2, and the pixel value of the first pixel in the predicted image is 231, the pixel value of the second pixel is 214, the pixel value of the third pixel is 217, and the pixel value of the fourth pixel is 201, then it can be determined that the pixel value of the first pixel in the target image is 231, the pixel value of the second pixel is 224, the pixel value of the third pixel is 217, and the pixel value of the fourth pixel is 203.

[0222] It should be noted that when the terminal device does not support variable-resolution display of the original image, that is, when the terminal device cannot recognize the custom extension field in the target coded file, the terminal device can ignore the custom extension field in the target coded file, that is, it can ignore the variable-resolution file header and differential data in the target coded file, and can directly decode and display the image data according to the original JPEG progressive mode decoding method, that is, it can directly obtain each round of image data for decoding and display. Alternatively, when the display resolution of the terminal device is similar to the first resolution of the original image, the terminal device can also directly decode and display the image data according to the original JPEG progressive mode decoding method.

[0223] In this embodiment, when the display resolution of the terminal device is lower than the first resolution of the original image, the terminal device can obtain partial image data and differential data based on its own display resolution and the second resolution that the original image can support for display, so as to decode the partial image data and differential data, that is, to obtain a target image that meets the display requirements of the terminal device, which can effectively reduce the amount of data acquired when the terminal device displays an image, reduce the computational complexity when the terminal device decodes and displays an image, reduce the processing power consumption of the terminal device, increase the speed at which the terminal device presents images, and enhance the user experience.

[0224] [Example 2]

[0225] The image encoding method provided in this embodiment expands upon the PNG interlaced scanning mode to implement variable-resolution image encoding compatible with the PNG interlaced scanning mode. Therefore, when decoding and displaying an image encoded using this image encoding method, a terminal device can decode to obtain a specific low-resolution image or the original high-resolution image, meeting the display requirements of different scenarios. The following describes image encoding and decoding based on the PNG interlaced scanning mode in detail.

[0226] 1. Image encoding based on PNG interlaced scanning mode

[0227] PNG interlaced scanning mode uses a 7-pass interlaced scanning algorithm (Adam7) to perform multiple scans of the image in the spatial domain, with each scan covering only a portion of the image's pixels. This means that PNG interlaced scanning mode progressively replenishes image data over seven passes, initially quickly displaying the image's outline. Then, as the number of scan passes increases, the image data gradually increases, thereby gradually improving image clarity.

[0228] See also Figure 6 , Figure 6 FIG. 4 shows a scanning schematic diagram of the PNG interlaced scanning mode, wherein: Figure 6The numbers in are used to indicate the scanning rounds corresponding to the pixels at that position. Figure 6 As shown in (a) in the figure, the first round of scanning can start from the upper left corner of the original image (assuming this coordinate is (0, 0)) and perform pixel scanning with an offset of 8 pixels. That is, the pixels of the first scan can be (0, 0), (8, 0), ..., (0, 8), (8, 8), and so on. Figure 6 As shown in (b) in the figure, the second round of scanning starts from each pixel obtained in the first round of scanning, and scans pixels in the horizontal direction with 4 pixels as the offset unit. That is, the pixels of the second scan can be (4,0), (12,0), ..., (4,8), (12,8), and so on. Figure 6 As shown in (c) in the figure, the third round of scanning starts from each pixel obtained in the second round of scanning, and uses 4 pixels as the offset unit to perform pixel scanning in the vertical direction. That is, the pixels of the third scan can be (0,4), (4,4), (8,4), (12,4), ..., (0,12), (4,12), (8,12), (12,12), and so on. Figure 6 As shown in (d) in the figure, the fourth round of scanning starts from each pixel obtained in the third round of scanning and performs pixel scanning in the horizontal direction with 2 pixels as the offset unit. That is, the pixels of the fourth scan can be (2,0), (6,0), (10,0), (14,0) ..., (2,4), (6,4), (10,4), (14,4), ..., (2,8), (6,8), (10,8), (14,8), ..., (2,12), (6,12), (10,12), (14,12), and so on; Figure 6 As shown in (e) in the figure, the fifth round of scanning takes each pixel obtained in the fourth round of scanning as the starting point, and performs pixel scanning in the vertical direction with 2 pixels as the offset unit. That is, the pixels of the fifth scan can be (0,2), (2,2), (4,2), (6,2), ... (14,2), ... (0,6), (2,6), (4,6), (6,6), ... (14,6), ... (0,10), (2,10), (4,10), ... (14,10), ... (0,14), (2,14), (4,14), ... (14,14), and so on; Figure 6 As shown in (f) in the figure, the sixth round of scanning starts from each pixel obtained in the fifth round of scanning and uses 1 pixel as the offset unit to perform pixel scanning in the horizontal direction. That is, the pixels scanned in the sixth round can be (1,0), (3,0), (5,0), ... (15,0), ... (1,3), (3,3), (5,3), ... (15,3), ... (1,14), (3,14), (5,14), ... (15,14), and so on; Figure 6As shown in (g) in FIG, the seventh round of scanning is all the remaining unscanned pixels, that is, all the pixels in the 1st row, the 3rd row, ..., the 15th row.

[0229] In this embodiment, image encoding based on the PNG interlaced scanning mode can be encoded according to the scanning order of the PNG interlaced scanning mode, that is, the first round can encode the pixels scanned in the first round in the PNG interlaced scanning mode, the second round can encode the pixels scanned in the second round in the PNG interlaced scanning mode, the third round can encode the pixels scanned in the third round in the PNG interlaced scanning mode, the fourth round can encode the pixels scanned in the fourth round in the PNG interlaced scanning mode, the fifth round can encode the pixels scanned in the fifth round in the PNG interlaced scanning mode, the sixth round can encode the pixels scanned in the sixth round in the PNG interlaced scanning mode, and the seventh round can encode the pixels scanned in the seventh round in the PNG interlaced scanning mode.

[0230] Similar to the image encoding based on the JPEG progressive mode in the first embodiment, the image encoding method based on the PNG interlaced scanning mode provided in this embodiment may also include:

[0231] 1. Obtain a first resolution of the original image and a second resolution of the target image, and determine a target encoding round corresponding to the target image according to the first resolution and the second resolution.

[0232] Here, the method for determining the first resolution and the second resolution is the same as the method for determining the first resolution and the second resolution in S401 of the first embodiment. For details, please refer to the description of S401 and will not be repeated here.

[0233] In this embodiment, the terminal device can determine the target encoding round corresponding to the target image based on the first resolution of the original image, the second resolution of the target image, and the number of pixels corresponding to each encoding round. Similar to the first embodiment, the description in this embodiment uses any 8*8 pixel block as an example. That is, the number of pixels described below refers to the number of pixels in an 8*8 pixel block, and the minimum number of sampling points refers to the minimum number of points that need to be sampled in an 8*8 pixel block.

[0234] For example, the minimum number of sampling points required for the target image can be determined based on the first and second resolutions. Then, a number of pixels close to this minimum number of sampling points can be determined, and the encoding round corresponding to the determined number of pixels can be determined as the encoding round corresponding to the target image. The number of pixels corresponding to each encoding round refers to the total number of pixels encoded up to that encoding round.

[0235] For example, Figure 6 As shown in (a) in , the number of pixels corresponding to the first round of encoding can be 1; Figure 6As shown in (b) in , the number of pixels corresponding to the second round of encoding can be 2; Figure 6 As shown in (c) in FIG, the number of pixels corresponding to the third round of encoding can be 4; Figure 6 As shown in (d) in FIG, the number of pixels corresponding to the fourth round of encoding can be 8; Figure 6 As shown in (e) in FIG, the number of pixels corresponding to the fifth round of encoding can be 16; Figure 6 As shown in (f) in FIG, the number of pixels corresponding to the sixth round of encoding can be 32; Figure 6 As shown in (g) in FIG, the number of pixels corresponding to the seventh encoding round can be 64. Therefore, when the minimum number of sampling points is 1, that is, when the second resolution of the target image is 1 / 64 of the first resolution of the original image, the target encoding round corresponding to the target image can be determined as the first round; when the minimum number of sampling points is 4, that is, when the second resolution of the target image is 1 / 16 of the first resolution of the original image, the target encoding round corresponding to the target image can be determined as the third round; when the minimum number of sampling points is 16, that is, when the second resolution of the target image is 1 / 4 of the first resolution of the original image, the target encoding round corresponding to the target image can be determined as the fifth round, and so on.

[0236] It should be understood that the method for determining the minimum number of sampling points is the same as that in the first embodiment. For details, reference may be made to the description in the first embodiment and will not be repeated here.

[0237] 2. Obtain the intermediate image corresponding to the original image according to the target encoding round, and use the downsampling algorithm to downsample the intermediate image to obtain the predicted image.

[0238] Similar to Example 1, after multiple rounds of encoding of the original image based on the PNG interlaced scanning mode, the original encoded file described in Table 1 above can be obtained. At this time, the first round image data, the second round image data, the third round image data, ..., the nth round image data in Table 1 all refer to data obtained by directly encoding the pixels. This embodiment can obtain part of the image data in the original encoded file according to the target encoding round, and can obtain an intermediate image by decoding part of the image data. Among them, the content of downsampling the intermediate image using the downsampling algorithm to obtain the predicted image is the same as the content of "downsampling the intermediate image using the downsampling algorithm to obtain the predicted image" in Example 1. The specific content can refer to the description in Example 1 and will not be repeated here.

[0239] 3. Scale the original image according to the second resolution to obtain a scaled image corresponding to the original image, perform differential processing on the scaled image and the predicted image to obtain a differential image, and compress the differential image using a compression algorithm to obtain differential data corresponding to the differential image.

[0240] Here, the process of obtaining the differential data is the same as the process of obtaining the differential data in S403 of the first embodiment. For specific content, please refer to the description of S403 of the first embodiment, and will not be repeated here.

[0241] 4. Obtain a target encoding file of the original image according to the second resolution, the target encoding round, the downsampling algorithm, the compression algorithm, the differential data, and the original encoding file.

[0242] Similarly, the process of obtaining the target encoding file of the original image based on the second resolution, target encoding round, downsampling algorithm, compression algorithm, differential data and original encoding file is the same as the process of obtaining the target encoding file in S404 of Example 1. The specific content can be referred to the description in S404 of Example 1 and will not be repeated here.

[0243] In this embodiment, an intermediate image from a certain encoding round of the PNG interlaced scanning mode is selected as a basis, downsampled as a predicted image of a low-resolution image, and a differential image can be obtained based on the predicted image and the scaled image corresponding to the original image. Then, the differential data corresponding to the differential image is inserted into the original encoded file corresponding to the PNG interlaced scanning mode, so that the specific low-resolution image can be accurately restored by combining the predicted image and the differential data, thereby achieving variable-resolution image encoding compatible with the PNG interlaced scanning mode. At the same time, when sharing images encoded based on this image encoding method, partial image data can be transmitted according to the display resolution of the receiving device, thereby significantly reducing the amount of data to be transmitted, increasing image transmission speed, and enhancing the user experience.

[0244] 2. Image decoding based on PNG interlaced scanning mode

[0245] Similar to the image decoding based on the JPEG progressive mode in the first embodiment, the image decoding method based on the PNG interlaced scanning mode provided in this embodiment may also include:

[0246] 1. Obtain the display resolution of the terminal device and determine the target resolution of the target image based on the display resolution and the target encoding file.

[0247] Similar to S501 in Example 1, when decoding and displaying the original image is required, the terminal device can obtain the file header of the target encoded file and, based on the file header, determine the first resolution of the original image and the second resolution that the original image can support for display. Subsequently, the terminal device can determine the target resolution of the target image based on its own display resolution and the second resolution that the original image can support for display. The target resolution can be any of the second resolutions. The target encoded file can be an encoded file obtained by encoding the original image using the image encoding method described above in this embodiment.

[0248] Specifically, the terminal device may determine the second resolution that is closest to the display resolution of the terminal device among the second resolutions supported for display of the original image as the target resolution of the target image.

[0249] 2. Determine the target encoding round corresponding to the target image according to the target resolution, and obtain the image data and differential data corresponding to the target image according to the target encoding round.

[0250] Similar to S502 in Example 1, the variable-resolution file header of the target coded file records the encoding rounds corresponding to each second resolution. Therefore, after determining the target resolution of the target image (i.e., a certain second resolution), the terminal device can determine the target encoding round corresponding to the target image based on the target resolution and the variable-resolution file header in the target coded file. Subsequently, the terminal device can obtain all image data up to the target encoding round and the differential data corresponding to the target encoding round. In other words, only the partial image data and differential data corresponding to the target image need to be obtained, without obtaining all the image data in the target coded file. This greatly reduces the amount of image data to be obtained and can effectively improve the rendering speed of the target image.

[0251] 3. Decode the image data to obtain an intermediate image, and use a downsampling algorithm to downsample the intermediate image to obtain a predicted image.

[0252] Similar to S503 in Example 1, the variable resolution file header of the target coded file records the first algorithm type of the downsampling algorithm corresponding to each second resolution. In this embodiment, the terminal device can determine the downsampling algorithm based on the first algorithm type corresponding to the target resolution in the variable resolution file header. That is, the terminal device can use the downsampling algorithm used in image encoding to downsample the intermediate image to obtain a predicted image. In other words, the downsampling algorithm used for the intermediate image during image decoding is the same as the downsampling algorithm used for the intermediate image during image encoding, thereby facilitating subsequent decoding to obtain a clear, low-resolution image.

[0253] 4. Decode the differential data to obtain a differential image, and generate a target image of target resolution based on the predicted image and the differential image.

[0254] Similar to S504 in the first embodiment, the variable resolution file header of the target encoded file also records the second algorithm type of the compression algorithm corresponding to the differential data associated with each second resolution. In this embodiment, the terminal device can determine the compression algorithm corresponding to the differential image based on the second algorithm type corresponding to the target resolution in the variable resolution file header, and thus decode and restore the differential data according to the compression algorithm to obtain the differential image. The terminal device can then merge the predicted image and the differential image, for example, by adding the pixel values ​​of the pixels at the same position in the predicted image and the differential image, and using the resulting pixel value as the pixel value of the pixel at the same position in the target image, thereby obtaining a low-resolution, clear target image.

[0255] It should be noted that when the terminal device does not support variable-resolution display of the original image, that is, when the terminal device cannot recognize the custom extension field in the target encoded file, the terminal device can ignore the custom extension field in the target encoded file, that is, it can ignore the variable-resolution file header and differential data in the target encoded file, and can directly decode and display the image data according to the decoding method corresponding to the original PNG interlaced scanning mode, that is, it can directly obtain each round of image data for decoding and display. Alternatively, when the display resolution of the terminal device is similar to the first resolution of the original image, the terminal device can also directly decode and display the image data according to the decoding method corresponding to the original PNG interlaced scanning mode.

[0256] In this embodiment, when the display resolution of the terminal device is lower than the first resolution of the original image, the terminal device can obtain partial image data and differential data based on its own display resolution and the target resolution that the original image can support for display, so as to decode the partial image data and differential data, that is, to obtain a target image that meets the display requirements of the terminal device, which can effectively reduce the amount of data acquired when the terminal device displays an image, reduce the computational complexity when the terminal device decodes and displays an image, reduce the processing power consumption of the terminal device, increase the speed at which the terminal device presents images, and enhance the user experience.

[0257] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0258] The embodiment of the present application also provides a terminal device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, the terminal device implements the steps of any of the above-mentioned method embodiments. For example, the structure of the terminal device can be as follows: Figure 1 shown.

[0259] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a computer, the computer is enabled to implement the steps of any of the above method embodiments.

[0260] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device implements the steps of any of the above method embodiments.

[0261] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, a computer-readable storage medium cannot be an electric carrier signal or a telecommunication signal.

[0262] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0263] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0264] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0265] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0266] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. An image coding method, characterized in that: include: Obtaining a first resolution of an original image and a second resolution of a target image, and determining a target encoding round corresponding to the target image according to the first resolution and the second resolution; The target image is a low-resolution version of the original image; Acquire an intermediate image corresponding to the original image according to the target encoding round, and downsample the intermediate image using a downsampling algorithm to obtain a predicted image; Scaling the original image according to the second resolution to obtain a scaled image corresponding to the original image, and performing a difference process between the scaled image and the predicted image to obtain a difference image; compressing the differential image using a compression algorithm to obtain differential data corresponding to the differential image; A target encoding file of the original image is obtained according to the second resolution, the target encoding round, the downsampling algorithm, the compression algorithm, the differential data and the original encoding file, wherein the original encoding file is an encoding file obtained by encoding the original image using a JPEG progressive mode or a PNG interlaced scanning mode that does not have the capability of varying resolution; the target encoding file is an encoding file after a variable resolution file header and the differential data are inserted into the original encoding file, wherein the variable resolution file header is obtained by encoding the second resolution, the target encoding round, a first algorithm type corresponding to the downsampling algorithm, and a second algorithm type corresponding to the compression algorithm.

2. The method according to claim 1, characterized in that When the original encoded file is an encoded file obtained by encoding the original image in the JPEG progressive mode, determining the target encoding round corresponding to the target image according to the first resolution and the second resolution includes: Obtaining the number of encoded coefficients corresponding to each encoding round according to the original encoding file; A target coding round corresponding to the target image is determined according to the first resolution, the second resolution, and the number of encoded coefficients, where the target coding round is any one of the encoding rounds.

3. The method according to claim 1, characterized in that When the original encoded file is an encoded file obtained by encoding the original image in the JPEG progressive mode, determining the target encoding round corresponding to the target image according to the first resolution and the second resolution includes: Obtaining the total number of bytes of discrete cosine transform coefficients and the number of encoded bytes corresponding to each encoding round according to the original encoding file; A target encoding round corresponding to the target image is determined according to the first resolution, the second resolution, the total number of bytes, and the number of encoded bytes, where the target encoding round is any one of the encoding rounds.

4. The method according to claim 1, wherein When the original encoded file is an encoded file obtained by encoding the original image in the PNG interlaced scanning mode, determining the target encoding round corresponding to the target image according to the first resolution and the second resolution includes: Obtaining the number of encoded pixels corresponding to each encoding round according to the original encoding file; A target encoding round corresponding to the target image is determined according to the first resolution, the second resolution, and the number of encoded pixels, where the target encoding round is any one of the encoding rounds.

5. The method according to any one of claims 1 to 4, characterized in that The obtaining, according to the target encoding round, an intermediate image corresponding to the original image includes: Image data corresponding to the target encoding round is obtained from the original encoding file, and the image data is decoded to obtain an intermediate image corresponding to the original image.

6. The method according to any one of claims 1 to 5, characterized in that The obtaining of the target encoding file of the original image according to the second resolution, the target encoding round, the downsampling algorithm, the compression algorithm, the differential data, and the original encoding file includes: Determining a first position of the image data corresponding to the target encoding round in the original encoded file, and determining a second position of the differential data in the original encoded file based on the first position; inserting the differential data into the second position; Encoding the second resolution, the target encoding round, the first algorithm type of the downsampling algorithm, and the second algorithm type of the compression algorithm to obtain a variable resolution file header; The variable resolution file header is inserted into the original file header of the original encoded file.

7. An image decoding method, characterized in that: include: Obtaining a display resolution of a terminal device, and determining a target resolution of a target image based on the display resolution and a target encoding file, wherein the target encoding file is an encoding file obtained by encoding according to any one of claims 1 to 6; The target resolution is one of the second resolutions, where the second resolution is a resolution supported by the original image for display; Determining a target encoding round corresponding to the target image according to the target resolution, and acquiring image data and differential data corresponding to the target image according to the target encoding round; Decoding the image data to obtain an intermediate image, and downsampling the intermediate image using a downsampling algorithm to obtain a predicted image; The differential data is decoded to obtain a differential image, and a target image of the target resolution is generated according to the predicted image and the differential image.

8. The method according to claim 7, characterized in that The downsampling the intermediate image using a downsampling algorithm to obtain a predicted image includes: A downsampling algorithm corresponding to the intermediate image is acquired according to the target resolution, and the intermediate image is downsampled using the downsampling algorithm to obtain the predicted image.

9. The method according to claim 7 or 8, characterized in that The decoding of the differential data to obtain a differential image includes: A compression algorithm corresponding to the differential image is acquired according to the target resolution, and the differential data is decoded according to the compression algorithm to obtain the differential image.

10. The method according to any one of claims 7 to 9, characterized in that Generating the target image of the target resolution according to the predicted image and the differential image includes: The predicted image and the differential image are merged to obtain a target image of the target resolution.

11. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the terminal device is caused to implement the method according to any one of claims 1 to 10.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a computer, the computer is caused to implement the method according to any one of claims 1 to 10.

Images