Image processing method, related method, device, equipment and storage medium

Abstract

The invention discloses an image processing method, a related method, a related device, equipment and a storage medium. Comprising the following steps: acquiring image binary data of an original loading image; performing image decoding on the image binary data to obtain a plurality of decoding matrixes; performing inverse quantization on each decoding matrix according to a target quantization factor determined by the screen resolution to obtain a plurality of corresponding discrete cosine transform coefficient matrixes; and performing image merging based on color components corresponding to image pixel blocks obtained through inverse discrete cosine transform of each discrete cosine transform coefficient matrix, and generating a target loading image corresponding to the screen resolution according to the merged image under each color component. According to the embodiment of the invention, the target quantization factor determined by the screen resolution can zoom the pixels of the original loading image to be adaptive to the screen resolution during inverse quantization, and the target loading image adaptive to the screen resolution is generated, so that the pre-storage of the loading image is avoided, and the consumption of storage resources is reduced. The method can be used for various scenes such as image self-adaption and image processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and more particularly, to an image processing method and related method, device, equipment and storage medium. BACKGROUND

[0002] With the rapid development of Internet technology and communication technology, various communication electronic consumer products, such as desktop computers, tablet computers, mobile phones, etc., have become indispensable items in people's daily life. The graphical user interface (GUI) displayed by the application software in the electronic consumer product is a bridge for data transmission and interactive control between the hardware and software system of the electronic product and the user.

[0003] The graphical user interface itself and the images included therein are indispensable elements for the design and development of software products. For example, the application loading page in the application interface, the icons displayed in the form of images, and the images in the web interface. Since there are certain differences in the display requirements of image loading between electronic devices and operating systems, how to implement loading adaptive images for different types of devices and screens of different sizes has become an important problem.

[0004] Currently, the related technology proposes to design loading images suitable for the screen of different types of terminals. For example, the server can respond to the requests of different terminals and return adaptive loading images to the terminals. Each terminal can also save the loading images adapted to its screen resolution locally. However, these methods require the server or the terminal to pre-store multiple loading images, resulting in the consumption of a large amount of storage resources by the server and the terminal. SUMMARY

[0005] The embodiments of the present application provide an image processing method and related method, device, equipment and storage medium. To solve the problem that the server and the terminal consume a large amount of storage resources due to the pre-storage of multiple loading images in the related technology.

[0006] According to an aspect of the present application, an image processing method is provided, which comprises: obtaining image binary data obtained by image encoding of an original loading image; determining a target image format based on the image binary data, and performing image decoding on the image binary data through the target image format to obtain a plurality of decoding matrices; performing inverse quantization operation on each decoding matrix respectively according to a target quantization factor determined according to a screen resolution to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix; performing inverse discrete cosine transform on each discrete cosine transform coefficient matrix respectively to obtain an image pixel block corresponding to each discrete cosine transform coefficient matrix; performing image merging based on a color component corresponding to each image pixel block to obtain a merged image under each color component, and generating a target loading image corresponding to the screen resolution according to the merged image under each color component.

[0007] According to an aspect of the present application, an image encoding method is provided, which comprises: obtaining an original loading image; obtaining a screen resolution of a terminal in response to an image resource request sent by the terminal; when it is detected that image binary data corresponding to the screen resolution has not been returned, performing image encoding on the original loading image to obtain corresponding image binary data; determining a target compression strategy according to a network state and a computing performance of the terminal; performing data compression on the image binary data based on the target compression strategy to obtain image compression data, and returning the image compression data to the terminal, so that the terminal generates a target loading image based on the image compression data, wherein the target loading image is an image generated according to the above-mentioned image processing method.

[0008] According to an aspect of the present application, an image processing device is provided, which comprises:

[0009] A data obtaining module is configured to obtain image binary data obtained by image encoding of an original loading image;

[0010] An image decoding module is configured to determine a target image format based on the image binary data, and perform image decoding on the image binary data through the target image format to obtain a plurality of decoding matrices;

[0011] An inverse quantization module is configured to perform inverse quantization operation on each decoding matrix respectively according to a target quantization factor determined according to a screen resolution to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix;

[0012] An inverse transform module is configured to perform inverse discrete cosine transform on each discrete cosine transform coefficient matrix respectively to obtain an image pixel block corresponding to each discrete cosine transform coefficient matrix;

[0013] The image generation module is configured to perform image merging based on the color components corresponding to each of the image pixel blocks, to obtain a merged image under each color component, and to generate a target loading image corresponding to the screen resolution according to the merged image under each color component.

[0014] Optionally, the image decoding module can include an analysis unit, a reading unit, a determination unit, and a decoding unit. The analysis unit is configured to perform data analysis on the image binary data to obtain a first byte in the image binary data. The reading unit is configured to perform byte reading according to the first byte to determine a target image format corresponding to the image binary data. The determination unit is configured to determine a target decoding algorithm corresponding to the target image format. The decoding unit is configured to perform image decoding on the image binary data by using the target decoding algorithm to obtain a plurality of luminance decoding matrices, a plurality of blue decoding matrices, and a plurality of red decoding matrices, and determine the plurality of luminance decoding matrices, the plurality of blue decoding matrices, and the plurality of red decoding matrices as a plurality of decoding matrices.

[0015] Optionally, the discrete cosine transform coefficient matrix includes a luminance coefficient matrix, a blue coefficient matrix, and a red coefficient matrix. The inverse quantization module includes an acquisition unit, an analysis unit, an adjustment unit, a first inverse quantization unit, a second inverse quantization unit, and a third inverse quantization unit. The acquisition unit is configured to acquire a screen resolution. The analysis unit is configured to perform data analysis on the image binary data to obtain a first original quantization factor and a second original quantization factor. The adjustment unit is configured to adjust the first original quantization factor and the second original quantization factor based on the screen resolution to obtain a corresponding first quantization factor and a second quantization factor. The first inverse quantization unit is configured to perform inverse quantization operation on the plurality of luminance decoding matrices based on the first quantization factor to obtain a corresponding plurality of luminance coefficient matrices. The second inverse quantization unit is configured to perform inverse quantization operation on the plurality of blue decoding matrices according to the second quantization factor to obtain a corresponding plurality of blue coefficient matrices. The third inverse quantization unit is configured to perform inverse quantization operation on the plurality of red decoding matrices by using the second quantization factor to obtain a corresponding plurality of red coefficient matrices.

[0016] Optionally, the adjustment unit can be specifically configured to obtain a scaling factor based on the screen resolution and an original resolution of the original loading image, determine a reciprocal of the scaling factor as a basic quantization factor, adjust the first original quantization factor according to the basic quantization factor to obtain the first quantization factor, and adjust the second original quantization factor by using the basic quantization factor to obtain the second quantization factor.

[0017] Optionally, the image pixel blocks comprise a luminance pixel block, a blue pixel block and a red pixel block; the inverse transform module can comprise a first transform unit, a second transform unit and a third transform unit. The first transform unit is configured to perform inverse discrete cosine transform on the plurality of luminance coefficient matrices to obtain a luminance pixel block corresponding to each luminance coefficient matrix; the second transform unit is configured to perform inverse discrete cosine transform on the plurality of blue coefficient matrices to obtain a blue pixel block corresponding to each blue coefficient matrix; and the third transform unit is configured to perform inverse discrete cosine transform on the plurality of red coefficient matrices to obtain a red pixel block corresponding to each red coefficient matrix.

[0018] Optionally, the image processing apparatus can further comprise a scaling module configured to perform pixel scaling on each of the image pixel blocks based on the scaling factor to obtain a scaled pixel block corresponding to each discrete cosine transform coefficient matrix; and the image generation module can be further configured to perform image merging based on a color component corresponding to each of the scaled pixel blocks to obtain a merged image under each color component, and generate the target loading image corresponding to the screen resolution according to the merged image under each color component.

[0019] Optionally, the image processing apparatus can further comprise a memory allocation module configured to: obtain an image resolution and a bit depth of the original loading image from the image binary data; and allocate a preset memory space based on the image resolution and the bit depth; and the image generation module can be further configured to store the obtained merged image under each color component based on the preset memory space.

[0020] Optionally, the image generation module can comprise a thread acquisition unit, a region determination unit and an image merging unit. The thread acquisition unit is configured to acquire a plurality of concurrent processes; the region determination unit is configured to determine a merging region under different color components based on a number of the plurality of concurrent processes and the image resolution; and the image merging unit is configured to perform image merging on image pixel blocks in the merging region by each of the concurrent processes to obtain a merged image under each color component.

[0021] Optionally, the color components comprise a luminance component, a blue component and a red component, and the merged images comprise a luminance merged image, a blue merged image and a red merged image; and the image merging unit can be specifically configured to: perform image merging on image pixel blocks in the merging region under the luminance component by each of the concurrent processes to obtain the luminance merged image; perform image merging on image pixel blocks in the merging region under the blue component by each of the concurrent processes to obtain the blue merged image; and perform image merging on image pixel blocks in the merging region under the red component by each of the concurrent processes to obtain the red merged image.

[0022] Optionally, the image generation module can further include a space conversion unit, a channel merging unit, and an image optimization unit. The space conversion unit is configured to perform color space conversion on the luminance merged image, the blue merged image, and the red merged image respectively to obtain a red channel image, a green channel image, and a blue channel image. The channel merging unit is configured to perform channel merging on the red channel image, the green channel image, and the blue channel image to obtain an intermediate image. The image optimization unit is configured to perform an optimization operation on the intermediate image to obtain a target loading image corresponding to the screen resolution, wherein the optimization operation at least includes one of image scaling or image rendering.

[0023] Optionally, the image processing apparatus can further include a query module and a request module. The query module is configured to query whether a target loading image is cached locally in response to an image loading instruction. The request module is configured to send an image resource request to a server when the target loading image is not cached locally, so that the server performs data compression on image binary data obtained by encoding an original loading image, and returns compressed image compression data, wherein the compressed image compression data at least includes the image binary data.

[0024] Optionally, the data acquisition module can be specifically configured to: acquire the image compression data returned by the server; and perform data decompression on the image compression data to obtain image binary data obtained by encoding the original loading image.

[0025] According to an aspect of the present application, an image encoding apparatus is provided, which includes:

[0026] An image acquisition module is configured to acquire an original loading image.

[0027] A request response module is configured to acquire a screen resolution of a terminal in response to an image resource request sent by the terminal.

[0028] A data detection module is configured to perform image encoding on the original loading image to obtain corresponding image binary data when it is detected that the image binary data corresponding to the screen resolution has not been returned.

[0029] A strategy determination module is configured to determine a target compression strategy according to a network state and a computing performance of the terminal.

[0030] A data transmission module is configured to perform data compression on the image binary data based on the target compression strategy to obtain image compression data, and return the image compression data to the terminal, so that the terminal generates a target loading image based on the image compression data, wherein the target loading image is an image generated according to the image processing method.

[0031] According to an aspect of the present application, a computer readable storage medium is provided, which stores a computer program, wherein the computer program, when executed by a processor, performs the image processing method or the image encoding method.

[0032] According to an aspect of the present application, a computer device is provided, which comprises a processor and a memory, the memory storing a computer program, the computer program, when invoked by the processor, performing the image processing method or the image encoding method.

[0033] According to an aspect of the present application, a computer program product is provided, which comprises a computer program stored in a storage medium; a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that the computer device performs the image processing method or the image encoding method.

[0034] The embodiment of the present application can obtain image binary data of the original loaded image obtained by image encoding, determine a target image format based on the image binary data, and perform image decoding on the image binary data through the target image format to obtain a plurality of decoding matrices. Then, the inverse quantization operation is performed on each decoding matrix respectively according to a target quantization factor determined according to the screen resolution to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix. Since the target quantization factor is determined according to the screen resolution, the inverse quantization operation of the plurality of decoding matrices can be adapted to the screen resolution by using the target quantization factor, and the pixel information of the original loaded image is scaled to an adaptive degree that meets the screen display.

[0035] Further, the inverse discrete cosine transform is performed on each discrete cosine transform coefficient matrix respectively to obtain an image pixel block corresponding to each discrete cosine transform coefficient matrix. Then, image merging is performed based on the color components corresponding to each image pixel block to obtain a merged image under each color component, and a target loading image corresponding to the screen resolution is generated according to the merged image under each color component. In this way, when the terminal generates the target loading image based on the image binary data of the original loading image returned by the server, the target quantization factor determined by the screen resolution can guide the scaling of the pixels of the original loading image in the inverse quantization process, so that the scaled image pixels can be adapted to the screen resolution, and based on this, the target loading image adapted to the screen resolution can be generated through the inverse discrete cosine transform and image merging, and then the image is adaptively loaded while avoiding the terminal to pre-store the image to be loaded locally, thereby effectively reducing the consumption of storage resources. In addition, the server only needs to pre-store one original loading image to meet the adaptive loading of images by different versions or different types of terminals, instead of pre-storing multiple loading images for terminals with different resolutions, so that the storage resources consumed by the server to store the loading image can be effectively reduced, and the storage space of the server is optimized.

[0036] Other features and advantages of the present application will be set forth in the following description, and in part will be apparent from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the description and claims. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 A system architecture diagram provided by an embodiment of the present application is shown.

[0039] Figure 2 A system deployment diagram provided by an embodiment of the present application is shown.

[0040] Figure 3 An application scenario diagram of an image processing method provided by an embodiment of the present application is shown.

[0041] Figure 4 An application scenario diagram of another image processing method provided by an embodiment of the present application is shown.

[0042] Figure 5 A flow diagram of an image processing method provided by an embodiment of the present application is shown.

[0043] Figure 6 A start-up interface diagram of an application provided by an embodiment of the present application is shown.

[0044] Figure 7 A business card display interface diagram of an application provided by an embodiment of the present application is shown.

[0045] Figure 8 A flow diagram of image decoding provided by an embodiment of the present application is shown.

[0046] Figure 9 A generation flow diagram of a target loading image provided by an embodiment of the present application is shown.

[0047] Figure 10 A merging diagram of image merging provided by an embodiment of the present application is shown.

[0048] Figure 11 A flow diagram of another image processing method provided by an embodiment of the present application is shown.

[0049] Figure 12 An application diagram of an image processing method provided by an embodiment of the present application is shown.

[0050] Figure 13 A timing diagram of an image processing method provided by an embodiment of the present application is shown.

[0051] Figure 14 A timing diagram of acquiring image binary data provided by an embodiment of the present application is shown.

[0052] Figure 15 A flow diagram of merging pixel blocks provided by an embodiment of the present application is shown.

[0053] Figure 16 A color space conversion diagram provided by an embodiment of the present application is shown.

[0054] Figure 17 A module block diagram of an image processing apparatus provided by an embodiment of the present application is shown.

[0055] Figure 18 A module block diagram of an image encoding apparatus provided by an embodiment of the present application is shown.

[0056] Figure 19 A module block diagram of a computer device provided by an embodiment of the present application is shown.

[0057] Figure 20is a module block diagram of a computer readable storage medium provided by an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order for those skilled in the art to better understand the scheme of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. The embodiments described by reference to the drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0059] In some processes described in the specification, claims and the above drawings, a plurality of steps appearing in a specific order are included, but it should be clearly understood that these steps can be executed or executed in parallel without the order appearing in the text, and the step number is only used to distinguish different steps, and the number itself does not represent any execution order. In addition, the descriptions such as "first" and "second" or "target" in this paper are used to distinguish similar objects, and do not necessarily describe a specific order, sequence or quantity.

[0060] It is worth noting that in the specific embodiments of the present application, data related to the original loaded image is involved, and when the above embodiments of the present application are applied to specific products or technologies, the permission or consent of the subject is required, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards. For example, when the embodiments of the present application need to obtain the original loaded image, a pop-up window or a jump to a confirmation page can be used to obtain individual permission or individual consent for the original loaded image, and after obtaining individual permission or individual consent, the necessary related data for the normal operation of the embodiments of the present application is obtained.

[0061] Before further detailing the embodiments of the present application, the terms and terms involved in the embodiments of the present application are explained, and the terms and terms involved in the embodiments of the present application are applicable to the following explanations:

[0062] Image Data Compression is a technique to represent the original pixel matrix with less bits, either lossy or lossless, also called image coding. The reason why image data can be compressed is that there are redundancies in the data, mainly spatial redundancies caused by the correlation between adjacent pixels in the image; temporal redundancies caused by the correlation between different frames in the image sequence; and spectral redundancies caused by the correlation between different color planes or spectral bands. The purpose of compression is to reduce the number of bits needed to represent the data by removing these redundancies. Because of the huge amount of image data, compression is very important in storage, transmission and processing.

[0063] Image Data Compression includes lossy and lossless data compression. Lossless compression is preferred for technical drawings, charts or cartoons, because lossy compression methods, especially at low bit rates, will introduce compression distortion. Compression of valuable content such as medical images or scanned images for archiving also prefers lossless compression methods. Lossy methods are very suitable for natural images, where a small loss of image quality is acceptable, so that the bit rate can be greatly reduced.

[0064] Discrete Cosine Transform (DCT) is a transform related to Fourier Transform. Discrete Fourier Transform requires complex number operation, although Fourier Transform can improve the operation speed, it is very inconvenient in image coding, especially in real-time processing. According to the properties of Discrete Fourier Transform, the Fourier Transform of real even function only contains real cosine terms, so a real field Discrete Cosine Transform is constructed. The basis vectors of the transform matrix of Discrete Cosine Transform are very close to the eigenvectors of Toeplitz matrix, which reflects the correlation characteristics of human language and image signals. Therefore, in the orthogonal transform of the determined transform matrix of speech and image signals, Discrete Cosine Transform is considered as a quasi-optimal transform.

[0065] The process of lossy compression of the matrix after DCT is Quantization. After DCT processing, the high frequency information of the image will concentrate in the lower right corner of the matrix, and the low frequency information will concentrate in the upper left corner of the matrix. That is, the higher the frequency of the trigonometric function, the more to the right and lower corner of the matrix, and vice versa. The human eye is not so sensitive to high frequency information, and is much more sensitive to low frequency information. Based on this characteristic, the high frequency information in the matrix after DCT can be trimmed through quantization, so as to achieve the purpose of (lossy) compression.

[0066] In the process of software design and development, the use of images is indispensable. However, with the wide use of different devices and their screen sizes, how to achieve adaptive loading of images becomes a problem that cannot be ignored. In actual scenarios, developers first need to understand the screen sizes and resolutions of different devices. In this way, an adaptive image loading scheme can be provided for different devices. For example, the to-be-loaded images matching the screen resolutions of each terminal are pre-stored on different manufacturers' or versions of terminals, or multiple to-be-loaded images suitable for different terminal screen resolutions are pre-stored on the server.

[0067] Specifically, the developer can provide multiple versions of different sizes and resolutions for each image, and let the application select a suitable image from the server according to the display requirements of the device itself to load. The images required by the application to be loaded can also be directly pre-stored on the device. In this way, the best display effect and loading speed can be achieved on different devices. For example, responsive images, by using Hyper Text Markup Language (HTML) tags, can automatically adjust the size of the image according to the size of the display area. <picture>Elements and <source> Elements, developers can provide multiple versions of different sizes and resolutions for each image, and let the browser select the appropriate image to load according to the characteristics of the device. In the front-end web page, CSS media queries are used, that is, different CSS styles are provided for different devices, or different images are dynamically loaded using JS. When loading images in terminal such as native Apple mobile operating system (iPhoneOS, IOS) / Android operating system, the device type is added to the request parameter of the server, and the server returns the pre-stored image.

[0068] However, the above related technologies all need to pre-set multiple images on the local or server, and the terminal loads them on demand. If the terminal manufacturer produces a new device with a new screen resolution size, the application of the terminal needs to add corresponding pre-stored images. With the iteration of device types and application functions, the page style will gradually increase, and more pre-stored images are needed, so this will cause the server or terminal to occupy more storage space to store images of different resolutions.

[0069] In order to solve the above problems, the inventors have conducted research and proposed the image processing method and related method provided in the embodiments of the present application. The system architecture of the above method and related application scenarios involved in the present application will be described below.

[0070] Please refer to Figure 1 , Figure 1 A system architecture diagram is shown. As Figure 1 shown, the above method provided in the embodiments of the present application can be applied in system 100.

[0071] As Figure 1 shown, the image processing method and related method provided in the embodiments of the present application can be applied in computer system 100, which can be a Client-Server (C / S) system architecture, including a client side and a service side. The service side can include a business server 110, a resource server 130 and a storage service 150, and the client side can include a first terminal 120 and a second terminal 140, each of which is installed with a client capable of image processing, that is, an application such as a game application, a social application, etc.

[0072] The service server 110 is configured to receive a request from a terminal and provide a corresponding result to the terminal. The service server 110 can be in network communication with the first terminal 120 and the second terminal 140, receive an image resource request sent by the first terminal 120 and the second terminal 140, and forward the image resource request to the resource server 130. The resource server 130 can provide a lossless original loading image to different terminals in response to an image resource request of the terminal. The storage service 150 can be configured to temporarily store image binary data (Image Data) of the original loading image.

[0073] The server described above can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (CDN), blockchain, and big data and artificial intelligence platform. The terminal can be a smartphone, a notebook computer, a tablet computer, a desktop computer, etc.

[0074] The storage service 150 can be a cloud object storage (COS). The cloud object storage can adopt a distributed storage technology architecture to store data on multiple storage nodes to realize concurrent reading and writing of data and load balancing, and improve the access speed and overall performance of the data. For example, the image binary data of the original loading image is distributedly stored through hash mapping.

[0075] It should be noted that, Figure 1 The system architecture described in the embodiments of the present application is for more clearly illustrating the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application. For example, the first terminal 120 can generally refer to one of a plurality of terminals, and the second terminal 140 can generally refer to one of a plurality of terminals. The embodiments of the present application only take the first terminal 120 and the second terminal 140 as examples for illustration.

[0076] Please refer to Figure 2 , Figure 2 A system deployment schematic diagram is shown. For example, the image processing method and related method provided by the embodiments of the present application can also be deployed in a system as shown in Figure 2 The system includes a terminal 240, an Internet 230, a gateway 220, a server 210, etc.

[0077] The terminal 240 can include a desktop computer, a laptop computer, a personal digital assistant (PDA), a mobile phone, a car terminal, a home theater terminal, a dedicated terminal, a smart watch, and the like. In addition, it can be a single device or a collection of multiple devices. The terminal 240 can communicate with the Internet 230 in a wired or wireless manner and exchange data. For example, the terminal 240 can communicate with the Internet 230 through a wireless router 250.

[0078] The server 210 refers to a computer system capable of providing certain services to the terminal 240. Compared with the ordinary terminal 240, the server 210 has higher requirements in stability, security, performance, and the like. The server 210 can be a high-performance computer in a network platform, a cluster of multiple high-performance computers, a part of a high-performance computer (for example, a virtual machine), a combination of parts of multiple high-performance computers (for example, virtual machines), and the like.

[0079] The gateway 220 is also called an inter-network connector or a protocol converter. The gateway realizes network interconnection at the transport layer and is a computer system or device that acts as a conversion role. In the case of two systems using different communication protocols, data formats or languages, or even having completely different architectures, the gateway is a translator. At the same time, the gateway can also provide filtering and security functions. The messages sent by the terminal 240 to the server 210 can be transmitted to the corresponding server 210 through the gateway 220. The messages sent by the server 210 to the terminal 240 can also be transmitted to the corresponding terminal 240 through the gateway 220.

[0080] The image processing method and related method of the embodiment can be partially implemented on the server 210 and partially implemented on the terminal 240.

[0081] As an implementation manner, the terminal 240 can send an image resource request to the server 210. When the server 210 receives the image resource request, the corresponding original loading image can be determined according to the image resource request, and the original loading image is image encoded to obtain corresponding image binary data. Further, the image binary data is returned to the terminal 240 after data compression. When the terminal 240 receives the compressed image binary data, the compressed image binary data is data decompressed to obtain image binary data. Further, the terminal 240 performs decoding, inverse quantization, inverse discrete cosine transform, and pixel merging on the image binary data, and the like to obtain a target loading image adapted to the screen resolution of the terminal 240, and displays the target loading image on the screen.

[0082] It should be noted that, Figure 2 This is only a system deployment diagram provided by the embodiments of the present application. The system deployment scheme described in the embodiments of the present application is only for more clearly illustrating the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application. For example, the terminal 240 can generally refer to one of a plurality of terminals, and the embodiments of the present application only take three terminals shown in FIG. 1 as an example. Those skilled in the art can know that, as the system deployment scheme evolves, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. Figure 2

[0083] The embodiments of the present application can be applied in various scenarios, such as Figure 3 image loading scenarios of different version devices as shown in FIG. 2, Figure 4 image loading scenarios of different types of devices as shown in FIG. 3, and the like.

[0084] (1) Image loading scenario of different version devices

[0085] The update iteration of smart phones is constantly evolving and changing. Every year, major mobile phone manufacturers will launch new versions of smart phones, including improvements in hardware such as processors, battery capacity, and display screens. These hardware improvements can improve the performance and user experience of the phone, thereby better meeting the needs of users. However, the resolution of the display screen used by different versions of the phone will also differ. Therefore, in actual scenarios, developers need to understand the screen resolution of different versions of the phone. In this way, an image loading scheme with strong adaptability can be provided for different versions of the phone.

[0086] For example, the screen resolutions of three mobile phone models under a certain mobile phone brand are 1080x1920, 1125x2436, and 1242x2688. Assume that the three mobile phones are all installed with a music playing application A, and each mobile phone will display a loading page when starting the music playing application A. In order to ensure that the loading page displayed by the three mobile phones can be adapted to the resolution of their respective screens, the relevant technology will pre-store the corresponding loading page for each mobile phone on the server. The contents of the three loading pages are the same, and only the resolutions of the pages are different. This image loading method is prone to cause excessive consumption of storage resources. The image processing method and the image loading method provided by the present application can effectively reduce the problem of consumption of storage resources.

[0087] Exemplarily, an application service provider provides an application system, which can include an application server and a client. Among them, the application server is responsible for data processing and management, and the client is responsible for completing the interaction task with the user. For example, Figure 3 ​As shown, application server 310 may include business server 311, resource server 312, and storage server 313. Users can access the services provided by the application system by installing a client. Terminals 321, 323, and 325 all have the client installed. For example, terminal 325 has client 301 installed. Terminals 321, 323, and 325 are three different versions of mobile phones from a certain brand, with screen resolutions of 1080×1920, 1125×2436, and 1242×2688, respectively.

[0088] Service server 311 is used to receive service requests from different terminals and send service requests of the corresponding service type to the corresponding resource server. For example, taking terminal 325 as an example, when client 301 needs to display the target loading image, it can send an image resource request to service server 311. Service server 311 then sends this image resource request to resource server 312 for processing image loading services. When resource server 312 receives this image resource request, it can find the lossless original loading image corresponding to the image resource request, for example, the original loading image with a resolution of 1956×2473, and perform image encoding on this original loading image to obtain the corresponding image binary data.

[0089] Furthermore, resource server 312 temporarily stores the image binary data on storage server 325 and returns the storage address of the image binary data on storage server 325 to terminal 325. Upon receiving the storage address, client 301 of terminal 325 can retrieve the image binary data from storage server 325 based on the storage address. It then decodes, dequantizes, and inversely discrete cosine transforms the image binary data to obtain multiple pixel blocks. Since the resolution of the pixel blocks is adjusted according to the resolution of the terminal screen during the dequantization process, the adjusted pixel blocks can be merged to obtain a target loaded image adapted to the resolution of terminal 325's screen. The resolution of the target loaded image is 1242×2688. Through the same process, terminals 321 and 323 can also obtain target loaded images adapted to their respective screen resolutions.

[0090] (2) Image loading scenarios for different types of devices

[0091] like Figure 4 As shown, the application system is responsible for data processing and management in the background. The application server 310 can be a cloud server cluster, including multiple servers with various functions. The client in the application system is responsible for completing the task of interacting with the user and can be installed on different types of terminals, such as a tablet computer 322, a smartphone 324, and a smartwatch 326.

[0092] For example, when the tablet 322, the smart phone 324 and the smart watch 326 need to display the start interface of the client, the three can send an image resource request to the application server 310, and then the application server 310 can send the image binary data of the lossless original image of the start interface to the three respectively when receiving the image resource request sent by the three. Further, the tablet 322, the smart phone 324 and the smart watch 326 can perform image adaptive loading display on the image binary data according to the resolution of the screen of the self.

[0093] Specifically, taking the smart phone 324 installed with the client 301 as an example, the client 301 can decode the image binary data to obtain a decoding matrix, further adjust the quantization matrix according to the screen resolution of the client 301, and perform inverse quantization on the decoding matrix based on the adjusted quantization matrix to obtain a DCT coefficient matrix, further perform inverse discrete cosine coding on the DCT coefficient matrix to obtain a plurality of pixel blocks under different color components, then merge the plurality of pixel blocks under different color components to obtain a complete image, and render the complete image to obtain a target loading image. The same processing process can also be used by the tablet 322 and the smart watch 326 to obtain a target loading image adapted to the resolution of the screen of the self.

[0094] It should be noted that, Figure 3 and Figure 4 are only two application scenario diagrams provided by the embodiments of the present application. The application scenarios described in the embodiments of the present application are only for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. For example, when the server sends the image binary data to the terminal, the image binary data can be compressed before being sent. For another example, the image processing can be an image element in the display interface of the terminal. Those skilled in the art can know that, as the application scenarios evolve, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0095] According to an embodiment of the present application, an image processing method is provided. The image processing method can be executed by a computer device (the computer device can be a server, an edge computing device or other terminal device with certain processing capability), which has at least the functions of storage, calculation and communication. Figure 5 A flowchart of the image processing method provided by the embodiments of the present application is shown. As shown in Figure 5 the image processing method can specifically include:

[0096] Step 110: Obtain image binary data obtained by image coding of an original loading image.

[0097] In actual application scenarios, the same application will be downloaded and installed on terminals of different manufacturers or different versions. In order to ensure that the same image can be accurately loaded and displayed on different terminals, the related technology either pre-stores the image corresponding to the image in the appropriate terminal in each terminal locally, or pre-stores the image corresponding to the image in the appropriate terminal for each type or version of terminal on the server. Considering that the number of images required to be loaded by the application program will gradually increase as the function and interface style of the application program are iterated, the pre-stored loading image method of the related technology will cause the server or the local terminal to require more storage space to store images of different resolutions, and the storage resource consumption is large.

[0098] Therefore, the application proposes an image adaptive loading method based on an original loading image for different terminals. The terminal can automatically generate a target loading image adapted to the screen resolution of the terminal itself according to the original loading image corresponding to the image to be loaded when the image needs to be loaded. Thus, the problem of large storage resource consumption caused by pre-storing a large number of images is avoided, and the effective utilization rate of storage resources is improved. The image format of the original loading image is a lossless image format, which means that the image format can completely preserve the original image information. Therefore, the resolution, bit depth and other image parameter values of the original loading image can be the maximum value.

[0099] In the embodiments of the application, the original loading image refers to the original image of the image to be loaded by the terminal (i.e., the image to be loaded), which can include an image of the entire interface displayed by a computer program running on the terminal capable of interacting with the user or having a visual interface or an image corresponding to various interface elements (Interface Element) in the interface. As shown in Figure 6 As shown in Figure 7 As shown in

[0100] The image binary data refers to binary data obtained by encoding an image by an encoding algorithm. The encoding algorithm can be a Base64 encoding algorithm. The image binary data can include resolution, bit depth, encoding parameters, and other data of the image. For example, the image binary data of a JPEG (Joint Photographic Experts Group) format image is organized in the form of a segment data stream, and a series of markers are defined at the beginning of each segment. Each marker starts with 0xFF, followed by a 1-byte marker identifier and a 2-byte marker length and a payload corresponding to the marker. The marker length part has high bits in front and low bits in back, and does not include the first two bytes of the marker. Common markers include SOI (Start of Image) to specify the beginning of the file, DHT (Define Huffman Table) to specify a Huffman table, and DQT (Define Quantization Table) to specify a quantization table.

[0101] In some embodiments, when an application needs to load an image, it can first determine whether the target loaded image is cached locally. If the target loaded image is cached, it means that the loading task for the target loaded image has been performed previously, and thus the cached target loaded image can be directly loaded and displayed. If the target loaded image is not cached, the terminal can obtain image binary data of the target loaded image from the server to generate the target loaded image by image decoding.

[0102] As an implementation, the terminal can send a request for obtaining image binary data to the server, and then the terminal can receive image binary data returned by the server according to the request. Optionally, the image binary data received by the terminal can be compressed data obtained by image compression, so as to improve the efficiency of data transmission. Therefore, the terminal can decompress the received compressed data to obtain the image binary data. For example, a user clicks to start a music APP on a mobile phone, and then the music APP can start a thread for loading a "start interface", and try to read the start interface from the content of the mobile phone. If the start interface is not read, the music APP sends a request to the server of the music APP service provider to obtain image binary data of the original loaded image corresponding to the start interface, and then receives the image binary data returned by the server.

[0103] Step 120: determining a target image format based on the image binary data, and performing image decoding on the image binary data by the target image format to obtain a plurality of decoding matrices.

[0104] According to the principle of image coding, the image can be obtained after image coding, and the image binary data can be used to reconstruct the image after image decoding. Image decoding is the inverse process of image coding. Therefore, the application can generate a target loaded image based on the decoding matrix obtained by decoding the image binary data. Wherein, the target image format refers to the image format of the original loaded image.

[0105] Since the encoding algorithm adapted to the image format of the original loaded image is required when the original loaded image is encoded, the decoding algorithm adapted to the original loaded image is also required when the image binary data is decoded. For example, the encoding algorithm based on Huffman table is used when the JPEG format image is encoded, and correspondingly, the decoding algorithm based on Huffman table is also required when it is decoded.

[0106] In some embodiments, the image format can be determined by parsing the bytes in the image binary data, and then the decoding algorithm determined according to the image format can be used to decode the image binary data to obtain the decoding matrix under different color components. Specifically, step 120 can include:

[0107] (1) data parsing of the image binary data to obtain the first byte in the image binary data;

[0108] (2) byte reading according to the first byte to determine the target image format corresponding to the image binary data;

[0109] (3) determining the corresponding target decoding algorithm based on the target image format;

[0110] (4) image decoding of the image binary data by the target decoding algorithm to obtain a plurality of luminance decoding matrices, a plurality of blue decoding matrices and a plurality of red decoding matrices, and the plurality of luminance decoding matrices, the plurality of blue decoding matrices and the plurality of red decoding matrices are determined as the plurality of decoding matrices.

[0111] Wherein, the first byte refers to the byte at the first position in the header file of the image binary data, which usually represents the image format of the image in hexadecimal form, for example, the first byte 0xFF is the JPEG image format, the first byte 0x89 is the PNG image format, the first byte 0x47 is the GIF image format, the first byte 0x49 is the TIFF image format, etc. In addition, the computer needs to use a data model, i.e. color space, for image processing, such as RGB color space and YUV color space. Similarly, image coding also needs to be coded in different color components of the color space.

[0112] For human eyes, the light and dark changes in the image are more easily perceived due to the structure of human eyes. There are two kinds of light-sensitive cells on the retina of human eyes: rod cells capable of perceiving brightness changes, and cone cells capable of perceiving colors. Since the number of rod cells is much larger than that of cone cells, human eyes are more likely to perceive light and dark details. The first thing that image coding needs to do is to "separate important information from unimportant information", try to retain important information and delete unimportant information. In the YUV color space, "Y" represents luminance (Luminance or Luma), that is, the gray scale value, and "U" and "V" represent chrominance (Chrominance or Chroma) that can describe image color and saturation, used to specify the color of a pixel.

[0113] Therefore, based on the characteristics of the YUV color space, the image is converted from the RGB color space to the YUV color space during image coding, so as to accurately separate the luminance information and color difference information in the image. Further, during image coding and decoding, the image can be coded and decoded under different color components of the YUV color space. The YUV color space includes a luminance component, a blue component and a red component. The luminance component refers to the gray scale in the YUV color space, the blue component refers to the blue chrominance (Blue Chrominance, C b ) offset in the YUV color space representing color, and the red component refers to the red chrominance (Red Chrominance, C r ) offset in the YUV color space representing color. Correspondingly, image decoding of the image binary data can obtain a plurality of luminance decoding matrices under the luminance component, a plurality of blue decoding matrices under the blue component, and a plurality of red decoding matrices under the red component, which are determined as a plurality of decoding matrices.

[0114] As an implementation, when the image binary data is obtained, the image binary data can be parsed to obtain a first byte represented by hexadecimal coding in the image binary data, and the first byte can be compared with the coding of different image formats by enumeration, so as to determine the target image format. Further, the decoding interface (Application Programming Interface, API) corresponding to the target image format can be found, and the decoding interface can be called to perform image decoding on the image binary data to obtain a plurality of luminance decoding matrices, a plurality of blue decoding matrices and a plurality of red decoding matrices.

[0115] Please refer to Figure 8 , Figure 8 A flowchart of image decoding is shown. When a terminal acquires image binary data, it can determine the image format according to the first byte in the parsed image binary data, and further determine whether a decoding algorithm corresponding to the image format can be acquired. If the decoding algorithm can be acquired, the terminal decodes the image binary data based on the acquired decoding algorithm. If the decoding algorithm cannot be acquired, the terminal can request the server to decode the image binary data. Further, the terminal can acquire a quantization factor to dequantize the decoded matrix, and then perform inverse discrete cosine transform to obtain an image pixel block.

[0116] Step 130: Dequantize each decoded matrix according to the target quantization factor determined according to the screen resolution to obtain a discrete cosine transform coefficient matrix corresponding to each decoded matrix.

[0117] The present application proposes to dynamically perform image adaptive loading for different terminals using an original loaded image. It is considered that in the quantization process of image encoding, the value of the quantization factor (i.e., the quantization matrix) will affect the quality and size of the encoded data. Similarly, when dequantizing the decoded matrix, the value of the quantization factor will also affect the element values in the discrete cosine transform coefficient matrix obtained after dequantization, and further affect the quality and size of the target loaded image. Therefore, according to this principle, different terminals can adjust the quantization factor according to their screen resolution, and then perform personalized dequantization based on the adjusted quantization factor, to finally dynamically perform image adaptive loading.

[0118] The quantization is a lossy compression process of the discrete cosine transform coefficient matrix in image encoding. After the discrete cosine transform in image encoding, the high-frequency information in the image will be concentrated in the lower right corner of the discrete cosine transform coefficient matrix, and the low-frequency information will be concentrated in the upper left corner of the discrete cosine transform coefficient matrix. The so-called high-frequency information can be understood as very subtle changes in a picture, such as color mutations, for example, the boundaries of objects or small patterns in the picture, while the low-frequency information is the part of the picture that does not change much, which can be understood as color blocks. The human eye is not very sensitive to high-frequency information, but is much more sensitive to low-frequency information. Based on this characteristic, the high-frequency information in the discrete cosine transform coefficient matrix can be cropped, so as to achieve the purpose of (lossy) compression. This cropping operation is called quantization. Dequantization is the inverse process of quantization.

[0119] In some embodiments, the DCT coefficient matrix can also include a luminance coefficient matrix for the luminance component, a blue coefficient matrix for the blue component, and a red coefficient matrix for the red component according to different color components of the color space. The terminal can calculate the target quantization factor through its own screen resolution to perform the inverse quantization operation on the decoding matrix of different color components to obtain the DCT coefficient matrix corresponding to each decoding matrix. Specifically, step 130 can further include:

[0120] (1) performing data analysis on the image binary data to obtain a first original quantization factor and a second original quantization factor;

[0121] As an implementation, the terminal can obtain two quantization tables (Quantization Table) by performing data analysis on the image binary data, including a luminance quantization table, i.e., the first original quantization factor, and a chrominance quantization table, i.e., the second original quantization factor. The first original quantization factor is used to perform inverse quantization on the luminance decoding matrix of the luminance component, and the second original quantization factor is used to perform inverse quantization on the blue decoding matrix and the red decoding matrix of the chrominance component (blue component and red component).

[0122] (2) adjusting the first original quantization factor and the second original quantization factor based on the screen resolution to obtain corresponding first and second quantization factors;

[0123] As an implementation, the original quantization factor can be adjusted in combination with the scaling factor determined based on the screen resolution of the terminal and the original resolution of the original loaded image. Step (2) can further include:

[0124] (2.1) obtaining a scaling factor based on the screen resolution and the original resolution of the original loaded image;

[0125] (2.2) determining the reciprocal of the scaling factor as a basic quantization factor;

[0126] (2.3) adjusting the first original quantization factor according to the basic quantization factor to obtain the first quantization factor;

[0127] (2.4) adjusting the second original quantization factor through the basic quantization factor to obtain the second quantization factor.

[0128] Since the quantization factor is usually fixed and has no direct correlation with the screen resolution, this application proposes to set a scaling factor and use the scaling factor as an intermediate medium between the screen resolution and the quantization factor, that is, to adjust the quantization factor with the help of the scaling factor determined based on the screen resolution. In actual application scenarios, the process of image adaptation by the terminal is the process of scaling the lossless original loaded image. The scaling process must reduce the original loaded image with the highest resolution to a size that can be fully adapted and displayed on the terminal screen, while maintaining the aspect ratio of the original loaded image. Therefore, the scaling factor can take the minimum value between the width ratio and height ratio corresponding to the screen size and the image size.

[0129] Specifically, the screen width and screen height can be obtained based on the screen resolution. Similarly, the image width and image height of the original loaded image can be obtained based on the original resolution. The calculation formula is as follows:

[0130] F scale =min (W screen / W image ,H screen / H image ) Formula (1)

[0131] Among them, F scale Indicates the scaling factor (scale_factor), W screen and H screen Respectively represent the screen width and screen height of the screen, W image and H image Represent the image width and image height of the original loaded image, respectively. Taking the minimum value ensures that the scaled image can be fully displayed on the terminal screen with the aspect ratio of the original loaded image. Furthermore, based on the reciprocal adjustment of the scaling factor, the first original quantization factor and the second original quantization factor are adjusted respectively to obtain the corresponding first quantization factor and second quantization factor. The calculation formula is as follows:

[0132]

[0133] Among them, F base_q Represents the basic quantization factor (base_quantization_factor), represents the first quantization factor (new1_quantization_factor), represents the second quantization factor (new2_quantization_factor), Represents the first original quantization factor (preset1_quantization_factor), The first original quantization factor (preset2_quantization_factor) is represented.

[0134] Generally, the less the quality loss of the quantized image, the smaller the quantization factor. According to the calculation formula of the scaling factor, the larger the scaling factor means the terminal screen size is closer to the original loaded image size, so the smaller the degree of image scaling, and the less the image quality loss, and correspondingly, the smaller the quantization factor. Conversely, when the scaling factor is small, the larger the degree of image scaling, and a larger quantization factor is needed for quantization. Therefore, the quantization factor and the scaling factor are inversely proportional, and thus the reciprocal of the scaling factor can be used to adjust the first original quantization factor and the second quantization factor.

[0135] (3) Perform inverse quantization operation on the plurality of luminance decoding matrices based on the first quantization factor to obtain a plurality of corresponding luminance coefficient matrices.

[0136] Because the quantization process is to divide the discrete cosine transform coefficient matrix to be quantized by the quantization factor, in the inverse quantization, the decoding matrix can be multiplied by the quantization factor to obtain the discrete cosine transform coefficient matrix. Specifically, the terminal can multiply the first quantization factor with each luminance decoding matrix to obtain the corresponding luminance coefficient matrix.

[0137] (4) Perform inverse quantization operation on the plurality of blue decoding matrices according to the second quantization factor to obtain a plurality of corresponding blue coefficient matrices, and perform inverse quantization operation on the plurality of red decoding matrices according to the second quantization factor to obtain a plurality of corresponding red coefficient matrices.

[0138] Specifically, since the blue decoding matrix and the red decoding matrix both belong to the decoding matrix under the chroma component, the terminal can multiply the second quantization factor with each blue decoding matrix to obtain the corresponding blue coefficient matrix. And multiply the second quantization factor with each red decoding matrix to obtain the corresponding red coefficient matrix.

[0139] The quantization factor is adjusted by a scaling factor determined by the screen resolution of each terminal, and the adjusted quantization factor can be used for personalized inverse quantization. Since the scaling factor is calculated based on the screen size of the device and the original image size, the adjusted quantization factor based on the scaling factor can scale the pixel information of the image during inverse quantization, and thus the resolution of the finally generated target loading image is adapted to the resolution of the terminal screen. It should be noted that the "adaptation" mentioned in the present application can include the effect of filling the screen after the target loading image is scaled according to the screen resolution, for example, the display of the entire application startup interface. It can also include the effect of displaying a certain area on the screen after the target loading image is scaled according to the screen resolution, for example, the display of a certain image element in the application interface.

[0140] Step 140: performing inverse discrete cosine transform on each discrete cosine transform coefficient matrix to obtain an image pixel block corresponding to each discrete cosine transform coefficient matrix.

[0141] Discrete cosine transform is used to convert fixed-length time domain signal data into frequency domain signal to obtain a high-energy frequency domain range to remove redundancy. In the image encoding process, the matrix of each color component corresponding to the pixel block can be obtained through color space conversion, and the spatial values of the matrix can be converted into frequency through discrete cosine transform, thereby concentrating the pixel point energy and removing the correlation and reducing the redundancy.

[0142] wherein the pixel point energy refers to the energy value of each pixel point in the image, and in digital image processing, the pixel point energy represents the intensity of image information such as brightness, saturation or color of the point, and is used to describe the light and dark degree or color difference of different regions in the image. The inverse discrete cosine transform is the inverse process of the discrete cosine transform, which realizes the conversion of image information from frequency to space. According to different color components of the color space, the image pixel block includes a luminance pixel block, a blue pixel block and a red pixel block.

[0143] As an implementation manner, the terminal can perform inverse discrete cosine transform on a plurality of luminance coefficient matrices to obtain a luminance pixel block corresponding to each luminance coefficient matrix, perform inverse discrete cosine transform on a plurality of blue coefficient matrices to obtain a blue pixel block corresponding to each blue coefficient matrix, and perform inverse discrete cosine transform on a plurality of red coefficient matrices to obtain a red pixel block corresponding to each red coefficient matrix.

[0144] Step 150: merging images based on the color components corresponding to each image pixel block to obtain a merged image under each color component, and generating a target loading image corresponding to the screen resolution according to the merged image under each color component.

[0145] In the image encoding process, the image is divided into multiple pixel blocks, i.e. the minimum coded unit (Minimum Coded Unit), and the pixels of each pixel block are 8x8. For this purpose, after obtaining the pixel blocks under different color components through inverse discrete cosine transformation, the pixel blocks under each color component can be merged to obtain a merged image under each color component, and then a target loading image is generated based on the merged image.

[0146] Please refer to Figure 9 , Figure 9 A flowchart of generating a target loading image is shown. Figure 9 The image binary data in the image binary data is decoded to obtain a decoded matrix under different color components, and then the decoded matrix under different color components is inverse quantized to obtain a corresponding discrete cosine transform coefficient matrix, and each discrete cosine transform coefficient matrix is inverse discrete cosine transformed to obtain an image pixel block under different color components. Then, the image pixel blocks under each color component are respectively merged to obtain a merged image under each color component, including a luminance merged image, a blue merged image and a red merged image. Further, the merged image under each color component is respectively converted in color space to obtain a channel image in RGB color space, and the channel image is subjected to channel merging and optimization operation to finally obtain a target loading image.

[0147] Considering that the generation of merged images based on image pixel blocks of different color components and the execution of image rendering and other optimization operations can be performed in parallel, in order to ensure that the merging, rendering, optimization and other operations for each image do not affect each other, an independent storage space can be allocated for each merged image to form data isolation between multiple merged images. Specifically, the terminal can obtain the image resolution and bit depth of the original loading image from the image binary data; based on the image resolution and bit depth, a preset memory space is allocated, and then the merged image under each color component is stored based on the preset memory space.

[0148] The bit depth (Bit Depth) is also called color depth (Color Depth) and is used to measure how much color information is available for displaying or printing pixels in an image, and its unit is "bit". Common color depths include 1 bit, 8 bits, 24 bits and 32 bits, etc. 1 bit has two possible values: 0 or 1. A larger color depth (more bits of information per pixel) means that the original loading image has more available colors and more accurate color representation. For example, an original loading image with a bit depth of 8 bits, a pixel in the image can be any of 2 8 24 ​= any one of 16777216 colors.

[0149] In some embodiments, the operation of merging the image pixel blocks can be performed in parallel on multiple threads or processors, thereby improving the efficiency of image adaptation. The terminal can merge the image into multiple regions, and merge the pixel blocks in these regions on different threads respectively. Specifically:

[0150] (1) Obtain a plurality of concurrent threads.

[0151] (2) Based on the number of concurrent threads and the image resolution, determine the merging regions under different color components.

[0152] Wherein, the concurrent thread is the idle thread in the current thread pool that can be executed concurrently. For example, the terminal uses the getPoolSize() method and the getActiveCount() method in the ThreadPoolExecutor class to calculate the number of idle threads. The merging region refers to the different regions divided by the merging image.

[0153] (3) Merge the image pixel blocks in the merging region by each concurrent thread to obtain the merging image under each color component.

[0154] Wherein, the color component can include a luminance component, a blue component and a red component, and correspondingly, under each color component, the merging image can include a luminance merging image, a blue merging image and a red merging image.

[0155] As an implementation, based on each concurrent thread, the image pixel blocks in the merging region are merged under the luminance component to obtain the luminance merging image. According to each concurrent thread, the image pixel blocks in the merging region are merged under the blue component to obtain the blue merging image. Through each concurrent thread, the image pixel blocks in the merging region are merged under the red component to obtain the red merging image.

[0156] Please refer to Figure 10 , Figure 10 The merging of three images is shown. If the image A to be merged can be determined as a rectangle according to the image resolution, the image A to be merged can be taken as a merging area, and the merging of the image can be performed from the top left corner of the merging area by using one thread. Specifically, the merging of the image is to traverse each pixel block and merge the pixel blocks in a scanning order, for example, the pixel blocks are merged in a zigzag scanning order in the figure, until all the pixel blocks are traversed, and a merged image is obtained. For the image B to be merged which is divided into two merging areas, two threads can be used to merge the pixel blocks of the two merging areas in parallel, one thread starts from the top left corner, and the other thread starts from the bottom right corner. For the image C to be merged which is divided into five merging areas, five threads can be used to merge the pixel blocks of the five merging areas in parallel, each thread starts from the outer side of the merging area. It should be noted that, Figure 10 The merging of the pixel blocks in only one color component is exemplified. In actual scenarios, the pixel blocks in different color components need to be merged respectively.

[0157] In some embodiments, in order to improve the resolution of the target loaded image to be accurately adapted to the resolution of the terminal screen, the terminal can also perform pixel scaling on each image pixel block based on the scaling factor, to obtain a scaled pixel block corresponding to each discrete cosine transform coefficient matrix, and then perform image merging based on the color components corresponding to each scaled pixel block, to obtain a merged image in each color component, and generate a target loaded image corresponding to the screen resolution according to the merged image in each color component. In this way, by performing secondary scaling on the image pixel blocks, the resolution of the target loaded image is more adapted to the resolution of the terminal screen.

[0158] In the image encoding process, in order to remove the image information which is not sensitive to the human eye in the image, the image is converted from the RGB color space to the YUV color space. Therefore, when the merged image in the YUV color space is converted to an image in the RGB color space, i.e., a channel image, during the generation of the target loaded image, the image in the RGB color space includes a red channel, a green channel and a blue channel, and the image corresponding to each channel can be displayed on the terminal only after being merged.

[0159] As an implementation, the luminance merged image, the blue merged image and the red merged image are respectively converted in color space to obtain a red channel image, a green channel image and a blue channel image, and the red channel image, the green channel image and the blue channel image are merged to obtain an intermediate image. Further, the intermediate image can be optimized to obtain a target loaded image corresponding to the screen resolution. The optimization operation at least includes one of image scaling or image rendering. Then, the terminal can display the target loaded image to be loaded on the screen.

[0160] The embodiment can obtain image binary data of an original loaded image obtained by image encoding, wherein the image format of the original loaded image is a lossless image format, and determine a target image format based on the image binary data, and perform image decoding on the image binary data through the target image format to obtain a plurality of decoding matrices, and then perform inverse quantization operation on each decoding matrix respectively according to a target quantization factor determined based on the screen resolution to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix. Further, inverse discrete cosine transform is performed on each discrete cosine transform coefficient matrix respectively to obtain an image pixel block corresponding to each discrete cosine transform coefficient matrix. Then, image merging is performed based on the color components corresponding to each image pixel block to obtain a merged image under each color component, and a target loaded image corresponding to the screen resolution is generated according to the merged image under each color component.

[0161] In this way, when the terminal generates a target loaded image based on the image binary data of the original loaded image returned by the server, the target quantization factor determined based on the screen resolution can guide the scaling of the pixels of the original loaded image in the inverse quantization process, so that the scaled image pixels can be adapted to the screen resolution. Based on this, the target loaded image adapted to the screen resolution of the terminal can be generated through inverse discrete cosine transform and image merging, and then the image is adaptively loaded while avoiding the terminal to pre-store the image to be loaded locally, thereby effectively reducing the consumption of storage resources and optimizing the storage space of limited storage resources such as mobile terminals and embedded systems. In addition, the application program of the terminal no longer needs to write additional image adaptive code for terminals with different resolutions. When a terminal with a new screen resolution size is added, the application program code does not need to be modified, thereby reducing the maintenance cost and complexity of the application program. And by dynamically adjusting the resolution and bit depth of the image according to the screen size and resolution of the terminal, the display effect of the image on different terminals can be ensured to be closer to the original image, improving user experience and satisfaction. In addition, by using multi-threading, GPU acceleration and hardware acceleration, image loading can be more efficient on the terminal.

[0162] According to an embodiment of the present application, an image processing method is provided. The image processing method can be executed by a computer device (the computer device can be a server, an edge computing device, or other terminal device with certain processing capability), which at least has the functions of storage, calculation and communication. Figure 11 A flowchart of another image processing method provided by the embodiment is shown. Figure 12 An application diagram of the image processing method provided in the embodiment is shown.

[0163] Exemplarily, the image processing method can be applied in a scenario of a game APP loading interface. Different game players can download and install a game APP on different terminals, for example, install a game APP on a tablet computer and a smart phone. The game APP can perform data interaction with a background server during running, and complete the entire game service. For example, Figure 12 As shown, when the game APP displays a character selection interface, the game character image in the character selection interface needs to be loaded. However, the game character images displayed on the character selection interface of the tablet computer and the smart phone have different resolution requirements. Therefore, the terminal can perform adaptive loading of the image on the tablet computer and the smart phone respectively according to the image processing method and related method provided in the present application. Specifically, the timing diagram of the image processing method shown in Figure 13 will be used to explain the application scenario.

[0164] 201: The terminal queries whether the target loading image is cached locally in response to an image loading instruction.

[0165] The image loading instruction is a computer instruction for instructing the terminal to load an image. For example, when the game APP displays a character selection interface, an image loading instruction for loading a game character image can be generated.

[0166] Exemplarily, as shown in Figure 13 in response to the image loading instruction, the application program can attempt to read the local cache to determine whether the target loading image is cached locally. If the target loading image is cached, it indicates that the target loading image has been loaded before, and then the cached target loading image is directly obtained and rendered and displayed. If the target loading image is not cached, the target loading image can be generated by sending an image resource request to the server.

[0167] 202: When the target loading image is not cached locally, the terminal sends an image resource request to the server.

[0168] Exemplarily, if the terminal does not find the target loading image in the local cache, the terminal can send an image resource request to the server through the network. The server performs data compression on the image binary data obtained by encoding the original loading image, and returns the compressed image compression data, wherein the compressed image compression data at least includes the image binary data.

[0169] 203: The server obtains the original loading image.

[0170] Exemplarily, the original loading image can be stored in an image database of the server. The image format of the original loading image is a lossless image format. When the server receives an image resource request sent by the terminal, the original loading image of the image required to be loaded by the terminal can be found in the image database according to the image resource request. For example, when the server receives an image resource request sent by a game APP, the image identity (Identity document, ID) of the game character image required to be loaded by the game APP can be parsed, and then the original loading image corresponding to the game character image can be found in the image database according to the image identity.

[0171] 204: The server acquires the screen resolution of the terminal in response to the image resource request sent by the terminal.

[0172] Exemplarily, in order to improve the efficiency of image loading, the server can acquire the screen resolution of the terminal from the image resource request in response to the image resource request sent by the terminal, and further determine whether the image binary data of the original loading image of the image required to be loaded has been returned to the terminal corresponding to the screen resolution.

[0173] For example, the server acquires the screen resolution of a smartphone in response to an image resource request of a game character image sent by the smartphone, and determines whether the image binary data of the original loading image of the game character image has been sent to other devices corresponding to the screen resolution. If the image binary data has been sent, the image binary data is directly returned to the smartphone. If the image binary data has not been sent, the original loading image of the game character image is image encoded.

[0174] 205: When it is detected that the image binary data corresponding to the screen resolution has not been returned, the server image encodes the original loading image to obtain corresponding image binary data.

[0175] Exemplarily, the server can select a corresponding encoding algorithm according to the image format of the original loading image, and then image encode the original loading image according to the encoding algorithm. For example, the image format of the original loading image of the game character image to be loaded is PNG, and a DEFLATE encoding algorithm is selected to image encode the original loading image to obtain corresponding image binary data. The DEFLATE encoding algorithm is a lossless data compression algorithm using both LZ77 algorithm and Huffman Coding.

[0176] Considering that the server needs to serve a large number of terminals, the generated image binary data is also in a mass level. In order to improve the high expansibility and security of data storage, the server can store the generated image binary data in a storage object in the cloud. Then the terminal can obtain the image binary data from the storage object. For example, the server can store the generated image binary data in the cloud object storage (COS). When storing the image binary data in the COS, due to the large data size of the image binary data, the image binary data is stored in different storage spaces of the cloud object storage. Therefore, the cloud object storage can perform hash mapping on the storage addresses of the image binary data in the cloud object storage, that is, the storage addresses of the image binary data stored in different storage spaces are stored in the form of a single linked list through a hash table.

[0177] 206: The server determines a target compression strategy according to the network state and the computing performance of the terminal, and performs data compression on the image binary data based on the target compression strategy to obtain image compression data.

[0178] In order to improve the efficiency of network transmission of image binary data, the server can first compress the image binary data before transmitting it to the terminal. The network state refers to the state of the network currently connecting the server and the terminal, for example, traffic direction, network speed, packet loss rate, etc. The computing performance of the terminal can include the computing power of the system, the response time, the central processing unit (CPU), the operation time of the image processor, the memory utilization rate, etc. which can be used to judge the running performance of the terminal system. Here, no limitation is made.

[0179] Illustratively, when the network state and the terminal computing performance are relatively poor, the server can perform lossy compression on the image binary data. When the network state or the terminal computing performance is relatively good, the server can perform lossless compression on the image binary data. The good or bad of the network state and the terminal computing performance can be judged by the developer through experimental data to set specific reference indexes, which is not limited here. Optionally, the lossy compression can at least include JPEG, and the lossless compression can at least include PNG or WEBP. Thus, the size of the transmission data can be effectively reduced in the process of data transmission, thereby reducing the time and bandwidth required for network transmission, which is particularly important for mobile devices and low-speed network environments.

[0180] 207: The server returns the image compression data to the terminal.

[0181] Exemplarily, the server can find the storage space corresponding to the storage address in the cloud object storage according to the storage address stored in the hash table in response to the image resource request sent by the terminal, and then take out the image binary data in the storage space and compress the image binary data to obtain corresponding image compression data, and further send the image compression data to the terminal.

[0182] 208: The terminal obtains the image compression data returned by the server and performs data decompression on the image compression data to obtain the image binary data of the original loading image obtained by image encoding.

[0183] Exemplarily, the terminal can perform data decompression on the image compression data returned by the server to obtain the image binary data of the original loading image obtained by image encoding. For example, Figure 14 The timing diagram for obtaining the image binary data is shown. The game APP on the smart phone sends an image resource request of a game character image to the server, and then the server receives the image resource request, obtains the image binary data of the original loading image of the game character image by image encoding, and stores the image binary data in the COS. The COS can perform hash mapping on the storage address of the image binary data. Further, the server sends the data compressed image compression data to the game APP on the smart phone through the COS, and further, the game APP decompresses the image compression data to obtain the image binary data. Further, the image binary data is decoded and other operations are performed to generate a target loading image.

[0184] 209: The terminal determines a target image format based on the image binary data, and performs image decoding on the image binary data through the target image format to obtain a plurality of decoding matrices.

[0185] Exemplarily, the terminal can read relevant information required for decoding, including image resolution, quantization table and other information, based on the file storage mode of the image binary data. The process of decoding the image binary data is described by taking the JPEG file data storage mode as an example. The image binary data in the JPEG file data storage mode is organized in the form of segment as a data stream, and a series of markers (Marker) are defined at the beginning of each segment.

[0186] Each Marker starts with 0xFF, followed by a 1-byte marker identifier, a 2-byte marker length, and a payload corresponding to the marker. The high bits of the marker length part are in front, and the low bits are in the back, and the first two bytes of the marker are not included. The data in the entropy encoding part is inserted with 0x00 after 0xFF by the encoder, and this byte is skipped when decoding and is not processed.

[0187] Specifically, the terminal can read these markers Marker, including SOI: Start Of Image, the beginning of the image, a fixed value of FFD8. SOFO: Start Of Frame, the beginning of the frame image, records the data length of each frame image, the number of bits of sample data, the width and height of the image, the number of color components, color component information. DQT: Define Quantization Table, DCT quantization table, record quantization table length, quantization accuracy, quantization table, table entry (length is 64 bits (8-bit precision), which records the quantization step of each pixel after 8x8 DCT transformation. DHT0: Define Huffman Table, define Huffman code table. Table length, table ID (0: luminance 1: chrominance), table type (0: direct current 1: alternating current) Different number of code words (16 bytes record the number of code words with length 1 to 16), weight. EOI: End Of Image, image end marker.

[0188] Further, the terminal can obtain a plurality of decoding matrices according to the information specified by the read markers, by establishing a Huffman table, parsing the pixel data stream of the image, and the like. The plurality of decoding matrices include a plurality of decoding matrices under the luminance component Y, a plurality of decoding matrices under the blue component C b , and a plurality of decoding matrices under the red component C r .

[0189] 210: The terminal respectively performs inverse quantization operation on each decoding matrix according to the target quantization factor determined according to the screen resolution, to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix.

[0190] Exemplarily, the terminal can determine the target quantization factor through a scaling factor calculated from the screen resolution and the resolution of the original loaded image, and further perform inverse quantization operation on the decoding matrices under different color components to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix. The calculation formula of inverse quantization is as follows:

[0191] DCT = F new_q × A Formula (3)

[0192] Wherein, DCT represents the discrete cosine transform coefficient matrix, F new_q is the target quantization factor, and A represents the decoding matrix. It should be noted that formula (3) only represents the calculation process of inverse quantization under one color component. In actual calculation, a plurality of decoding matrices under the luminance component Y, a plurality of decoding matrices under the blue component C b , and a plurality of decoding matrices under the red component C r The terminal performs inverse quantization calculation on each of the plurality of decoding matrices. Since the DCT coefficient matrices are independent of each other, the terminal can use GPU to perform the inverse quantization calculation in parallel.

[0193] 211: The terminal performs inverse DCT on each of the plurality of DCT coefficient matrices to obtain an image pixel block corresponding to each of the plurality of DCT coefficient matrices.

[0194] For example, the terminal can perform inverse DCT on the plurality of luminance coefficient matrices to obtain a luminance pixel block corresponding to each of the plurality of luminance coefficient matrices, perform inverse DCT on the plurality of blue coefficient matrices to obtain a blue pixel block corresponding to each of the plurality of blue coefficient matrices, and perform inverse DCT on the plurality of red coefficient matrices to obtain a red pixel block corresponding to each of the plurality of red coefficient matrices. The calculation formula of the inverse DCT is as follows:

[0195]

[0196] wherein S xy represents an image pixel block, i.e., a pixel value. N is the number of DCT coefficients, N = 8. Optionally, for a 2D image, inverse DCT can be performed on each row first, and then on each column to obtain complete pixel values. u , C v represents a normalizing factor:

[0197]

[0198] 212: The terminal merges images based on the color components corresponding to each of the image pixel blocks to obtain a merged image for each color component, and generates a target loading image corresponding to a screen resolution according to the merged image for each color component.

[0199] For example, the terminal can splice the 8x8 pixel blocks after inverse DCT into a complete image in a process opposite to the blocking process. The process of splicing the pixel blocks is shown in the flowcharts of FIGS. 7 and 8. Figure 9 and Figure 15 Before splicing the pixel blocks, sufficient memory is pre-allocated to store the merged image.

[0200] The game APP can calculate the memory size based on the resolution and bit depth of the original loading image obtained by decoding the image binary data. Specifically, the memory size P = W image x H image x (n / 8), where W image and H image For the width and height of the image, n is the bit depth. For example, the original loaded image resolution is 60x50, and the bit depth is 24. The size of each pixel is 24 bits (3 bytes, 1 byte = 8 bits), and the theoretical size of the picture is (60*50)*(24 / 8) = 9000B ≈ 8.79KB. Further, the game APP can use the malloc function to allocate a memory space of the calculated memory size for storing the merged image.

[0201] Further, the game APP can use a two-dimensional array or matrix to represent image data during the merging of the pixel blocks. This facilitates the access and modification of the value of each pixel. For example, the decoded image information is described using the cv::Mat object of the OpenCV library. Alternatively, a plurality of 8x8 pixel blocks are merged into a complete image in sequence, and the merging sequence can be from left to right and from top to bottom. During the merging of the pixel blocks, each pixel block needs to be traversed. In order to improve efficiency, the copying of the pixel values can be accelerated based on loop unrolling and vectorization operations. Moreover, the operation of merging the pixel blocks can be performed in parallel on multiple threads, CPUs, and GPUs. For example, the image is divided into multiple regions according to the image size, and then the blocks of these regions are merged on different threads.

[0202] During the merging of the pixel blocks, attention should be paid to the fact that the image size can not be an integer multiple of 8. This means that the pixel blocks in the last row and the last column can be incomplete. In addition, during the merging of the pixel blocks, it is necessary to ensure that invalid pixel blocks are merged. Alternatively, an empty image array can be created in an Abnormal thread, and then each pixel block is traversed and the position of the pixel block is calculated. If the pixel block is in the last row or the last column, the filled pixel coordinates are ignored.

[0203] Further, the game APP can convert the image data in the YUV color space back to the RGB color space. As shown in the color space conversion diagram. Figure 16 Specifically, the calculation formula of the color space conversion is as follows:

[0204]

[0205] where R represents the image in the red channel, G represents the image in the green channel, and B represents the image in the blue channel. Y represents the merged image in the luminance component, C b represents the merged image in the blue component, and C r represents the merged image in the red component. In the embodiments of the present application, YC b C r and the value range of RGB is [0, 255].

[0206] Further, the game APP can perform channel merging on the image under the red channel, the image under the green channel, and the image under the blue channel, and perform rendering and other optimization operations on the merged image to obtain a target loading image. Optionally, the game APP can modify the bit depth of the image according to an API supported by the smartphone system. For example, the game APP can modify the bit depth of the image by drawing the image on a canvas using the canvas API of JS and using an ImageData object. Further, the game APP can use the cv::imshow() function in OpenCV to display the image in a window. This function automatically processes color space conversion, scaling, rendering, and other operations.

[0207] Referring to Figure 14 , a structural block diagram of an image processing apparatus 300 is shown. The apparatus 300 can include:

[0208] A data acquisition module 310 is configured to acquire image binary data of an original loading image obtained through image encoding.

[0209] An image decoding module 320 is configured to determine a target image format based on the image binary data, and perform image decoding on the image binary data through the target image format to obtain a plurality of decoding matrices.

[0210] A dequantization module 330 is configured to perform dequantization on each decoding matrix respectively according to a target quantization factor determined based on a screen resolution, to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix.

[0211] A reverse transformation module 340 is configured to perform inverse discrete cosine transform on each discrete cosine transform coefficient matrix respectively, to obtain an image pixel block corresponding to each discrete cosine transform coefficient matrix.

[0212] An image generation module 350 is configured to perform image merging based on a color component corresponding to each image pixel block to obtain a merged image under each color component, and generate a target loading image corresponding to the screen resolution according to the merged image under each color component.

[0213] In some embodiments, the image decoding module 320 can include an analysis unit, a reading unit, a determination unit, and a decoding unit.

[0214] The analysis unit is configured to perform data analysis on the image binary data to obtain a first byte in the image binary data.

[0215] The reading unit is configured to perform byte reading according to the first byte to determine a target image format corresponding to the image binary data.

[0216] determining unit, configured to determine a corresponding target decoding algorithm based on the target image format;

[0217] decoding unit, configured to perform image decoding on the image binary data by using the target decoding algorithm to obtain a plurality of luminance decoding matrices, a plurality of blue decoding matrices and a plurality of red decoding matrices, and determine the plurality of luminance decoding matrices, the plurality of blue decoding matrices and the plurality of red decoding matrices as a plurality of decoding matrices.

[0218] In some embodiments, the discrete cosine transform coefficient matrix includes a luminance coefficient matrix, a blue coefficient matrix and a red coefficient matrix; the inverse quantization module 330 includes an acquisition unit, an analysis unit, an adjustment unit, a first inverse quantization unit, a second inverse quantization unit and a third inverse quantization unit.

[0219] The acquisition unit is configured to acquire a screen resolution.

[0220] The analysis unit is configured to perform data analysis on the image binary data to obtain a first original quantization factor and a second original quantization factor.

[0221] The adjustment unit is configured to adjust the first original quantization factor and the second original quantization factor based on the screen resolution to obtain a corresponding first quantization factor and a second quantization factor, respectively.

[0222] The first inverse quantization unit is configured to perform inverse quantization operation on the plurality of luminance decoding matrices based on the first quantization factor to obtain a corresponding plurality of luminance coefficient matrices.

[0223] The second inverse quantization unit is configured to perform inverse quantization operation on the plurality of blue decoding matrices according to the second quantization factor to obtain a corresponding plurality of blue coefficient matrices.

[0224] The third inverse quantization unit is configured to perform inverse quantization operation on the plurality of red decoding matrices by using the second quantization factor to obtain a corresponding plurality of red coefficient matrices.

[0225] In some embodiments, the adjustment unit can be specifically configured to: obtain a scaling factor based on the screen resolution and an original resolution of the original loaded image; and determine an inverse of the scaling factor as a basic quantization factor.

[0226] adjust the first original quantization factor based on the basic quantization factor to obtain a first quantization factor; and adjust the second original quantization factor by using the basic quantization factor to obtain a second quantization factor.

[0227] In some embodiments, the image pixel block includes a luminance pixel block, a blue pixel block, and a red pixel block; the inverse transform module 340 can include a first transform unit, a second transform unit, and a third transform unit.

[0228] The first transform unit is configured to perform inverse discrete cosine transform on the plurality of luminance coefficient matrices to obtain a luminance pixel block corresponding to each luminance coefficient matrix.

[0229] The second transform unit is configured to perform inverse discrete cosine transform on the plurality of blue coefficient matrices to obtain a blue pixel block corresponding to each blue coefficient matrix.

[0230] The third transform unit is configured to perform inverse discrete cosine transform on the plurality of red coefficient matrices to obtain a red pixel block corresponding to each red coefficient matrix.

[0231] In some embodiments, the image processing apparatus can further include a scaling module configured to perform pixel scaling on each of the image pixel blocks based on the scaling factor to obtain a scaled pixel block corresponding to each discrete cosine transform coefficient matrix.

[0232] The image generation module 350 can be further configured to perform image merging based on a color component corresponding to each of the scaled pixel blocks to obtain a merged image under each color component, and generate a target loading image corresponding to the screen resolution according to the merged image under each color component.

[0233] In some embodiments, the image processing apparatus can further include a memory allocation module configured to: obtain an image resolution and a bit depth of the original loading image from the image binary data; and allocate a preset memory space based on the image resolution and the bit depth.

[0234] The image generation module 350 can be further configured to store the merged image under each color component based on the preset memory space.

[0235] In some embodiments, the image generation module 350 can include a thread acquisition unit, a region determination unit, and an image merging unit.

[0236] The thread acquisition unit is configured to acquire a plurality of concurrent processes.

[0237] The region determination unit is configured to determine a merging region under different color components based on a number of the plurality of concurrent processes and the image resolution.

[0238] The image merging unit is configured to perform image merging on image pixel blocks in the merging region by each of the concurrent processes to obtain a merged image under each color component.

[0239] In some embodiments, the color components include a luminance component, a blue component and a red component, and the merged images include a luminance merged image, a blue merged image and a red merged image; the image merging unit can be specifically configured to: perform image merging on the image pixel blocks in the merging area under the luminance component based on the each concurrent process to obtain the luminance merged image; perform image merging on the image pixel blocks in the merging area under the blue component according to the each concurrent process to obtain the blue merged image; and perform image merging on the image pixel blocks in the merging area under the red component by the each concurrent process to obtain the red merged image.

[0240] In some embodiments, the image generation module 350 can further include a spatial conversion unit, a channel merging unit and an image optimization unit.

[0241] The spatial conversion unit is configured to perform color space conversion on the luminance merged image, the blue merged image and the red merged image respectively to obtain a red channel image, a green channel image and a blue channel image.

[0242] The channel merging unit is configured to perform channel merging on the red channel image, the green channel image and the blue channel image to obtain an intermediate image.

[0243] The image optimization unit is configured to perform an optimization operation on the intermediate image to obtain a target loading image corresponding to the screen resolution, and the optimization operation at least includes one of image scaling or image rendering.

[0244] In some embodiments, the image processing apparatus can further include a query module and a request module.

[0245] The query module is configured to query whether the target loading image is cached locally in response to an image loading instruction.

[0246] The request module is configured to, when the target loading image is not cached locally, send an image resource request to a server, so that the server performs data compression on image binary data obtained by encoding the original loading image, and returns compressed image compression data, and the compressed image compression data at least includes the image binary data.

[0247] In some embodiments, the data acquisition module can be specifically configured to: acquire the image compression data returned by the server; and perform data decompression on the image compression data to obtain image binary data of the original loading image obtained through image encoding. The application can acquire the image binary data of the original loading image obtained through image encoding, determine a target image format based on the image binary data, and perform image decoding on the image binary data through the target image format to obtain a plurality of decoding matrices. Then, the target quantization factor determined according to the screen resolution is used to perform inverse quantization operation on each decoding matrix respectively to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix. Since the target quantization factor is determined according to the screen resolution, the target quantization factor can be used to perform inverse quantization operation on the plurality of decoding matrices that is adaptive to the screen resolution, so as to scale the pixel information of the original loading image to an adaptive degree that meets the screen display. Further, inverse discrete cosine transform is performed on each discrete cosine transform coefficient matrix respectively to obtain an image pixel block corresponding to each discrete cosine transform coefficient matrix. Then, image merging is performed based on the color components corresponding to each image pixel block to obtain a merged image under each color component, and a target loading image corresponding to the screen resolution is generated according to the merged image under each color component. In this way, in the process of generating the target loading image based on the image binary data of the original loading image returned by the server, the target quantization factor determined according to the screen resolution can guide the pixel of the original loading image to be scaled in a targeted manner during inverse quantization, so that the scaled image pixel can be adapted to the screen resolution. Based on this, the inverse discrete cosine transform and the image merging can be performed to generate the target loading image to be loaded that is adapted to the screen resolution of the terminal, and then the image is adaptively loaded, and the terminal is prevented from pre-storing the image to be loaded locally, thereby effectively reducing the consumption of storage resources.

[0248] Referring to Figure 18 which shows a structural block diagram of an image encoding device 400 provided by an embodiment of the application. The device 400 can include:

[0249] An image acquisition module 410, configured to acquire an original loading image.

[0250] A request response module 420, configured to acquire the screen resolution of the terminal in response to an image resource request sent by the terminal.

[0251] A data detection module 430, configured to perform image encoding on the original loading image to obtain corresponding image binary data when it is detected that the image binary data corresponding to the screen resolution has not been returned.

[0252] The policy determination module 440 is configured to determine a target compression policy according to a network state and a computing performance of the terminal.

[0253] The data transmission module 450 is configured to perform data compression on the image binary data based on the target compression policy to obtain image compression data, and return the image compression data to the terminal, so that the terminal generates a target loading image based on the image compression data, wherein the target loading image is an image generated by the image processing method.

[0254] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device, module and the like can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0255] In several embodiments provided in the present application, the coupling between the modules can be electrical, mechanical or other forms of coupling.

[0256] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically independently, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0257] As shown in the Figure 19 The embodiments of the present application also provide a computer device 500, which includes a processor 510, a memory 520, a power supply 530 and an input unit 540. The memory 520 stores a computer program. When the computer program is called by the processor 510, various method steps provided in the above embodiments can be executed. Those skilled in the art can understand that the structure of the computer device shown in the figure does not constitute a limitation on the computer device, and can include more or fewer components than those shown in the figure, or combine certain components, or different component arrangements. Among them:

[0258] The processor 510 can include one or more processing cores. The processor 510 utilizes various interfaces and lines to connect various parts within the entire battery management system, invokes data stored in the memory 520, performs various functions and processes data of the battery management system, and performs various functions and processes data of the computer device by running or executing instructions, programs, instruction sets, or program sets stored in the memory 520, thereby performing overall control of the computer device. Alternatively, the processor 510 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA). The processor 510 can integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs, etc.; the GPU is responsible for rendering and drawing display content; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 510, but can be realized by a separate communication chip.

[0259] The memory 520 can include a random access memory (RAM) and can also include a read-only memory (ROM). The memory 520 can be used to store instructions, programs, instruction sets, or program sets. The memory 520 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described above, etc. The data storage area can also store data created by the computer device in use (such as a phone book and audio / video data), etc. Accordingly, the memory 520 can also include a memory controller to provide access for the processor 510 to the memory 520.

[0260] The power supply 530 can be logically connected to the processor 510 through a power management system, so as to realize functions of managing charging, discharging, and power consumption management, etc. through the power management system. The power supply 530 can also include one or more than one direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power state indicator, etc. any component.

[0261] The input unit 540 can be used to receive inputted digital or character information, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0262] Although not shown, the computer device 500 can further include a display unit, etc., which will not be described herein. In particular, in the present embodiment, the processor 510 in the computer device will load the executable files corresponding to the threads of one or more computer programs into the memory 520 according to the instructions, and run the telephone book and audio / video data stored in the memory 520 by the processor 510, so as to implement the various method steps provided by the foregoing embodiments.

[0263] As shown in Figure 20 The present embodiment further provides a computer-readable storage medium 600 having a computer program 610 stored therein, which can be invoked by a processor to execute the various method steps provided by the present embodiment.

[0264] The computer-readable storage medium can be an electronic storage such as a flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk or ROM. Alternatively, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium 600 has storage space for computer programs to execute any of the method steps in the above embodiments. These computer programs can be read out from or written into one or more computer program products. The computer programs can be compressed in a suitable form.

[0265] According to an aspect of the present application, a computer program product is provided, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the various method steps provided by the above embodiments.

[0266] The terms "include", "comprise" and "comprising" in the specification and throughout the claims, shall be understood to encompass the meaning of "consist of", "consisting of", "consists of", "consisting essentially of", "consisting essentially of", and "consists essentially of", as well as the meaning of "include", "comprise" and "comprising", unless otherwise specified. The term "module" or "unit" refers to a computer program or a part of a computer program having a predetermined function and working together with other related parts to achieve a predetermined target, and can be implemented in whole or in part by using software, hardware such as a processing circuit or a memory, or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0267] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or", used to describe the association relationship of associated objects, means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents a "or" relationship between the associated objects before and after it. "At least one of the following" or the like means any combination of the items, including single item or any combination of multiple items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be singular or plural.

[0268] It should be understood that, in the description of the embodiments of the present application, the meaning of multiple (or multiple items) is two or more, greater than, less than, more than, etc. is not included in the number, above, below, etc. is included in the number.

[0269] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutual objects can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0270] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed to multiple network units. Part or all of the units may be selected according to actual needs to achieve the purpose of the embodiment scheme. In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0271] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. It should be understood that the various embodiments provided by the present application can be combined in any way to achieve different technical effects.

[0272] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any person skilled in the art can make slight changes or modifications to the above disclosed technical content without departing from the scope of the technical solutions of the present application, and any equivalent embodiments with equivalent changes are equivalent to the above embodiments. Any modification, equivalent change and modification of the above embodiments according to the technical essence of the present application, as long as it does not deviate from the technical solutions of the present application, is still within the scope of the technical solutions of the present application.< / picture>

Claims

1. An image processing method, characterized in that: The method comprises: Obtain image binary data obtained by image encoding of the original loaded image; determining a target image format based on the image binary data, and performing image decoding on the image binary data using the target image format to obtain a plurality of decoding matrices; Performing an inverse quantization operation on each decoding matrix according to a target quantization factor determined by the screen resolution to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix; Performing inverse discrete cosine transform on each discrete cosine transform coefficient matrix to obtain an image pixel block corresponding to each discrete cosine transform coefficient matrix; Image merging is performed based on the color component corresponding to each of the image pixel blocks to obtain a merged image under each color component, and a target loaded image corresponding to the screen resolution is generated based on the merged image under each color component.

2. The method according to claim 1, characterized in that The determining of a target image format based on the image binary data, and performing image decoding on the image binary data using the target image format to obtain a plurality of decoding matrices includes: Parsing the binary image data to obtain the first byte of the binary image data; Read the byte according to the first byte to determine the target image format corresponding to the image binary data; Determining a corresponding target decoding algorithm based on the target image format; The image binary data is image decoded using the target decoding algorithm to obtain multiple brightness decoding matrices, multiple blue decoding matrices, and multiple red decoding matrices, and the multiple brightness decoding matrices, multiple blue decoding matrices, and multiple red decoding matrices are determined as multiple decoding matrices.

3. The method according to claim 2, characterized in that The discrete cosine transform coefficient matrix includes a brightness coefficient matrix, a blue coefficient matrix, and a red coefficient matrix; the target quantization factor determined according to the screen resolution is used to perform an inverse quantization operation on each decoding matrix to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix, including: Get the screen resolution; Performing data parsing on the image binary data to obtain a first original quantization factor and a second original quantization factor; Adjusting the first original quantization factor and the second original quantization factor respectively based on the screen resolution to obtain corresponding first quantization factor and second quantization factor; Performing an inverse quantization operation on the plurality of luminance decoding matrices based on the first quantization factor to obtain a corresponding plurality of luminance coefficient matrices; Performing an inverse quantization operation on the plurality of blue decoding matrices according to the second quantization factor to obtain a corresponding plurality of blue coefficient matrices; Performing an inverse quantization operation on the multiple red decoding matrices using the second quantization factor to obtain corresponding multiple red coefficient matrices.

4. The method according to claim 3, characterized in that The adjusting the first original quantization factor and the second original quantization factor based on the screen resolution to obtain corresponding first quantization factor and second quantization factor respectively includes: Obtaining a scaling factor based on the screen resolution and the original resolution of the originally loaded image; Determining the reciprocal of the scaling factor as a basic quantization factor; Adjusting the first original quantization factor according to the basic quantization factor to obtain a first quantization factor; The second original quantization factor is adjusted by the basic quantization factor to obtain a second quantization factor.

5. The method according to claim 4, characterized in that The image pixel blocks include luminance pixel blocks, blue pixel blocks, and red pixel blocks; and performing inverse discrete cosine transform on each discrete cosine transform coefficient matrix to obtain the image pixel blocks corresponding to each discrete cosine transform coefficient matrix includes: Performing inverse discrete cosine transform on the multiple brightness coefficient matrices to obtain a brightness pixel block corresponding to each brightness coefficient matrix; Performing inverse discrete cosine transform on the multiple blue coefficient matrices to obtain a blue pixel block corresponding to each blue coefficient matrix; Performing inverse discrete cosine transform on the multiple red coefficient matrices to obtain a red pixel block corresponding to each red coefficient matrix.

6. The method according to claim 4, characterized in that After performing inverse discrete cosine transform on each discrete cosine transform coefficient matrix to obtain an image pixel block corresponding to each discrete cosine transform coefficient matrix, the method further includes: Perform pixel scaling on each of the image pixel blocks based on the scaling factor to obtain a scaled pixel block corresponding to each discrete cosine transform coefficient matrix; The image merging is performed based on the color component corresponding to each of the image pixel blocks to obtain a merged image under each color component, and the target loaded image corresponding to the screen resolution is generated according to the merged image under each color component, including: Image merging is performed based on the color component corresponding to each of the scaled pixel blocks to obtain a merged image under each color component, and a target loaded image corresponding to the screen resolution is generated according to the merged image under each color component.

7. The method according to any one of claims 1 to 6, characterized in that Before performing image merging based on the color components corresponding to each of the image pixel blocks to obtain a merged image under each color component, the method further includes: Obtaining, from the image binary data, an image resolution and a bit depth of the original loaded image; Allocating a preset memory space based on the image resolution and the bit depth; Obtaining a merged image under each color component includes: The combined image under each color component is stored based on the preset memory space.

8. The method according to claim 7, characterized in that The performing image merging based on the color component corresponding to each of the image pixel blocks to obtain a merged image under each color component includes: Get multiple concurrent processes; determining, based on the number of the multiple concurrent processes and the image resolution, merging regions under different color components; Each concurrent process performs image merging on the image pixel blocks in the merging area to obtain a merged image under each color component.

9. The method according to claim 8, characterized in that The color components include a brightness component, a blue component, and a red component, and the merged image includes a brightness merged image, a blue merged image, and a red merged image; The image merging is performed on the image pixel blocks in the merging area by each concurrent process to obtain a merged image under each color component, including: Performing image merging under a luminance component on the image pixel blocks in the merging area based on each concurrent process to obtain a luminance merged image; Performing image merging of the image pixel blocks in the merging area under the blue component according to each concurrent process to obtain a blue merged image; By performing image merging under the red component on the image pixel blocks in the merging area through each of the concurrent processes, a red merged image is obtained.

10. The method according to claim 9, characterized in that Generating the target loaded image corresponding to the screen resolution according to the merged image under each color component includes: Performing color space conversion on the brightness merged image, the blue merged image, and the red merged image, respectively, to obtain a red channel image, a green channel image, and a blue channel image; Merging the red channel image, the green channel image, and the blue channel image to obtain an intermediate image; An optimization operation is performed on the intermediate image to obtain a target loaded image corresponding to the screen resolution, wherein the optimization operation includes at least one of image scaling or image rendering.

11. The method according to claim 1, wherein Before obtaining the image binary data obtained by image encoding the original loaded image, the method further includes: In response to the image loading instruction, query whether the target loading image is cached locally; When the target loaded image is not cached locally, an image resource request is sent to the server, so that the server compresses the image binary data obtained by encoding the original loaded image and returns the compressed image data, which at least includes the image binary data.

12. The method according to claim 11, characterized in that The step of obtaining binary image data obtained by encoding the original loaded image includes: Obtaining the compressed image data returned by the server; The image compressed data is decompressed to obtain image binary data obtained by image encoding of the original loaded image.

13. An image coding method, characterized in that: The method comprises: Get the original loaded image; In response to an image resource request sent by a terminal, obtaining a screen resolution of the terminal; When it is detected that no image binary data corresponding to the screen resolution has been returned, performing image encoding on the original loaded image to obtain corresponding image binary data; Determining a target compression strategy based on a network state and computing performance of the terminal; The image binary data is compressed based on the target compression strategy to obtain image compression data, and the image compression data is returned to the terminal so that the terminal generates a target loading image based on the image compression data, wherein the target loading image is an image generated by the image processing method according to any one of claims 1 to 12.

14. An image processing device, characterized in that: The device comprises: A data acquisition module is used to obtain image binary data obtained by image encoding of the original loaded image; an image decoding module, configured to determine a target image format based on the image binary data, and perform image decoding on the image binary data using the target image format to obtain a plurality of decoding matrices; An inverse quantization module is used to perform an inverse quantization operation on each decoding matrix according to a target quantization factor determined by the screen resolution to obtain a discrete cosine transform coefficient matrix corresponding to each decoding matrix; an inverse transform module, configured to perform an inverse discrete cosine transform on each discrete cosine transform coefficient matrix to obtain an image pixel block corresponding to each discrete cosine transform coefficient matrix; The image generation module is used to merge images based on the color components corresponding to each image pixel block to obtain a merged image under each color component, and generate a target loaded image corresponding to the screen resolution based on the merged image under each color component.

15. An image coding device, characterized in that The device comprises: Image acquisition module, used to obtain the original loaded image; A request response module, configured to obtain the screen resolution of the terminal in response to an image resource request sent by the terminal; a data detection module, configured to, when detecting that no image binary data corresponding to the screen resolution has been returned, perform image encoding on the original loaded image to obtain corresponding image binary data; A strategy determination module, configured to determine a target compression strategy based on a network state and the computing performance of the terminal; A data transmission module is used to compress the image binary data based on the target compression strategy to obtain image compression data, and return the image compression data to the terminal, so that the terminal generates a target loading image based on the image compression data, wherein the target loading image is an image generated according to the image processing method according to any one of claims 1 to 12.

16. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the image processing method according to any one of claims 1 to 12 or the image encoding method according to claim 13 is implemented.

17. A computer device, characterized in that: include: Memory; A processor, wherein a computer program is stored on the memory, and when the computer program is executed by the processor, the image processing method according to any one of claims 1 to 12 or the image encoding method according to claim 13 is implemented.

18. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the image processing method according to any one of claims 1 to 12 or the image encoding method according to claim 13 is implemented.

Images