A method and system for real-time image processing using the HTTP protocol

Abstract

This invention discloses a real-time image processing method and system using the HTTP protocol. It constructs an image processing cluster and uses a request load balancer to distribute image request pressure across multiple image processing worker nodes. It verifies in real-time whether the original image cached on the user terminal is consistent with the image repository to avoid duplicate image downloads by the user terminal, thereby reducing the service's image processing pressure and ensuring the timeliness of the user terminal image cache. An external second-level cache is constructed to cache image processing results to handle duplicate image processing requests, enabling multiple users to share the image processing result cache, reducing the workload of the image processing worker nodes and lowering the average response latency. A URL parsing plugin mechanism is constructed, allowing different business applications to customize URL parsing logic according to their own characteristics, achieving the goal of one image service supporting the image processing needs of different business websites without intruding on the URL definitions of the business websites.

Description

Technical Field

[0001] This invention relates to the field of Internet image services, and in particular to a method and system for real-time image processing using the HTTP protocol. Background Technology

[0002] With the development of the internet, many websites deploy a large amount of image data (e.g., e-commerce websites deploy a large number of product images). To achieve service categorization and management, these websites usually separate image resources into an independent image service and assign it an independent website domain (usually a subdomain). As we all know, websites often change the display specifications and formats of images according to layout design needs. Early approaches involved reading the original image from the image storage service and then adapting the specifications and formats on the terminal, resulting in significant network transmission overhead. Later, image preprocessing solutions were used to pre-produce images of fixed specifications and formats based on the original image and store them for website applications to read and display. This approach created significant storage pressure for pre-processed images, and when a new specification or format was added, all original images on the website needed to be pre-processed, which was time-consuming and prolonged the business product iteration cycle. Currently, real-time image processing technology is widely used, which calculates the target image of specified specifications and formats in real time based on the original image and directly displays the target image on the website. However, due to the large computational load of image processing, this solution faces problems of high CPU consumption and long response latency. Meanwhile, many internet companies want a single image system to serve multiple business websites within their organization, without intruding on the URL definitions of each website. This invention proposes a real-time image processing solution to address these issues. It utilizes technologies such as two-level caching and cluster load balancing to improve real-time processing efficiency. Furthermore, it introduces custom URL parsing logic as a plugin, ensuring that the URL definitions for specific business applications are not compromised.

[0003] Existing technologies such as "A method, apparatus, electronic device and medium for displaying images based on caching" (CN111324586A), "A method and apparatus for image caching" (CN 107577711 A), "A method and apparatus for displaying thumbnails" (CN 101807199 B), and "Local thumbnail high-speed cache" (CN 101167047 B) all aim to solve the problem of caching image processing results on the user end. However, in many scenarios, especially in Internet scenarios, the caching on the user end cannot achieve the goal of solving the problem of image processing results being repeatedly downloaded by multiple users, thus failing to fundamentally solve the problem of high real-time processing pressure on the image server.

[0004] The existing technology, "A Real-Time Image Processing System and Method Based on URL" (CN 201811417229X), mainly relies on CDN caching of image processing results to reduce the computational pressure on image processing services. However, this solution suffers from the problem of non-real-time CDN cache updates and incurs CDN rental costs. Furthermore, similar technical solutions do not support flexible customization of URL parsing logic, significantly limiting the application scope of this type of image service. In addition, the system uses a relational database to store the original image's metadata, which increases system complexity and reduces system response efficiency.

[0005] Therefore, there is a need for a more efficient, convenient, widely applicable, and seamless real-time image processing method and system. Summary of the Invention

[0006] The technical problem this invention aims to solve is to overcome the shortcomings of existing technologies and address the problems existing in current image service technologies. This invention provides a real-time image processing method and system based on the HTTP protocol. It constructs an image processing cluster and uses a request load balancer to distribute image request pressure across multiple image processing nodes. By real-time verification of whether the original image cached on the user terminal matches the image in the image repository, it avoids duplicate image downloads by the user terminal, thereby reducing the service's image processing pressure and ensuring the timeliness of the user terminal's image cache. By constructing an external second-level cache to cache image processing results, it addresses duplicate image processing requests, enabling multiple users to share the image processing result cache, reducing the workload of image processing nodes and lowering the average response latency. Furthermore, by constructing a URL parsing plugin mechanism, different business applications can customize URL parsing logic according to their own characteristics, achieving the goal of one image service supporting the image processing needs of different business websites without intruding on the URL definitions of the business websites.

[0007] To address the aforementioned technical problems, this invention provides a real-time image processing method using the HTTP protocol, characterized by the following steps:

[0008] Step 1: The preset object storage system supports storing the original image's unique identifier and original image blob given during upload, and supports reading the original image blob based on the original image's unique identifier; it also supports storing the object's metadata and reading the metadata based on the original image's unique identifier. Users upload the original image to the object storage cluster by providing the original image's unique identifier and original image blob through the preset object storage system's own client, and write the original image's version identifier into the metadata.

[0009] Step 2: Preset an HTTP request load balancer to support the distribution of image access requests to worker nodes in the real-time image processing cluster according to specified strategies;

[0010] Step 3: The preset request processing controller in the worker node of the real-time image processing cluster receives the image access request and calls the URL parsing module to parse the URL of the image access request into the original image unique identifier and the image processing target parameters; the URL parsing module finds the corresponding URL processing logic body based on the domain name contained in the URL of the image access request, and further calls the first interface implemented in the logic body to obtain the original image unique identifier and the image processing target parameters corresponding to the current image access request.

[0011] Step 4: The request processing controller calls the preset terminal cache update determination module to determine whether the current request's cache on the terminal needs to be updated; the terminal cache update determination module uses the original image's unique identifier to obtain the version identifier of the corresponding original image from the object storage system; when the version identifier is consistent with the terminal cache version identifier contained in the request, it directly returns a 304 status code; otherwise, it proceeds to step 5.

[0012] Step 5: A secondary cache system is preset, supporting the writing and reading of cached data. The secondary cache system has the ability to automatically evict cold data and scale horizontally online. The request processing controller calls the cache processing module to perform cache result lookup and return processing. The cache processing module uses the original image unique identifier to read the current cache version identifier from the secondary cache. When the cache version identifier is consistent with the version identifier obtained from the object storage system in Step 4, the original image unique identifier and the image processing target parameter are combined into a request unique identifier for the current image request. The module attempts to use the request unique identifier to look up the cache result of the current image request in the secondary cache system. If the cache result exists, it is returned directly; otherwise, proceed to Step 6.

[0013] Step 6: Preset image processing engine, which supports image thumbnailing, format conversion, rotation, and watermarking; The request processing controller uses the unique identifier of the original image contained in the request to read the original image and version identifier from the object storage system, and then calls the image processing engine to process the original image in real time into the image specifications and format specified in the request.

[0014] Step 7: The cache processing module uses the processing result from Step 6 to write the original image unique identifier and its cache version identifier pair, and the current request unique identifier and processing result information pair to the second-level cache system;

[0015] Step 8: The request processing controller returns the processing result from Step 7 to the user terminal.

[0016] In step 1, the original image unique identifier and the original image blob are stored as key-value pairs, and the version identifier is the image upload timestamp; if the same unique identifier is used to upload the original image repeatedly, the last uploaded original image and its version identifier are read.

[0017] In step 2, the real-time image processing cluster includes multiple real-time image processing worker nodes. Each real-time image processing worker node is connected to HTTP image processing requests, which is used in conjunction with the HTTP request load balancer to achieve real-time processing pressure sharing and system high availability.

[0018] In step 3, the URL parsing module defines a set of URL parsing plugin APIs. The URL parsing plugin APIs include two interfaces: the first interface is used to convert the URL into the target parameter for image processing, and the second interface is used to return the domain name corresponding to the plugin. The URL parsing module is responsible for loading the specific implementation of the user-implemented URL parsing plugin API and pairing the business domain name with the plugin processing logic body, supporting the retrieval of the corresponding URL processing logic body based on the business domain name.

[0019] In step 4, the cache update determination module uses the original image's unique identifier to obtain only the latest version identifier of the original image from the object storage system. When the version identifier matches the terminal cache version identifier included in the request, it directly returns a 304 status code.

[0020] In step 5, the cache processing module uses the original image unique identifier to read the image version identifier in the second-level cache. When the cached version identifier is consistent with the latest version identifier obtained from the object storage system in step 4, the original image unique identifier and the image processing target parameter are combined into a request unique identifier for the current image request, and the request unique identifier is used to search for the cached result of the current image request in the second-level cache system. If the cached result exists, it is returned directly.

[0021] In step 6, the image processing engine is built into the real-time image processing work node and works in the same process as other modules in the real-time image processing work node; the request processing controller calls the corresponding interface of the image processing engine according to the image processing target parameters obtained by the URL parsing module. The image processing target parameters include the target image size, image conversion format, and image rotation angle.

[0022] In step 7, the cache writing order is as follows: first, write the information pair of the current request unique identifier and the processing result, and then write the original image unique identifier and its cache version identifier pair.

[0023] A real-time image processing system based on the HTTP protocol, characterized in that it includes an HTTP request load balancer, a real-time image processing cluster, a secondary caching system, and an object storage system;

[0024] The HTTP request load balancer is used to receive website user requests and distribute them to the worker nodes in the real-time image processing cluster according to a round-robin strategy.

[0025] The real-time image processing cluster includes more than one independent real-time image processing node. Each processing node includes a request processing controller, a URL parsing module, a cache processing module, a terminal cache update determination module, and an image processing engine.

[0026] The request processing controller is responsible for scheduling the image request processing process, including the control process from receiving the request to returning the result;

[0027] The URL parsing module is responsible for parsing the image request URL into image-related parameters that the system can recognize;

[0028] The cache processing module encapsulates the logic related to image processing result caching, including writing, eviction, and update operations of cached results;

[0029] The terminal cache update determination module is responsible for determining whether the image cached by the user terminal needs to be updated based on the terminal information provided in the image request.

[0030] The image processing engine is responsible for calculating image requests, including image thumbnail calculation and image format conversion calculation;

[0031] The secondary caching system is used to cache the real-time image processing results; it is also used to cache the cache version identifier of the original image, so that the real-time image processing node can determine whether to refresh the cache based on the cache version identifier.

[0032] The object storage system is used to store the original image and also to store original image version identifier metadata.

[0033] The preferred HTTP request load balancer is Nginx; the preferred secondary caching system is Redis; the preferred object storage system is Minio; and the preferred image processing engine is ImageMagick.

[0034] The beneficial effects achieved by this invention are as follows:

[0035] This invention interfaces with an external secondary cache system, which can cache real-time image processing results to the maximum extent, reducing the CPU consumption and average response latency of the system's real-time processing. Since multiple worker nodes in the image processing cluster share the same secondary cache system, the problems of repeated image processing and repeated caching can be avoided.

[0036] This invention holds the original image upload timestamp in the image processing cache, thereby achieving real-time synchronization between the secondary cache results and the original image in the object storage system, ensuring that the image processing results always originate from the latest original image.

[0037] This invention can quickly determine whether to return an unmodified code (i.e., 304) based on the image timestamp contained in the terminal cache (e.g., browser), reducing repeated requests for unmodified images by the terminal and reducing server pressure; at the same time, the server ensures that the user terminal cache is always based on the latest original image by verifying the terminal cache timestamp, avoiding the problem of user terminal cache updates not being updated in real time interfering with user experience.

[0038] This invention supports the parsing of different business URLs in the form of plugins. Image requests from different domains can select the appropriate URL parsing plugin based on the domain name. This allows different business applications to customize URL parsing logic according to their own characteristics, so as to achieve the goal of supporting image processing of different business websites with a single image service, without intruding on the URL definition of the business websites. Attached Figure Description

[0039] Figure 1 A simplified flowchart of a method according to an exemplary embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a business URL plugin, which is an exemplary embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram illustrating the data state changes in the method flow of an exemplary embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of the system module structure in an exemplary embodiment of the present invention. Detailed Implementation

[0043] The following are explanations of the terms used in this technical solution, which are conventional expressions in the field, including:

[0044] Original image blob: The binary form of image data.

[0045] Image processing target parameters: For example, in the case of image thumbnails, it is necessary to specify the target length and width of the thumbnail image. The target length and width here are the processing target parameters.

[0046] Original image unique identifier: a key used to write and read the original image from the object storage system.

[0047] Unique identifier for image requests: For example, in the case of image thumbnails, the unique identifier for the current image request must include the unique identifier of the original image, the length and width of the thumbnail target.

[0048] like Figure 1 The flowchart shown in this embodiment of the invention illustrates a real-time image processing method using the HTTP protocol. The specific steps include:

[0049] Step 1: Write the original image data to the object storage system: Users upload business images that need to be stored through the client or tool corresponding to a specific object storage system, and the business system holds a unique identifier for the business image. The object storage system stores the unique image identifier, the image blob key-value pair, and its upload timestamp. Subsequently, it needs to be able to retrieve the image blob and its upload timestamp from the object storage system based on the unique identifier. At this point, if... Figure 3 As shown in the object storage system status 301, the unique identifier for the written data is table1~1, and the timestamp of the uploaded original image is T1.

[0050] Step 2: Distribute image processing requests to specific worker nodes according to a round-robin strategy: Nginx is selected and deployed as the load balancer in this embodiment to distribute image processing requests to specific worker nodes according to a round-robin strategy, thereby achieving high availability of the system of the present invention; when the system is overloaded, additional worker nodes can be deployed according to the actual load.

[0051] Step 3: Implement URL parsing plugins for different business applications and load the plugins: Business application developers implement the URL parsing plugin API interface and deliver it as a business plugin package, such as... Figure 2 The business URL plugins shown have domain names www.test1.com and www.test2.com for the URL parsing plugin packages 401 and 402 delivered by the business applications, respectively. The worker nodes load the URL parsing plugin packages, forming a memory mapping between the business domain names and the URL parsing plugin logic bodies. URL parsing plugin logic bodies 403 and 404 correspond to the two business applications with domain names www.test1.com and www.test2.com, respectively. When a new business application needs to be integrated into the system of this invention, only the corresponding business plugin package needs to be delivered and loaded; there is no need to deploy separate image processing systems for different businesses.

[0052] Step 4: Invoke different URL parsing plugins according to business needs to convert the image request URL into image processing target parameters: After receiving an image request from the load balancer, the request processing controller in the worker node calls the URL parsing module to parse the request URL into image processing parameters that the system can understand. Specifically, the URL parsing module obtains the domain name of the current HTTP request, and then from... Figure 2The URL parsing plugin logic body 403 and 404, as shown in the mapping pairs with the business domain name, are used to read the corresponding URL parsing plugin logic body and parse the target parameters of the current image request. This system can handle various scenarios such as image thumbnails, image watermarks, and image format conversions. However, for ease of explanation, this embodiment uses the image thumbnail processing scenario as an example. In this scenario, the target parameters parsed by the URL plugin include: the original image's unique identifier, the thumbnail width, and the thumbnail height, such as... Figure 3 The three parameters shown are: table1~1, 200, and 200.

[0053] Step 5: Handling the first occurrence of an image request: When processing the first request to convert the original image uniquely identified as table1~1 into a thumbnail with a width of 200 and a height of 200, such as... Figure 3 As shown in user terminal cache status 302, the user terminal cache corresponding to the current request is empty. Therefore, the terminal cache update judgment module reads the HTTP protocol header information If-Modified-Since and finds it empty, and then calls the cache processing module to attempt to read the second-level cache using the current request key combination table1~1~200~200; at this time, as Figure 3 As indicated by status 303 in the secondary cache, there is currently no secondary cache value corresponding to the requested key combination table1~1~200~200. The real-time image processing process then begins. Specifically, the original image identified as table1~1 and its timestamp (T1) are read from the object storage system and passed along with the image processing target parameters (thumbnail width 200, thumbnail height 200) to the image processing engine's thumbnail interface. The image processing engine calculates the target thumbnail image based on the input parameters. The request processing controller calls the cache processing module to write two key-value pairs to the secondary cache, achieving... Figure 3 The secondary cache status is 306; the image thumbnail result is returned as the HTTP message body, and the original image timestamp (T1) is returned as the HTTP last-modified-time header. The user terminal caches the corresponding processing results, achieving... Figure 3 The user terminal cache status is 305.

[0054] Step 6: Handling scenarios where the user's client cache is valid for a specific image request: If the same user initiates another request to read the same thumbnail from tables 1-1 (thumbnail width 200, thumbnail height 200), the terminal cache update judgment module reads the HTTP protocol header If-Modified-Since to determine that the cached version of the corresponding request in the current user's terminal is T1. It then reads the last upload time of tables 1-1 from the object storage system as T1, and compares the two to find they are the same. Figure 3As shown in user terminal cache status 305 and object storage system status 304, the user terminal cache is deemed valid, and a 304 code is returned directly.

[0055] Step 7: Handling a scenario where a second-level cache hit occurs for an image request: Another user initiates the same thumbnail request to read tables1-1 (thumbnail width 200, thumbnail height 200). The terminal cache update judgment module reads the HTTP protocol header If-Modified-Since and finds it empty. Then, it reads the original image timestamps corresponding to tables1-1 from the second-level cache and object storage system. At this point, the two timestamps are consistent. Figure 3 The second-level cache status is 306 and the object storage system status is 304, indicating a second-level cache hit. The abbreviated results corresponding to table1~1~200~200 are read from the second-level cache and returned as the HTTP message body. The original image timestamp (T1) is returned as the HTTP last-modified-time header. The user terminal caches the corresponding processing results, achieving... Figure 3 The user terminal cache status is 305.

[0056] Step 8: Handling scenarios where cache and original image changes are kept synchronized in real time: When a user modifies the original image corresponding to table1~1, the timestamp of the original image corresponding to table1~1 in the object storage system is modified to T2, such as... Figure 3 As shown in object storage system state 307, a user initiates a request to read the same thumbnails of tables 1-1 (thumbnail width 200, thumbnail height 200). The terminal cache update judgment module reads the HTTP protocol header information If-Modified-Since, obtains that the user's terminal cache version is T1, and then reads the original timestamp of tables 1-1 from the object storage system as T2. Comparing the two, an inconsistency is found, as follows: Figure 3 As shown in user terminal cache status 305 and object storage system status 307, the user terminal cache is deemed invalid. Further reading of the original timestamps from tables 1-1 in the secondary cache yields T1; a comparison reveals an inconsistency between the two. Figure 3 As shown in state 306 of the secondary cache and state 307 of the object storage system, the secondary cache is deemed invalid. The real-time image processing procedure described in step 5 then proceeds. After processing is complete, the user terminal receives the T2 version of the processing result, and both the user terminal cache and the secondary cache are updated, as shown below. Figure 3 The user terminal cache status 308 and the secondary cache status 309 are shown in the diagram.

[0057] This concludes the description of the exemplary process of the method of the present invention.

[0058] like Figure 4The diagram shown is a system architecture diagram of a real-time image processing method using the HTTP protocol in an embodiment of the present invention. The system architecture specifically includes four main components: an HTTP image request load balancer, a real-time image processing cluster, a secondary cache system, and an object storage system.

[0059] The HTTP image request load balancer is used to receive website user requests and distribute them to the worker nodes in the real-time image processing cluster according to a certain strategy. Since the system of this invention uses the HTTP protocol to connect requests, Nginx software can be preferred as the load balancer.

[0060] The aforementioned real-time image processing cluster comprises multiple independent real-time image processing nodes. Each node includes a URL parsing module, a cache processing module, a terminal cache update determination module, and an image processing engine, all connected to a request processing controller. The request processing controller is responsible for scheduling the entire image request processing process, including the entire control process from receiving the request to returning the result. The URL parsing module is responsible for parsing the image request URL into image-related parameters that the system can recognize. The cache processing module connects to the secondary cache system and encapsulates the logic related to image processing result caching within the system, including writing, evicting, and updating cached results. The terminal cache update determination module is responsible for determining whether the images cached on the user's terminal (e.g., browser-cached images) need to be updated based on the terminal information provided in the image request. The image processing engine is responsible for the specific calculations of the image request, such as image thumbnail calculation and image format conversion calculation; ImageMagick software is preferred as the image processing engine.

[0061] The secondary caching system is used to cache the real-time image processing results; it is also used to cache the original image upload timestamp, so that the real-time image processing node can determine whether to refresh the cache based on the timestamp; Redis software is preferred as the external secondary caching system.

[0062] The object storage system is used to store the original image and also to store metadata such as the upload timestamp of the original image; the Minio object storage system is preferred as the object storage system in this invention.

[0063] The main beneficial effects achieved by this invention are as follows: it provides a real-time image processing method and system based on the HTTP protocol.

[0064] This invention interfaces with an external secondary cache system, which can cache real-time image processing results to the maximum extent, reducing the CPU consumption and average response latency of the system's real-time processing. Since multiple worker nodes in the image processing cluster share the same secondary cache system, the problems of repeated image processing and repeated caching can be avoided.

[0065] This invention holds the original image upload timestamp in the image processing cache, thereby achieving real-time synchronization between the secondary cache results and the original image in the object storage system, ensuring that the image processing results always originate from the latest original image.

[0066] This invention can quickly determine whether to return an unmodified code (i.e., 304) based on the image timestamp contained in the terminal cache (e.g., browser), reducing repeated requests for unmodified images by the terminal and reducing server pressure; at the same time, the server ensures that the user terminal cache is always based on the latest original image by verifying the terminal cache timestamp, avoiding the problem of user terminal cache updates not being updated in real time interfering with user experience.

[0067] This invention supports the parsing of different business URLs in the form of plugins. Image requests from different domains can select the appropriate URL parsing plugin based on the domain name. This allows different business applications to customize URL parsing logic according to their own characteristics, so as to achieve the goal of supporting image processing of different business websites with a single image service, without intruding on the URL definition of the business websites.

[0068] The above embodiments are not intended to limit the present invention in any way. Any other improvements and applications made to the above embodiments by equivalent transformations shall fall within the protection scope of the present invention.

Claims

1. A picture real-time processing method of HTTP protocol, characterized in that, The method comprises the following steps: Step 1: preset object storage system, support to upload the original picture unique identifier and original picture blob given at the time, support to read the original picture blob according to the original picture unique identifier; support the storage of the meta information of the object, support to read the meta information according to the original picture unique identifier, the user gives the original picture unique identifier and the original picture blob through the preset object storage system own client, uploads the original picture to the object storage cluster, and writes the version identifier of the original picture into the meta information; Step 2: preset HTTP request load balancer, support to distribute picture access request to the working node in the picture real-time processing cluster according to the specified strategy; Step 3: the request processing controller in the working node in the picture real-time processing cluster receives the picture access request, calls the URL parsing module to parse the URL of the picture access request into the original picture unique identifier and the picture processing target parameter; The URL parsing module finds the corresponding URL processing logic body according to the domain name contained in the URL of the picture access request, and further calls the first interface implemented in the logic body to obtain the original picture unique identifier and the picture processing target parameter corresponding to the current picture access request; Step 4: the request processing controller calls the preset terminal cache update judgment module to judge whether the cache of the terminal needs to be updated; the terminal cache update judgment module uses the original picture unique identifier to obtain the version identifier of the corresponding original picture from the object storage system; when the version identifier obtained from the object storage system is consistent with the terminal cache version identifier contained in the request, a 304 status code is directly returned, otherwise step 5 is entered; Step 5: preset secondary cache system, support to write and read cache data, the secondary cache system has the ability of cold data automatic evict and online horizontal expansion; the request processing controller calls the cache processing module to do cache result searching and return processing; The cache processing module uses the original picture unique identifier to read the current cache version identifier from the secondary cache, when the cache version identifier is consistent with the version identifier obtained from the object storage system in step 4, the original picture unique identifier and the picture processing target parameter are combined into a request unique identifier for the current picture request, and the request unique identifier is used to search the cache result of the current picture request in the secondary cache system, if the cache result exists, it is directly returned; Otherwise, step 6 is entered; Step 6: preset picture processing engine, the picture processing engine supports picture thumbnail, format conversion, rotation, watermark processing; The request processing controller uses the original picture unique identifier contained in the request to read the original picture and the version identifier from the object storage system, and further calls the picture processing engine to process the original picture into the picture specification and format specified by the request in real time; Step 7: the cache processing module uses the processing result in step 6 to write the original picture unique identifier and its cache version identifier pair, and the current request unique identifier and the information pair of the processing result into the secondary cache system; Step 8: the request processing controller returns the processing result in step 7 to the user terminal.

2. The picture real-time processing method of HTTP protocol according to claim 1, wherein, In the step 1, the original picture unique identifier and the original picture blob are stored in a key-value pair, and the version identifier is the upload timestamp of the picture; if the original picture is uploaded repeatedly using the same unique identifier, the last uploaded original picture and its version identifier are read.

3. The picture real-time processing method of HTTP protocol according to claim 2, wherein, In the step 2, the picture real-time processing cluster includes a plurality of picture real-time processing work nodes, each of which accesses an HTTP picture processing request to realize real-time processing pressure sharing and system high availability in cooperation with an HTTP request load balancer.

4. The picture real-time processing method of HTTP protocol according to claim 3, wherein, In the step 3, the URL parsing module formulates a set of URL parsing plug-in APIs, including two interfaces, the first interface being used for converting a URL into picture processing target parameters, and the second interface being used for returning an interface corresponding to the domain name of the plug-in; the URL parsing module is responsible for loading the specific implementation of the URL parsing plug-in API implemented by the user, and pairing the business domain name with the plug-in processing logic body, supporting obtaining the corresponding URL processing logic body according to the business domain name.

5. The picture real-time processing method of HTTP protocol according to claim 4, wherein, In the step 4, the cache update determination module uses the original picture unique identifier to obtain only the latest version identifier of the original picture from the object storage system, and when the version identifier is consistent with the terminal cache version identifier included in the request, a 304 status code is directly returned.

6. The picture real-time processing method of HTTP protocol according to claim 5, wherein, In the step 5, the cache processing module uses the original picture unique identifier to read the picture version identifier in the cache from the secondary cache, and when the cached version identifier is consistent with the latest version identifier obtained from the object storage system in the step 4, the original picture unique identifier and the picture processing target parameters are combined into a request unique identifier for the current picture request, and the request unique identifier is used to search for the cache result of the current picture request in the secondary cache system, and if the cache result exists, the cache result is directly returned.

7. The picture real-time processing method of HTTP protocol according to claim 6, wherein, In the step 6, the picture processing engine is built into the picture real-time processing work node and works in the same process as other modules in the picture real-time processing work node; the request processing controller calls the corresponding interface of the picture processing engine according to the picture processing target parameters obtained by the URL parsing module, and the picture processing target parameters include target picture specification size, picture conversion format, and picture rotation angle.

8. The picture real-time processing method of HTTP protocol according to claim 7, wherein, In the step 7, the cache writing sequence is: first, write the information pair of the current request unique identifier and the processing result, and then write the pair of the original picture unique identifier and its cache version identifier.

9. A picture real-time processing system of the HTTP protocol operating according to the method of one of claims 1 to 8, characterized in that, The system comprises an HTTP request load balancer, a picture real-time processing cluster, a secondary cache system, and an object storage system. The HTTP request load balancer is used to receive website user requests and distribute the requests to the work nodes in the picture real-time processing cluster according to a round-robin strategy. The picture real-time processing cluster includes more than one independent picture real-time processing node, and each processing node includes a request processing controller, a URL parsing module, a cache processing module, a terminal cache update determination module, and a picture processing engine. The request processing controller is responsible for the process scheduling of picture requests, including the control process from receiving a request to returning a result. The URL parsing module is responsible for parsing the picture request URL into picture-related parameters recognizable by the system; The cache processing module encapsulates the logic related to the cache processing of the picture processing result, including the write, eviction, and update operations of the cache result; The terminal cache update determination module is responsible for determining whether the user terminal cache needs to be updated according to the terminal information provided by the picture request; The picture processing engine is responsible for the calculation of the picture request, including picture thumbnail calculation and picture format conversion calculation; The secondary cache system is used to cache the real-time processing results of pictures and the cache version identifiers of original pictures, so that the picture real-time processing node can determine whether to refresh the cache according to the cache version identifier; The object storage system is used to store the original pictures and the original picture version identifier meta information.

10. The picture real-time processing system of HTTP protocol as claimed in claim 9, wherein, The HTTP request load balancer is Nginx software; the secondary cache system is Redis software; the object storage system is Minio software; and the picture processing engine is ImageMagick software.

Images