Container mirror image acceleration method, system and device

Abstract

The invention provides a container mirror image acceleration method, system and device, and belongs to the technical field of container mirror image acceleration. The method comprises the following steps: controlling a data center to load and store mirror image files in a mirror image warehouse; under the condition that the control data center receives a mirror image updating request sent by the fusion terminal, the control data center sends a mirror image file to all the master stations which have established communication connection; controlling each master station to receive and store the mirror image file; controlling each master station to send a mirror image file to all the fusion terminals which are in communication connection and are not subjected to mirror image updating under the condition of receiving mirror image updating requests sent by the other fusion terminals which are not subjected to mirror image updating; and controlling each fusion terminal to store the mirror image file. According to the invention, the defects that the existing power distribution Internet of Things equipment is usually directly distributed to each equipment through a central mirror image warehouse, but the speed is slow when a large amount of data is distributed, the overload of the central mirror image warehouse is easily caused, and the overall performance of the system is influenced are overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of container image acceleration, and particularly to a container image acceleration method, a container image acceleration system, a container image acceleration device, a machine-readable storage medium, and a computer program product. Background Art

[0002] With the construction of the distribution Internet of Things, millions of distribution devices are connected to the distribution network. The distribution network interconnects these devices through a dedicated network to complete important tasks such as operation monitoring, data calculation, and operation and maintenance control. Many of these distribution devices use Docker technology to simplify the software deployment and update processes. The core advantage of Docker technology is that the device only needs to build the image once, and all devices that need the image can use it, greatly improving the deployment efficiency.

[0003] Existing distribution Internet of Things devices usually directly distribute images to each device through a central image repository. However, this solution is slow in distributing a large amount of data and is prone to overloading the central image repository, affecting the overall performance of the system. Therefore, there is an urgent need for a more efficient image distribution solution to reduce the load on the central image repository, improve the image update speed, and optimize the upgrade experience of devices in the distribution network. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a container image acceleration method, system, and device to solve the defect that existing distribution Internet of Things devices usually directly distribute images to each device through a central image repository, but this solution is slow in distributing a large amount of data and is prone to overloading the central image repository, affecting the overall performance of the system.

[0005] To achieve the above purpose, the embodiments of the present invention provide a container image acceleration method, including:

[0006] Controlling the data center to load and store the image files in the image repository;

[0007] Controlling the data center to send the image files to all master stations that have established a communication connection when receiving an image update request sent by a fusion terminal;

[0008] Controlling each master station to receive and store the image files;

[0009] Controlling each master station to send the image files to all the non-image-updated fusion terminals that have established a communication connection when receiving an image update request sent by the remaining non-image-updated fusion terminals;

[0010] Controlling each fusion terminal to store the image files.

[0011] Optionally, after controlling each of the fusion terminals to store the mirror file, the method further includes:

[0012] Controlling each of the fusion terminals to perform file integrity verification on the mirror file;

[0013] When the verification passes, controlling each of the fusion terminals to send a mirror update completion notification to the corresponding master station with which a communication connection is established.

[0014] Optionally, after controlling each of the fusion terminals to send a mirror update completion notification to the corresponding master station with which a communication connection is established, the method further includes:

[0015] Controlling each master station to send an update time instruction to all the fusion terminals with which a communication connection has been established; the update time instruction includes the mirror version and the effective time of the mirror file;

[0016] Controlling each master station to send a unified start instruction to all the fusion terminals with which a communication connection has been established, so that all the fusion terminals perform unified start of the mirror file based on the time stamp of the effective time.

[0017] Optionally, the method further includes:

[0018] In the case where the mirror version of one of the fusion terminals is inconsistent with the mirror versions of the other fusion terminals, the steps of controlling each master station to send an update time instruction to all the fusion terminals with which a communication connection has been established and the step of controlling each master station to send a unified start instruction to all the fusion terminals with which a communication connection has been established are executed again until the mirror versions of all the fusion terminals are consistent.

[0019] Optionally, the method further includes:

[0020] During the mirror start process, controlling each master station to adopt a resource control strategy so that the resource usage of all the fusion terminals with which a communication connection has been established is less than a set resource quota.

[0021] Optionally, the method further includes:

[0022] In the case where the mirror start time of the fusion terminal exceeds a set time threshold, controlling the fusion terminal to trigger a rollback operation.

[0023] Optionally, the mirror request time at which the fusion terminal sends the mirror update request to the data center is calculated based on a set communication acceleration ratio, the number of fusion terminals, the number of master stations, the depth of the distribution tree from the fusion terminal to the data center, and the Https request time of a single node; wherein, the set communication acceleration ratio represents the communication acceleration ratio between the communication mode between the data center and the master station and the communication mode between the master station and the fusion terminal.

[0024] Optionally, the mirror request time is calculated by the following formula:

[0025] T2 = N * a * t + n * (k - 1) * t;

[0026] Wherein, N represents the number of fusion terminals, n represents the number of master stations, k is the depth of the distribution tree from the fusion terminal to the data center, t is the Https request time of a single node, T2 represents the mirror request time, and a represents the set communication acceleration ratio.

[0027] On the other hand, an embodiment of the present invention further provides a container image acceleration device, including:

[0028] A first control module, configured to control the data center to load and store the mirror files in the mirror repository;

[0029] A second control module, configured to control the data center to send the mirror files to all the master stations with which communication connections have been established when receiving a mirror update request sent by a fusion terminal;

[0030] A third control module, configured to control each master station to receive and store the mirror files;

[0031] A fourth control module, configured to control each master station to send the mirror files to all the non-mirrored updated fusion terminals with which communication connections have been established when receiving a mirror update request sent by the remaining non-mirrored updated fusion terminals;

[0032] A fifth control module, configured to control each fusion terminal to store the mirror files.

[0033] Optionally, the device further includes:

[0034] A sixth control module, configured to control each fusion terminal to perform file integrity verification on the mirror files;

[0035] A seventh control module, configured to control each fusion terminal to send a mirror update completion notification to the corresponding master station with which a communication connection has been established when the verification is passed.

[0036] Optionally, the device further includes:

[0037] The eighth control module is used to control each master station to send an update time instruction to all the fusion terminals with which communication connections have been established; the update time instruction includes the mirror version and the effective time of the mirror file;

[0038] The ninth control module is used to control each master station to send a unified start instruction to all the fusion terminals with which communication connections have been established, so that all the fusion terminals perform unified start of the mirror file based on the time stamp of the effective time.

[0039] Optionally, the device further includes:

[0040] A retry module is used to, when the mirror version of one of the fusion terminals is inconsistent with the mirror versions of the other fusion terminals, execute the eighth control module and the ninth control module again until the mirror versions of all the fusion terminals are consistent.

[0041] Optionally, the device further includes:

[0042] The tenth control module is used to, during the mirror start process, control each master station to adopt a resource control strategy so that the resource usage amounts of all the fusion terminals with which communication connections have been established are less than the set resource quota.

[0043] Optionally, the device further includes:

[0044] The eleventh control module is used to, when the mirror start time of the fusion terminal exceeds the set time threshold, control the fusion terminal to trigger a rollback operation.

[0045] Optionally, the mirror request time for the fusion terminal to send the mirror update request to the data center is calculated based on the set communication acceleration ratio, the number of fusion terminals, the number of master stations, the depth of the distribution tree from the fusion terminal to the data center, and the Https request time of a single node; wherein, the set communication acceleration ratio represents the communication acceleration ratio between the communication mode between the data center and the master station and the communication mode between the master station and the fusion terminal.

[0046] Optionally, the mirror request time is calculated by the following formula:

[0047] T2 = N * a * t + n * (k - 1) * t;

[0048] Wherein, N represents the number of fusion terminals, n represents the number of master stations, k is the depth of the distribution tree from the fusion terminal to the data center, t is the Https request time of a single node, T2 represents the mirror request time, and a represents the set communication acceleration ratio.

[0049] On the other hand, an embodiment of the present invention further provides a container image acceleration system, including:

[0050] A data center, configured to load and store image files in an image repository and send the image files to all master stations with which communication connections have been established when receiving an image update request sent by a converged terminal;

[0051] A plurality of master stations, each of the master stations being communicatively connected to the data center, each master station being configured to receive and store the image files, and send the image files to all the converged terminals that have not been updated with the image when receiving an image update request sent by the remaining converged terminals that have not been updated with the image;

[0052] A plurality of converged terminals, each of the master stations being communicatively connected to at least one of the converged terminals, each converged terminal being configured to store the image files.

[0053] On the other hand, the present invention further provides a machine-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned container image acceleration method is implemented.

[0054] On the other hand, the present invention further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above-mentioned container image acceleration method is implemented.

[0055] Through the above technical solutions, in the embodiment of the present invention, the data center of the distribution network is used as the root node, and the master stations and terminal devices are used as branch nodes. When there is an image update request, the image repository of the data center first transmits the image files to each master station node, and each master station node then distributes the image files to the subordinate converged terminals through the internal network, thereby forming a step-by-step distribution method, reducing the pressure on the central image repository, improving the distribution efficiency of the image files, optimizing the upgrade experience of the devices in the distribution network, and reducing the impact on the overall performance of the system.

[0056] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the specification. They are used to explain the embodiments of the present invention together with the following specific implementation manners, but do not constitute a limitation to the embodiments of the present invention. In the drawings:

[0058] Figure 1 is one of the flow schematic diagrams of the container image acceleration method provided by the present invention;

[0059] Figure 2 is the structural schematic diagram of the Internet of Things network topology provided by the present invention;

[0060] Figure 3 It is the second flowchart of the container image acceleration method provided by the present invention;

[0061] Figure 4 It is a schematic diagram of the comparison of the image request times of the image update requests before and after optimization provided by the present invention;

[0062] Figure 5 It is a schematic structural diagram of the container image acceleration device provided by the present invention;

[0063] Figure 6 It is a schematic structural diagram of the container image acceleration system provided by the present invention. Detailed implementation manners

[0064] The following will describe in detail the specific implementation manners of the embodiments of the present invention with reference to the accompanying drawings. It should be understood that the specific implementation manners described herein are only for explaining and illustrating the embodiments of the present invention, and are not used to limit the embodiments of the present invention.

[0065] The image distribution of the current power distribution Internet of Things system depends on the direct distribution of the central image repository. When a large number of devices update images simultaneously, the central repository bears huge traffic and processing pressures, which is extremely likely to cause the distribution speed to slow down and affect the device update efficiency. On the other hand, gray-scale upgrades will also result in differences in software versions in the terminals, leading to different data processing logics, and the upgrade process is time-consuming and laborious. The traditional image distribution method ignores the tree-like structure of Internet of Things devices and fails to effectively utilize the main stations at all levels to share the image distribution burden. Therefore, there are the following disadvantages:

[0066] 1. Distribution delay: The direct distribution method is likely to cause high load on the image repository, resulting in reduced distribution efficiency, increasing the upgrade waiting time of devices, and affecting system operation.

[0067] 2. Bandwidth occupation: When all devices request the central image repository simultaneously, the network bandwidth and the load of the repository will increase sharply, affecting the normal operation of other system tasks.

[0068] 3. Resource waste: The traditional distribution method fails to effectively utilize the network and computing resources of the main stations and fails to achieve hierarchical sharing of the distribution load, resulting in low resource utilization.

[0069] In view of this, the embodiments of the present invention aim to provide a container image acceleration method, system and device to solve the defect that existing power distribution Internet of Things devices are usually directly distributed to each device through a central image repository, but this solution is slow in distributing a large amount of data and is likely to cause overload of the central image repository, affecting the overall performance of the system.

[0070] Method embodiment

[0071] Please refer to Figure 1 , an embodiment of the present invention provides a method for accelerating container images, including:

[0072] Step 100, control the data center to load and store the image files in the image repository.

[0073] Step 200, when the data center receives an image update request sent by the fusion terminal, send the image file to all master stations that have established a communication connection.

[0074] First, an embodiment of the present invention provides a hierarchical heterogeneous network architecture for realizing fast distribution of container images, fast deployment of fusion terminals (referred to as terminals for short), and consistency control. The hierarchical heterogeneous network architecture is an Internet of Things network topology structure. In the embodiment of the present invention, the Internet of Things network topology structure takes a three-layer tree network topology structure as an example, and the Internet of Things network topology structure is as Figure 2 shown. It should be noted that the actual network topology structure has up to ten layers, and the actual Internet of Things network topology structure is simplified here for illustrative purposes.

[0075] The Internet of Things network topology structure design in the embodiment of the present invention adopts a hierarchical tree-like distribution method. The core node is the data center, and the data center is connected to the master stations in each region downward. The master stations manage the fusion terminals in multiple regions. This architecture communicates efficiently between the data center and the master stations through the HTTPS (Hypertext Transfer Protocol Secure) protocol, and communicates efficiently between the master stations and the fusion terminals through the RPC (Remote Procedure Call) protocol. Thus, the Internet of Things network topology structure in the embodiment of the present invention communicates efficiently between each layer through the HTTPS protocol and the RPC protocol, supports data storage acceleration and compression, minimizes bandwidth usage, and improves the image update speed.

[0076] The embodiment of the present invention performs unified management of image updates through the image upgrade management module in the data center. Please refer to Figure 3 , control the data center to start the image update process. First, perform hot loading of the image files in the image repository, and when the loading is completed, cache the image files in the storage corresponding to the memory type. When receiving an image update request sent by the fusion terminal, distribute the image files to the master stations in each region through the HTTPS protocol. The HTTPS protocol ensures the security of data transmission and encrypts the transmitted data to prevent the data from being tampered with midway.

[0077] Step 300, control each of the master stations to receive and store the image file.

[0078] Step 400: Control each of the master stations to send the mirror file to all the un - mirrored updated fusion terminals with which communication connections have been established when receiving a mirror update request sent by the remaining un - mirrored updated fusion terminals.

[0079] Control the master station to hierarchically store the mirror image through a distributed cache system (such as Redis or Ceph) after receiving the mirror file, automatically detect the latest version, and provide cache hit and invalidation management. When each master station receives a mirror update request sent by the remaining un - mirrored updated fusion terminals in the hierarchical tree - shaped topology, the master station interacts with the fusion terminal using RPC communication according to the cache status. In the case of cache hit, RPC communication is specifically implemented using gRPC to achieve low - latency transmission, and gradually send the mirror file to each fusion terminal.

[0080] Step 500: Control each of the fusion terminals to store the mirror file.

[0081] Control the fusion terminal to store the mirror file on the local disk after receiving it. In the embodiment of the present invention, when the data center loads the mirror file, it does not send the mirror file to the master station. Instead, when the fusion terminal sends a mirror update request to the data center, the mirror update request is sent to the master station. If the master station does not have the corresponding mirror file, the mirror update request will be sent to the data center, so that the first fusion terminal obtains the mirror file. At the same time, the mirror file in the data center is cached in the master station. For subsequent mirror update requests to the master station, since the master station has the corresponding mirror file, the mirror update request does not need to be sent to the data center.

[0082] Thus, in the embodiment of the present invention, the data center of the distribution network is used as the root node, and the master stations and fusion terminals are used as branch nodes. When starting the mirror update process, the mirror repository in the data center first transmits the mirror file to each master station node, and each master station node then distributes the mirror file to its subordinate fusion terminals through the internal network, thus forming a progressive distribution method, reducing the pressure on the central mirror repository, improving the distribution efficiency of the mirror file, optimizing the upgrade experience of the devices in the distribution network, and reducing the impact on the overall system performance.

[0083] In other aspects of the embodiment of the present invention, after step 500: Control each of the fusion terminals to store the mirror file, it further includes:

[0084] Step 600: Control each of the fusion terminals to perform file integrity verification on the mirror file.

[0085] Step 700: When the verification passes, control each of the fusion terminals to send a mirror update completion notification to the corresponding master station with which a communication connection has been established.

[0086] After controlling each of the fusion terminals to store the mirror file, control each fusion terminal to perform file integrity verification on the received mirror file to ensure that each fusion terminal successfully receives a complete mirror file. Among them, the file integrity verification can adopt any one of the MD5 algorithm, the SHA-256 algorithm, and the CRC32 algorithm. In the case of passing the verification, control each of the fusion terminals to send a mirror update completion notification to the corresponding master station established for communication connection. Thus, each of the fusion terminals informs the corresponding master station of the received mirror file and is ready to start the mirror file startup process, so as to prepare for the unified startup of all fusion terminals.

[0087] It should be noted that if the verification fails, the re-download process is triggered. The data center starts the mirror update process again.

[0088] In other aspects of the embodiments of the present invention, the embodiments of the present invention implement a multi-way tree calculation model in the fusion terminal hierarchy to improve the mirror distribution efficiency. Please refer to Figure 4 , assuming that there are a total of N fusion terminals, if the mirror update request from the fusion terminal to the data center needs to pass through k routes, the depth of the tree is k, and the request time of a single node for Https is t, then the total time of the mirror update request before optimization:

[0089] T1 = N * k * t;

[0090] The total time of the mirror update request after optimization is calculated based on the set communication acceleration ratio, the number of fusion terminals, the number of master stations, the depth of the distribution tree from the fusion terminal to the data center, and the request time of a single node for Https; wherein, the set communication acceleration ratio characterizes the communication acceleration ratio between the communication method between the data center and the master station and the communication method between the master station and the fusion terminal. In one implementation, the communication method between the data center and the master station is HTTPS protocol communication, and the communication method between the master station and the fusion terminal is gRPC protocol communication. Thus, the mirror request time is calculated by the following formula:

[0091] T2 = N * a * t + n * (k - 1) * t;

[0092] Among them, N represents the number of fusion terminals, n represents the number of master stations, k is the depth of the distribution tree from the fusion terminal to the data center, t is the request time of a single node for Https, T2 represents the mirror request time, and a represents the set communication acceleration ratio, that is, the communication acceleration ratio between HTTPs and gRPC.

[0093] Without considering network congestion caused by excessive request bandwidth, and when the number of fusion terminals N is much larger than the number of master stations n, the efficiency of the distributed tree acceleration algorithm is as follows:

[0094] x = T2 / T1 = (N * a * t + n * (k - 1) * t) / (N * k * t) ≈ a / k。

[0095] It can be seen that by implementing the multi - fork tree calculation model on the fusion terminal hierarchy, the mirror distribution efficiency is improved in the embodiments of the present invention.

[0096] In other aspects of the embodiments of the present invention, after step 700, controlling each of the fusion terminals to send a mirror update completion notification to the corresponding master station that has established a communication connection, the following steps are further included:

[0097] Step 800, controlling each master station to send an update time instruction to all the fusion terminals that have established a communication connection; the update time instruction includes the mirror version and the effective time of the mirror file.

[0098] Step 900, controlling each master station to send a unified start instruction to all the fusion terminals that have established a communication connection, so that all the fusion terminals start the mirror file uniformly based on the time stamp of the effective time.

[0099] In the traditional method, the mirror start of the fusion terminal mostly starts immediately after pulling the mirror file. In order to ensure the consistency of the mirror update of the fusion terminal in the embodiments of the present invention, a unified mirror start method is designed to achieve the synchronous update of all the fusion terminals in the network.

[0100] First, control each master station to send an update time instruction to all the fusion terminals that have established a communication connection. The update time instruction includes the mirror version and the effective time of the mirror file. The mirror version of the mirror file is used to distinguish whether the mirror files of different fusion terminals are consistent. The effective time is used to indicate the start time of the mirror file of the fusion terminal. When all the fusion terminals receive the update time instruction, control each master station to send a unified start instruction to all the fusion terminals that have established a communication connection, so that all the fusion terminals start the mirror file uniformly based on the time stamp of the effective time, thereby realizing the synchronization of the mirror start of all the fusion terminals.

[0101] In the existing gray-scale upgrade, some Internet of Things devices are upgraded first, and then another batch of Internet of Things devices are controlled to be upgraded. For example, in North China Region 1, the Docker software system in the fusion terminal is upgraded first, and then in North China Region 2, the Docker software system in the fusion terminal is upgraded. By means of time-sharing upgrade, the request pressure on the Docker image repository is reduced. However, there is still a problem, that is, the upgrade process is complex, bringing a great operation and maintenance pressure; on the other hand, due to the inconsistent order of mirror upgrades, there will be new and old differences in the results of software processing. The fusion terminal with the earlier mirror upgrade processes data according to the logic of the new software and algorithm, while the fusion terminal with the later mirror upgrade still processes problems according to the previous logic. In the embodiment of the present invention, by controlling each master station to send an update time instruction and a unified start instruction to all the fusion terminals with which communication connections have been established, synchronous update of the whole network terminals is realized, thus avoiding the problems that occur in the existing gray-scale upgrade.

[0102] In other aspects of the embodiment of the present invention, the container image acceleration method of the embodiment of the present invention further includes:

[0103] Step 1000, in the case that the mirror version of one of the fusion terminals is inconsistent with the mirror versions of other fusion terminals, the steps of controlling each master station to send an update time instruction to all the fusion terminals with which communication connections have been established and the step of controlling each master station to send a unified start instruction to all the fusion terminals with which communication connections have been established are executed again until the mirror versions of all the fusion terminals are consistent. It should be noted that step 1000 can be set after step 900.

[0104] In the case that the mirror version of one of the fusion terminals is inconsistent with the mirror versions of other fusion terminals, steps 800 and 900 are executed again, and the unified start of the mirror file is re-executed, so as to solve the out-of-sync or delay situation existing in the mirror file in the current fusion terminal and ensure that the mirror versions of all fusion terminals are consistent. Thus, the problem that the inconsistent order of mirror upgrades in the existing gray-scale upgrade leads to new and old differences in the results of software processing is avoided.

[0105] Thus, in the embodiments of the present invention, the data center uniformly distributes mirror update requests through the registration center of the fusion terminal. After the mirror update is completed, each master station will trigger the automatic mirror startup of the fusion terminal to achieve seamless connection between mirror file distribution and task startup, ensuring that the fusion terminal loads and executes the new mirror file in the shortest time. In view of the differences in the hardware architectures and software environments of heterogeneous devices in the power distribution Internet of Things, the embodiments of the present invention propose to deploy mirrors through container technology to shield device differences. By constructing the mirror repository of the data center as a compatible containerized mirror, each fusion terminal completes the consistency of the software environment by automatically pulling and running the mirror, ensuring that various fusion terminals can accurately and efficiently receive and execute the tasks issued by the master station.

[0106] In other aspects of the embodiments of the present invention, the container image acceleration method of the embodiments of the present invention further includes:

[0107] Step 1100, during the mirror startup process, control each master station to adopt a resource control policy so that the resource usage of all the fusion terminals that have established communication connections is less than the set resource quota. It should be noted that step 1100 can be set after step 900 or step 1000.

[0108] Since the docker technology itself supports resource limitations, the embodiments of the present invention can issue the set resource quota of each fusion terminal through the mirror upgrade management module of the data center. Among them, the set resource quota can include CPU usage time quota, memory usage quota, disk IO bandwidth quota, etc. These set resource quotas can be dynamically adjusted by relevant technical personnel according to factors such as the actual needs of the fusion terminal, business importance, and priority.

[0109] During the mirror startup process, when the resource usage of the fusion terminal reaches or exceeds its set resource quota, the master station can take measures such as restricting the resource usage of the fusion terminal, warning the fusion terminal user, or forcibly disconnecting the connection. Thus, the effect of adopting a resource control policy so that the resource usage of all the fusion terminals that have established communication connections is less than the set resource quota is achieved. Furthermore, it is realized to prevent the mirror update process from affecting other tasks running on the terminal.

[0110] In other aspects of the embodiments of the present invention, the container image acceleration method of the embodiments of the present invention further includes:

[0111] Step 1200, in the case where the mirror startup time of the fusion terminal exceeds the set time threshold, control the fusion terminal to trigger a rollback operation. It should be noted that step 1200 can be set after step 900, step 1000, or step 1100.

[0112] The present invention also designs a timeout protection mechanism. When the mirror startup time of the fusion terminal exceeds the set time threshold, the fusion terminal is controlled to trigger a rollback operation, that is, the fusion terminal is controlled to return to the previous mirror version to ensure the stability of the fusion terminal.

[0113] In addition, in order to improve the speed and bandwidth utilization rate during the mirror distribution process, the embodiments of the present invention design a variety of mirror distribution acceleration methods:

[0114] 1. Hierarchical caching and mirror prefetching: By deploying a distributed cache at the master station end, when a mirror update is triggered, the cache system will preferentially obtain the mirror file and warm it up to the storage location of the fusion terminal, reducing the network latency during each mirror file pull.

[0115] 2. HTTPS protocol optimization: Use the HTTPS protocol to transmit the mirror between the data center and the master station, and at the same time enable gzip compression to improve data transmission efficiency and reduce bandwidth consumption. In addition, the data center uses parallel stream transmission technology to distribute multiple sub-streams to improve the distribution efficiency.

[0116] 3. RPC acceleration and chunked transmission: Use gRPC between the master station and the fusion terminal to implement remote calls, accelerate the transmission based on RPC, and implement chunked transmission (chunking) of data during the transmission process. Each piece of data is compressed and CRC-checked when sent to ensure data integrity.

[0117] 4. Mirror acceleration scheduling algorithm: Sort the requests of the fusion terminal through a dynamic scheduling algorithm to maximize resource utilization. For example, the bandwidth resources of the fusion terminal can be allocated on demand through the Max Throughput Scheduling algorithm to optimize the mirror pull speed of each fusion terminal.

[0118] The embodiments of the present invention realize the fast distribution and parallel deployment of container images through hierarchical caching, chunked transmission, and unified startup mechanism, improve the overall efficiency of the distribution network system, and reduce the maintenance cost and bandwidth overhead.

[0119] In summary, the container image acceleration of the embodiments of the present invention has the following advantages:

[0120] 1. Improve mirror distribution efficiency: Through the multi-level mirror distribution method of the tree structure, the bottleneck problem of the central warehouse is avoided, the mirror distribution speed is greatly improved, and the mirror distribution efficiency can reach up to 87%, meeting the high-frequency mirror update requirements of the distribution Internet of Things.

[0121] 2. Reduce bandwidth and system resource consumption: Each master station reduces the bandwidth occupancy and distribution resource consumption of the data center by uniformly managing and sharing the mirror request traffic. At the same time, the integrated terminal can directly obtain mirror updates from the local master station, greatly reducing network latency and improving the stability of mirror distribution.

[0122] 3. Adaptability and management of heterogeneous devices: In the embodiments of the present invention, the mirror is run in the integrated terminal through container technology, effectively shielding the hardware and software differences of the devices and realizing cross-platform mirror compatibility. This method not only ensures the mirror consistency of all devices, but also simplifies the mirror update operation of the devices and improves the management efficiency of the terminal devices.

[0123] Device embodiments

[0124] Please refer to Figure 5 On the other hand, the embodiments of the present invention also provide a container mirror acceleration device, including:

[0125] The first control module 501 is used to control the data center to load and store the mirror files in the mirror repository;

[0126] The second control module 502 is used to control the data center to send the mirror file to all the master stations that have established a communication connection when receiving a mirror update request sent by the integrated terminal;

[0127] The third control module 503 is used to control each of the master stations to receive and store the mirror file;

[0128] The fourth control module 504 is used to control each of the master stations to send the mirror file to all the integrated terminals that have not been updated with the mirror when receiving a mirror update request sent by the remaining integrated terminals that have not been updated with the mirror;

[0129] The fifth control module 505 is used to control each of the integrated terminals to store the mirror file.

[0130] In the embodiments of the present invention, the data center of the distribution network is used as the root node, and the master stations and terminal devices are used as branch nodes. When there is a mirror update request, the mirror repository of the data center first transmits the mirror file to each master station node, and each master station node then distributes the mirror file to its subordinate integrated terminals through the internal network, thus forming a progressive distribution method, reducing the pressure on the central mirror repository, improving the mirror file distribution efficiency, optimizing the upgrade experience of the devices in the distribution network, and reducing the impact on the overall system performance.

[0131] Optionally, the device further includes:

[0132] The sixth control module is used to control each of the integrated terminals to perform file integrity verification on the mirror file;

[0133] The seventh control module is used to control each of the fusion terminals to send a mirror update completion notification to the corresponding master station with which a communication connection is established when the verification is passed.

[0134] Optionally, the device further includes:

[0135] The eighth control module is used to control each master station to send an update time instruction to all the fusion terminals with which a communication connection has been established; the update time instruction includes the mirror version and the effective time of the mirror file;

[0136] The ninth control module is used to control each master station to send a unified start instruction to all the fusion terminals with which a communication connection has been established, so that all the fusion terminals perform unified start of the mirror file based on the time stamp of the effective time.

[0137] Optionally, the device further includes:

[0138] The retry module is used to execute the eighth control module and the ninth control module again when the mirror version of one of the fusion terminals is inconsistent with the mirror versions of the other fusion terminals until the mirror versions of all the fusion terminals are consistent.

[0139] Optionally, the device further includes:

[0140] The tenth control module is used to control each master station to adopt a resource control strategy during the mirror start process so that the resource usage of all the fusion terminals with which a communication connection has been established is less than the set resource quota.

[0141] Optionally, the device further includes:

[0142] The eleventh control module is used to control the fusion terminal to trigger a rollback operation when the mirror start time of the fusion terminal exceeds the set time threshold.

[0143] Optionally, the mirror request time for the fusion terminal to send the mirror update request to the data center is calculated based on a set communication acceleration ratio, the number of fusion terminals, the number of master stations, the depth of the distribution tree from the fusion terminal to the data center, and the Https request time of a single node; wherein, the set communication acceleration ratio represents the communication acceleration ratio between the communication mode of the data center and the master station and the communication mode of the master station and the fusion terminal.

[0144] Optionally, the mirror request time is calculated by the following formula:

[0145] T2 = N * a * t + n * (k - 1) * t;

[0146] Wherein, N represents the number of fusion terminals, n represents the number of master stations, k is the depth of the distribution tree from the fusion terminal to the data center, t is the request time of Https for a single node, T2 represents the mirror request time, and a represents the set communication acceleration ratio.

[0147] The container image acceleration device includes a processor and a memory. The above first control module 501, second control module 502, third control module 503, fourth control module 504, and fifth control module 505 are all stored in the memory as program units, and the processor executes the above program units stored in the memory to implement corresponding functions.

[0148] The processor contains a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set.

[0149] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or forms such as non-volatile memory, such as read-only memory (ROM) or flash RAM (flash RAM), and the memory includes at least one storage chip.

[0150] Figure 6 Illustrated is a schematic diagram of the entity structure of a container image acceleration system, as Figure 6 shown, a container image acceleration system includes:

[0151] A data center 10, which is used to load and store the image files in the image repository when receiving a mirror update request sent by a fusion terminal 30, and send the image files to all master stations 20 that have established a communication connection. In the embodiment of the present invention, the data center 10 acts as an image source, distributes the base image and its updates to each master station 20, and manages the image version in real time according to the feedback of the master station 20. The data center 10 includes an image repository. The image repository stores all versions of the images, has a large storage capacity, and uses an SSD type hard disk for storage, saving storage costs; the data center 10 also includes an image reading center module, which reads from the image repository to a storage acceleration module, and the image storage acceleration module uses a storage technology based on DRAM type memory, including but not limited to distributed cache systems (such as Redis or Alluxio technology, etc.). The data center 10 also includes an image upgrade management module responsible for the unified upgrade management of the images and the management work of each module. An Https data transmission method is provided between the data center 10 and the master station 20 to ensure the security of data transmission.

[0152] Multiple master stations 20, each of the master stations 20 is communicatively connected to the data center 10. Each of the master stations 20 is configured to receive and store the mirror file, and in the case of receiving a mirror update request sent by the remaining un-mirrored updated fusion terminals 30, send the mirror file to all the un-mirrored updated fusion terminals 30 with which a communication connection has been established. The master station 20 acts as an intermediate node, responsible for receiving the mirror file from the data center 10 and transmitting it to the lower-level fusion terminals 30. To reduce the data transmission cost, the master station 20 is equipped with a mirror distribution module and a memory acceleration module. The memory acceleration module and the storage acceleration module in the data center 10 adopt the same acceleration processing device and processing logic. The mirror distribution module provides an RPC communication method for the fusion terminals 30, which belongs to the communication within the internal system and has a higher communication efficiency than the Https communication efficiency.

[0153] Multiple fusion terminals 30, each of the master stations 20 is communicatively connected to at least one of the fusion terminals 30. Each of the fusion terminals 30 is configured to store the mirror file. The fusion terminal 30 is responsible for pulling and storing the mirror file locally and serves as the final execution node of the mirror file. Each fusion terminal 30 receives update data from the master station 20 according to the actual configuration and starts or stops according to the command.

[0154] In the embodiment of the present invention, the mirror update is uniformly managed through the mirror upgrade management module in the data center 10. The data center 10 is controlled to start the mirror update process. First, the hot loading of the mirror file in the mirror repository is performed, and after the loading is completed, the mirror file is cached in the storage corresponding to the memory type. In the case of receiving a mirror update request sent by the fusion terminal 30, the mirror file is distributed to the master stations 20 in each region through the HTTPS protocol. The HTTPS protocol ensures the security of data transmission and encrypts the transmitted data to prevent the data from being tampered with midway.

[0155] After receiving the mirror file, the master station 20 is controlled to store the mirror in layers through a distributed cache system (such as Redis or Ceph), and automatically detect the latest version, providing cache hit and invalidation management. In the case of receiving a mirror update request sent by the remaining un-mirrored updated fusion terminals 30 in the hierarchical tree topology, the master station 20 interacts with the fusion terminal 30 using RPC communication according to the cache status. In the case of cache hit, to optimize data transmission. The RPC communication is specifically implemented using gRPC to achieve low-latency transmission, and the mirror file is gradually sent to each fusion terminal 30.

[0156] After the control fusion terminal 30 receives the mirror file, it stores it on the local disk. In the embodiment of the present invention, when the data center 10 loads the mirror file, it does not send the mirror file to the master station 20. Instead, when the fusion terminal 30 sends a mirror update request to the data center 10, the mirror update request is sent to the master station 20. If the master station 20 does not have the corresponding mirror file, the mirror update request will be sent to the data center 10. Thus, the first terminal obtains the mirror file, and at the same time, the mirror file in the data center 10 is cached in the master station 20. For subsequent mirror update requests to the master station 20, since the master station 20 has the corresponding mirror file, the mirror update request does not need to be sent to the data center 10.

[0157] In the embodiment of the present invention, the data center 10 of the distribution network is used as the root node, and the master station 20 and the terminal device are used as branch nodes. When there is a mirror update request, the mirror repository of the data center 10 first transmits the mirror file to each master station 20 node. Each master station 20 node then distributes the mirror file to its subordinate fusion terminal 30 through the internal network, thereby forming a progressive distribution method, reducing the pressure on the central mirror repository, improving the mirror file distribution efficiency, optimizing the upgrade experience of the devices in the distribution network, and reducing the impact on the overall system performance. By establishing a hierarchical distribution mechanism between the master station 20 and the terminal device, optimizing the mirror distribution path and efficiency, the mirror update can be quickly distributed among devices at each level, reducing the pressure on the central mirror repository, improving the mirror distribution speed. At the same time, the Internet of Things devices complete the update of the Docker container at the same time, reducing the processing differences brought to the container due to system upgrade, and finally realizing the efficient utilization of resources and the rapid upgrade of devices.

[0158] In addition, after controlling each of the fusion terminals 30 to store the mirror file, control each fusion terminal 30 to perform file integrity verification on the received mirror file to ensure that each fusion terminal 30 successfully receives a complete mirror file. Any one of the MD5 algorithm, SHA-256 algorithm, and CRC32 algorithm can be used for the file integrity verification. In the case of passing the verification, control each of the fusion terminals 30 to send a mirror update completion notification to the corresponding master station 20 that has established a communication connection. Thus, each of the fusion terminals 30 notifies the corresponding master station 20 of the received mirror file and is ready to start the mirror file, preparing for the unified start of all fusion terminals 30.

[0159] Control each master station 20 to send an update time instruction to all the fusion terminals 30 with which a communication connection has been established. The update time instruction includes the mirror version and the effective time of the mirror file. The mirror version of the mirror file is used to distinguish whether the mirror files of different fusion terminals 30 are consistent. The effective time is used to indicate the start time of the mirror file of the fusion terminal 30. When all the fusion terminals 30 receive the update time instruction, control each master station 20 to send a unified start instruction to all the fusion terminals 30 with which a communication connection has been established, so that all the fusion terminals 30 perform unified start of the mirror file based on the time stamp of the effective time, thereby realizing the mirror start synchronization of all the fusion terminals 30.

[0160] In the case where the mirror version of one of the fusion terminals 30 is inconsistent with the mirror versions of the other fusion terminals 30, re - execute the unified start of the mirror file again, thereby solving the out - of - sync or delay situation existing in the mirror files in the current fusion terminals 30, and ensuring that the mirror versions of all the fusion terminals 30 are consistent. Thus, it avoids the problem that in the existing gray - scale upgrade, the sequence of mirror upgrades is inconsistent, and there will be differences between the new and old software processing results.

[0161] During the mirror start process, when the resource usage of the fusion terminal 30 reaches or exceeds its set resource quota, the master station 20 can take measures such as restricting the resource usage of the fusion terminal 30, warning the user of the fusion terminal 30, or forcibly disconnecting the connection. Thus, it achieves the effect of making the resource usage of all the fusion terminals 30 with which a communication connection has been established less than the set resource quota by adopting a resource control strategy. Furthermore, it realizes preventing the mirror update process from affecting other tasks running on the terminal.

[0162] The present invention also designs a timeout protection mechanism. In the case where the mirror start time of the fusion terminal 30 exceeds the set time threshold, control the fusion terminal 30 to trigger a rollback operation, that is, control the fusion terminal 30 to return to the previous mirror version to ensure the stability of the fusion terminal 30.

[0163] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a machine-readable storage medium. When the computer program is executed by a processor, the computer can execute the container image acceleration method, and the method includes: controlling the data center to load and store the image files in the image repository; controlling the data center to send the image files to all master stations that have established communication connections when receiving an image update request sent by a fusion terminal; controlling each of the master stations to receive and store the image files; controlling each of the master stations to send the image files to all the non-image-updated fusion terminals that have established communication connections when receiving an image update request sent by the remaining non-image-updated fusion terminals; and controlling each of the fusion terminals to store the image files.

[0164] In another aspect, the present invention also provides a machine-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the container image acceleration method, and the method includes: controlling the data center to load and store the image files in the image repository; controlling the data center to send the image files to all master stations that have established communication connections when receiving an image update request sent by a fusion terminal; controlling each of the master stations to receive and store the image files; controlling each of the master stations to send the image files to all the non-image-updated fusion terminals that have established communication connections when receiving an image update request sent by the remaining non-image-updated fusion terminals; and controlling each of the fusion terminals to store the image files.

[0165] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0166] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A container image acceleration method, characterized in that: include: Control the loading of data centers and storage of image files in image repositories; The control data center sends the image file to all the master stations with which communication connections have been established when receiving the image update request sent by the fusion terminal; Control each of the master stations to receive and store the image file; Control each of the master stations to send the image file to all the non-mirror-updated fusion terminals with established communication connections when receiving the image update request sent by the remaining non-mirror-updated fusion terminals; Control each of the fusion terminals to store the image file.

2. The container image acceleration method according to claim 1, characterized in that: After controlling each of the fusion terminals to store the image file, the method further includes: Controlling each of the fusion terminals to perform file integrity check on the image file; When the verification is passed, each of the fusion terminals is controlled to send a mirror update completion notification to the corresponding master station with which the communication connection is established.

3. The container image acceleration method according to claim 2, characterized in that: After controlling each of the fusion terminals to send a notification of image update completion to the corresponding master station with which the communication connection is established, the method further includes: Control each master station to send an update time instruction to all the fusion terminals that have established communication connections; the update time instruction includes the image version and effective time of the image file; Each master station is controlled to send a unified start instruction to all the fusion terminals with which communication connection has been established, so that all the fusion terminals perform a unified start of the image file based on the timestamp of the effective time.

4. The container image acceleration method according to claim 3, characterized in that: The method further comprises: In the case where the image version of one of the fusion terminals is inconsistent with the image versions of the other fusion terminals, the step of controlling each master station to send an update time instruction to all the fusion terminals with which communication connections have been established, and the step of controlling each master station to send a unified startup instruction to all the fusion terminals with which communication connections have been established are performed again until the image versions of all the fusion terminals are consistent.

5. The container image acceleration method according to claim 1, characterized in that: The method further comprises: During the mirror startup process, each master station is controlled to adopt a resource control strategy so that the resource usage of all the fusion terminals with established communication connections is less than a set resource quota.

6. The container image acceleration method according to claim 1, characterized in that: The method further comprises: When the mirroring startup time of the fusion terminal exceeds a set time threshold, the fusion terminal is controlled to trigger a rollback operation.

7. The container image acceleration method according to claim 1, characterized in that: The image request time for the fusion terminal to send the image update request to the data center is calculated based on the set communication acceleration ratio, the number of fusion terminals, the number of master stations, the depth of the distribution tree from the fusion terminal to the data center, and the Https request time of a single node; wherein the set communication acceleration ratio represents the communication acceleration ratio between the communication mode between the data center and the master station and the communication mode between the master station and the fusion terminal.

8. The container image acceleration method according to claim 7, characterized in that: The image request time is calculated by the following formula: T2 = N*a*t+n*(k-1)*t; Among them, N represents the number of fusion terminals, n represents the number of master stations, k represents the depth of the distribution tree from the fusion terminal to the data center, t represents the Https request time of a single node, T2 represents the mirror request time, and a represents the set communication acceleration ratio.

9. A container image acceleration device, characterized in that: include: A first control module, used to control the data center to load and store image files in the image warehouse; A second control module is used to control the data center to send the image file to all the master stations with which communication connections have been established when receiving the image update request sent by the fusion terminal; A third control module, used for controlling each of the master stations to receive and store the image file; A fourth control module, configured to control each of the master stations to send the image file to all the non-mirror-updated fusion terminals with which communication connection has been established, upon receiving an image update request sent by the remaining non-mirror-updated fusion terminals; The fifth control module is used to control each of the fusion terminals to store the image file.

10. The container image acceleration device according to claim 9, characterized in that: The device also includes: A sixth control module, used to control each of the fusion terminals to perform a file integrity check on the image file; The seventh control module is used to control each of the fusion terminals to send a mirror update completion notification to the corresponding master station with which the communication connection is established when the verification is passed.

11. The container image acceleration device according to claim 10, characterized in that: The device also includes: An eighth control module, configured to control each master station to send an update time instruction to all the fusion terminals with which communication connections have been established; the update time instruction includes an image version and an effective time of the image file; The ninth control module is used to control each master station to send a unified startup instruction to all the fusion terminals with which communication connection has been established, so that all the fusion terminals can perform a unified startup of the image file based on the timestamp of the effective time.

12. The container image acceleration device according to claim 11, characterized in that: The device also includes: The retry module is used to execute the eighth control module and the ninth control module again when the image version of one of the fusion terminals is inconsistent with the image versions of other fusion terminals, until the image versions of all the fusion terminals are consistent.

13. The container image acceleration device according to claim 9, characterized in that: The device also includes: The tenth control module is used to control each master station to adopt a resource control strategy during the mirror startup process so that the resource usage of all the fusion terminals that have established communication connections is less than a set resource quota.

14. The container image acceleration device according to claim 9, characterized in that: The device also includes: The eleventh control module is used to control the fusion terminal to trigger a rollback operation when the mirroring startup time of the fusion terminal exceeds a set time threshold.

15. The container image acceleration device according to claim 9, characterized in that: The image request time for the fusion terminal to send the image update request to the data center is calculated based on the set communication acceleration ratio, the number of fusion terminals, the number of master stations, the depth of the distribution tree from the fusion terminal to the data center, and the Https request time of a single node; wherein the set communication acceleration ratio represents the communication acceleration ratio between the communication mode between the data center and the master station and the communication mode between the master station and the fusion terminal.

16. The container image acceleration device according to claim 15, characterized in that: The image request time is calculated by the following formula: T2 = N*a*t+n*(k-1)*t; Among them, N represents the number of fusion terminals, n represents the number of master stations, k represents the depth of the distribution tree from the fusion terminal to the data center, t represents the Https request time of a single node, T2 represents the mirror request time, and a represents the set communication acceleration ratio.

17. A container image acceleration system, characterized in that: include: The data center is used to load and store the image file in the image warehouse and send the image file to all the master stations with which the communication connection has been established when receiving the image update request sent by the fusion terminal; A plurality of master stations, each of which is communicatively connected to the data center, and each of which is used to receive and store the image file, and, upon receiving an image update request sent by the remaining fusion terminals that have not been updated, send the image file to all the fusion terminals that have not been updated and have established communication connections; A plurality of fusion terminals, each of the master stations is communicatively connected with at least one of the fusion terminals, and each of the fusion terminals is used to store the image file.

18. A machine-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the container image acceleration method according to any one of claims 1 to 8 is implemented.

19. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the container image acceleration method according to any one of claims 1 to 8 is implemented.

Images