A service image differential updating system and method based on an edge weak network environment

Abstract

A service image differential update system and method based on edge weak network environments, belonging to the field of software update and edge computing technology, is characterized by: including a cloud server and an edge node; the cloud server is equipped with a differential code generation module; the edge node is equipped with a local compilation environment image library, an image packaging tool engine, a service build project management module, an image repository submission module, a container scheduling and update module, and a service catalog synchronization module; only differential code is transmitted, which can greatly reduce update traffic and effectively reduce image size; it utilizes existing local basic components to avoid repeated packaging and significantly reduce image size; it supports breakpoint resume and cache collaboration, which significantly improves the update success rate; it supports rolling updates and automatic rollback to ensure service continuity; it supports local building in multiple languages ​​and can flexibly adapt to multiple languages; it is suitable for large-scale edge deployment and can coordinate update tasks among tens of thousands of edge nodes.

Description

Technical Field

[0001] This invention belongs to the field of software update and edge computing technology, and particularly relates to a service mirror differential update system and method based on edge weak network environment. Background Technology

[0002] As cloud computing extends to the edge, more and more services are being deployed on edge nodes closer to the data source. However, most of these nodes currently suffer from one or more of the following technical issues: Poor network conditions: low bandwidth, high latency, and unstable signal; limited resources: limited CPU, memory, and storage space; high update frequency: frequent service iterations in microservice architecture require frequent service image updates; prominent problems with traditional image update methods: full image updates result in a large amount of redundant data transmission; images contain complete runtime environments (such as JDK, Python, etc.), causing large file sizes; the characteristics of union file systems limit the possibility of direct compression at the image level; updates are prone to failure due to network interruptions, affecting business continuity.

[0003] Therefore, there is an urgent need for a service mirror update method that can adapt to weak network environments at the edge, which can significantly reduce data transmission while ensuring update efficiency. Summary of the Invention

[0004] This invention aims to solve the above problems and provides a service mirror differential update system and method based on edge weak network environments.

[0005] In a first aspect, the present invention provides a service image differential update system based on a weak edge network environment, comprising a cloud server and an edge node; the cloud server is equipped with a differential code generation module; the differential code generation module is equipped with a differential code transmission interface; the edge node is equipped with a local compilation environment image library, an image packaging tool engine, a service build project management module, an image repository submission module, a container scheduling and update module, and a service catalog synchronization module; The differential code generation module is used to generate differential code based on the differential code request submitted by the user. The differential code transmission interface is used to send differential codes from the cloud to the edge nodes; The local compilation environment image library is used to pre-configure compilation environment images for multiple languages ​​(such as Java, Python, Go, etc.); The image packaging tool engine is used to integrate image packaging tools; The service construction project management module is used to organize service image construction tasks; The image repository submission module is used to submit the built service image to a local or remote image repository. The container scheduling and update module is used for the deployment, rolling update and rollback of service images; The service catalog synchronization module is used to verify that the service metadata is consistent with the actual deployment status.

[0006] Secondly, this invention provides a service mirror differential update method based on a weak network environment at the edge, comprising the following steps: Step 1: After receiving the service version update request, the cloud server first compares the code, configuration files and dependencies of the currently running image on the edge node with the new version image to generate a comparison result. The differential code generation module automatically generates a differential file package based on the comparison results; After verifying the differential file package, save it as a differential package and mark the version number and dependency relationship; Step 2: Upload the differential packet to the edge node task queue via the differential code transmission interface and transmit it. During the transmission, the size and order of the transmission blocks are dynamically adjusted according to the network status of the edge node to ensure stable transmission under weak network conditions. At the same time, by adopting a differential code generation and transmission mechanism, the amount of data required for updates can be significantly reduced. Step 3: After receiving the differential packet, the edge node performs integrity verification on the packet body; after passing the verification, the differential packet is temporarily stored in the local cache in preparation for image update. Step 4: Load the corresponding compilation environment image from the local compilation environment image library to compile the differential package; Step 5: The image packaging tool engine starts the service build project, merging the updated code in the differential package with the existing local source code and dependency files; after the build is completed, a new service image file is generated; building with a local image can effectively avoid the bandwidth pressure caused by full image transmission; Step 6: Push the completed image to the local image repository on the edge node and update the version index file at the same time; the system records the image metadata before submission for subsequent rollback operations. Step 7: After the container scheduling module detects the new image version, it initiates the rolling update process; the system stops the old container and starts the new container, and verifies the health status and log output of the new container at each step; once the new instance is running stably, it gradually replaces the old version to ensure uninterrupted service. Step 8: After the new version of the container is launched, the service directory synchronization module updates the registry information so that the caller can correctly access the new service instance. Step 9: The system continuously monitors the operational metrics of the new version of the service (such as response time, error rate, etc.).

[0007] Furthermore, in the service image differential update method based on edge weak network environment described in this invention, step 9 of the system continuously monitors the operation indicators of the new version service, and automatically triggers the rollback process when an abnormal situation is detected, and re-activates the old version image container to achieve smooth switching and ensure business continuity.

[0008] Furthermore, in the service image differential update method based on edge weak network environment described in this invention, the construction process in step 5 adopts a multi-stage construction mechanism, including a compilation stage and a runtime stage; the compilation stage involves calling a pre-set compilation image to compile the differential code and generate intermediate products (such as JAR packages, model files, etc.); the runtime stage involves copying only the products required for runtime and the minimum dependencies, and removing the compilation toolchain and temporary files, thereby significantly reducing the image size.

[0009] Furthermore, the service mirror differential update method based on edge weak network environment described in this invention supports breakpoint resumption and data verification retransmission in the transmission process of step 2, which can still stably complete the service update in the weak network environment and improve the update success rate.

[0010] Furthermore, in the service image differential update method based on edge weak network environment described in this invention, step 4, when loading the compilation environment image from the local compilation environment image library, reuses the local cached image to avoid repeated downloads and improve efficiency.

[0011] Furthermore, in the service image differential update method based on edge weak network environment described in this invention, the differential file package in step 1 includes modified source code, configuration fragments, and dependency description files.

[0012] Furthermore, in the service image differential update method based on edge weak network environment described in this invention, the image metadata includes image tag, dependency version, and build time.

[0013] The service image differential update system and method based on edge weak network environments described in this invention, compared with existing technologies, can greatly reduce update traffic by transmitting only differential code instead of the entire image, saving more than 90% of bandwidth; effectively reduce image size by utilizing existing local basic components to avoid repeated packaging and significantly reduce image size; adapt to weak network environments, support breakpoint resume and caching collaboration, significantly improving update success rate; provide a local pre-built compilation environment to achieve "one-click build" of service images, improving build efficiency; support rolling updates and automatic rollback to ensure service continuity and enhance system robustness; support local builds in multiple languages ​​such as Java, Python, and Go, and can flexibly adapt to multiple languages; suitable for large-scale edge deployment, and can coordinate update tasks among tens of thousands of edge nodes. Attached Figure Description

[0014] Figure 1This is a schematic diagram of the service mirror differential update system structure based on the edge weak network environment described in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the implementation process of the service mirror differential update method based on a weak edge network environment as described in an embodiment of the present invention. Detailed Implementation

[0015] The following detailed description of the service mirror differential update system and method based on edge weak network environment described in this invention is provided with reference to the accompanying drawings and embodiments.

[0016] Example 1 This embodiment discloses a service mirror differential update system based on a weak network environment at the edge, including a cloud server and an edge node; such as Figure 1 As shown, the cloud server is equipped with a differential code generation module; the differential code generation module is equipped with a differential code transmission interface; the edge node is equipped with a local compilation environment image library, an image packaging tool engine, a service build project management module, an image repository submission module, a container scheduling and update module, and a service catalog synchronization module.

[0017] The differential code generation module is used to generate differential code based on differential code requests submitted by users; the differential code transmission interface is used to send differential code from the cloud to edge nodes; the local compilation environment image library is used to pre-configure compilation environment images for multiple languages ​​(e.g., Java, Python, Go, etc.); the image packaging tool engine is used to integrate image packaging tools; the service build project management module is used to organize service image build tasks; the image repository submission module is used to submit the built service image to a local or remote image repository; the container scheduling and update module is used for the deployment, rolling updates, and rollback of service images; and the service catalog synchronization module is used to verify that the service metadata is consistent with the actual deployment status.

[0018] The service mirror differential update system based on edge weak network environment described in this embodiment is particularly suitable for edge nodes with poor network conditions (such as low bandwidth, high latency, and frequent interruptions), and solves the problems of high failure rate, large resource consumption, and poor user experience caused by traditional full mirror update method.

[0019] Example 2 This embodiment discloses a service mirror differential update method based on a weak network environment at the edge, which is implemented based on the service mirror differential update system described in Embodiment 1 above.

[0020] The service mirror differential update method based on edge weak network environment described in this embodiment is as follows: Figure 2 As shown, the specific steps include: Step 1: After receiving the service version update request, the cloud server first compares the code, configuration files, and dependencies of the currently running image on the edge node with the new version image to generate a comparison result. The differential code generation module automatically generates a differential file package based on the comparison result. The differential file package includes the changed source code, configuration fragments, and dependency description files. After verification, the differential file package is saved as a differential package and marked with the version number and dependency relationship.

[0021] Step 2: The differential packet is uploaded to the edge node task queue via the differential code transmission interface and transmitted. During the transmission, the size and order of the transmission blocks are dynamically adjusted according to the network status of the edge node to ensure stable transmission under weak network conditions. At the same time, by adopting the differential code generation and transmission mechanism, the amount of data to be transmitted for the update can be significantly reduced. In this embodiment of the disclosure, the transmission process supports breakpoint resumption and data verification retransmission, which can still stably complete the service update in a weak network environment and improve the update success rate.

[0022] Step 3: After receiving the differential packet, the edge node performs integrity verification on the packet body; after passing the verification, the differential packet is temporarily stored in the local cache area in preparation for image update.

[0023] Step 4: Load the corresponding compilation environment image from the local compilation environment image library to compile the differential package; in this embodiment, when loading the compilation environment image from the local compilation environment image library, the local cached image is reused to avoid repeated downloads and improve efficiency.

[0024] Step 5: The image packaging tool engine starts the service build project, merging the updated code in the differential package with the existing local source code and dependency files; after the build is completed, a new service image file is generated; building with a local image can effectively avoid the bandwidth pressure caused by full image transmission.

[0025] The build process employs a multi-stage build mechanism, including a compilation stage and a runtime stage. The compilation stage involves calling a pre-built compilation image to compile the differential code and generate intermediate artifacts (such as JAR files, model files, etc.). The runtime stage only copies the artifacts required for runtime and the minimum dependencies, removing the compilation toolchain and temporary files, thereby significantly reducing the image size.

[0026] Step 6: Push the completed image to the local image repository on the edge node and update the version index file at the same time. Before committing, the system records the image metadata (including image tags, dependency versions and build time) for subsequent rollback operations.

[0027] Step 7: After the container scheduling module detects the new image version, it starts the rolling update process; the system stops the old container and starts the new container, and verifies the health status and log output of the new container at each step; when the new instance is running stably, it gradually replaces the old version to ensure that the business is not interrupted.

[0028] Step 8: After the new version of the container is launched, the service directory synchronization module updates the registry center information so that the caller can correctly access the new service instance.

[0029] Step 9: The system continuously monitors the operational metrics of the new version service (such as response time, error rate, etc.); when an anomaly is detected, the rollback process is automatically triggered to reactivate the old version image container to achieve a smooth switch and ensure business continuity.

[0030] In this embodiment, a visual inspection system is used as an example for illustration. This visual inspection system includes multiple edge nodes deployed in the field to perform real-time defect detection tasks. Each node is configured as follows: Hardware configuration: ARM64 architecture edge server, equipped with a 4-core CPU, 8GB memory, and 64GB local storage; Network environment: Hybrid network access, bandwidth fluctuation of 1-5 Mbps, average latency of 150ms; Software environment: Running K3s lightweight Kubernetes, with Docker used for container runtime.

[0031] During the initial deployment of the system, the edge nodes ran the ai-infer:1.0 image version, which included a Python 3.8 runtime environment and YOLOv5 model inference service. When the model version was upgraded to ai-infer:1.1 (which improved the model structure and preprocessing algorithm), the traditional full image update required retransferring approximately 1.2GB of image files, and transmission interruptions often occurred under weak network conditions.

[0032] The service mirror differential update method based on the weak network environment at the edge, as described in this embodiment, is used for updating, and specifically includes the following process: Step 1: Difference Generation The central cloud server extracted the changes between ai-infer:1.0 and ai-infer:1.1, including minor modifications to the YOLOv5 model file (approximately 60MB) and the Python inference script. The differential code generation module generated a differential package containing only model weights and part of the source code, with a total size of approximately 60MB, accounting for 5% of the original image size.

[0033] Step 2: Differential Transmission The differential code transmission interface divides the differential packets into 4MB blocks and sends them sequentially based on network monitoring results from the edge nodes. If the network is interrupted midway, the system automatically records the transmission breakpoint and resumes transmission to ensure data consistency.

[0034] Step 3: Local verification and compilation environment loading After receiving the complete differential packet, the edge node performs MD5 signature verification. Subsequently, it loads a pre-built Python 3.8 compilation image (python:3.8-slim) as the build base.

[0035] Step 4: Difference merging and multi-segment construction The image packaging tool merges the differential files into the local ai-infer:1.0 source code directory to generate a build project.

[0036] The first stage uses the compiled image to execute the model packaging script; the second stage only copies the runtime files and dependent libraries, removes unnecessary build toolchains, and generates the final runtime image ai-infer:1.1, with an image size of approximately 350MB.

[0037] Step 5: Image submission and scheduling update Once the build is complete, the image is pushed to the local Docker repository. The container scheduling module detects the new version and initiates a rolling update. Old containers are gradually decommissioned, and new containers start and register with the service directory. Step 6: Health Monitoring and Rollback Mechanism The system continuously monitors the running status of new containers. If abnormal latency or error rate exceeds the threshold within 10 minutes, it will automatically roll back to the ai-infer:1.0 image to ensure uninterrupted business operations.

[0038] Finally, the results were verified: In actual testing, the total transmission volume of differential updates was only 60MB, the update process time was reduced by 90%, and the update could still be completed stably in a weak network environment with a bandwidth of 2Mbps, with a success rate of 100%. The system runs stably and requires no manual intervention.

[0039] As can be seen from the above embodiments, through the complete process described above, the service image differential update method based on edge weak network environment described in this embodiment realizes the whole-link closed loop from differential generation, transmission, merging and construction to rolling update and automatic rollback. It can still complete service updates efficiently and stably in weak network environment, further verifying that the update system and method described in this embodiment are suitable for efficient version upgrade and deployment operations of containerized microservices, edge AI models and other services under conditions such as bandwidth limitation and network instability.

Claims

1. A service mirror differential update system based on edge weak network environment, characterized in that: It includes a cloud server and an edge node; the cloud server is equipped with a differential code generation module; the differential code generation module is equipped with a differential code transmission interface; the edge node is equipped with a local compilation environment image library, an image packaging tool engine, a service build project management module, an image repository submission module, a container scheduling and update module, and a service catalog synchronization module; The differential code generation module is used to generate differential code based on the differential code request submitted by the user. The differential code transmission interface is used to send differential codes from the cloud to the edge nodes; The local compilation environment image library is used to pre-configure compilation environment images for multiple languages; The image packaging tool engine is used to integrate image packaging tools; The service construction project management module is used to organize service image construction tasks; The image repository submission module is used to submit the built service image to a local or remote image repository. The container scheduling and update module is used for the deployment, rolling update and rollback of service images; The service catalog synchronization module is used to verify that the service metadata is consistent with the actual deployment status.

2. A service mirror differential update method based on edge weak network environment, characterized in that... Includes the following steps: Step 1: After receiving the service version update request, the cloud server first compares the code, configuration files and dependencies of the currently running image on the edge node with the new version image to generate a comparison result. The differential code generation module automatically generates a differential file package based on the comparison results; After verifying the differential file package, save it as a differential package and mark the version number and dependency relationship; Step 2: Upload the differential packets to the edge node task queue via the differential code transmission interface and transmit them; during the transmission process, dynamically adjust the size and order of the transmission blocks according to the network status of the edge nodes; Step 3: After receiving the differential packet, the edge node performs integrity verification on the packet body; After verification, the differential packet is temporarily stored in the local cache. Step 4: Load the corresponding compilation environment image from the local compilation environment image library to compile the differential package; Step 5: The image packaging tool engine starts the service build project, merging the updated code in the differential package with the existing local source code and dependency files for building; After the build is complete, a new service image file will be generated; Step 6: Push the completed image to the local image repository on the edge node, and update the version index file at the same time; The system records the image metadata before submission; Step 7: After the container scheduling module detects the new image version, it starts the rolling update process; the system stops the old container, starts the new container, and verifies the health status and log output of the new container at each step. Step 8: After the new version of the container is launched, the service directory synchronization module updates the registry information so that the caller can correctly access the new service instance. Step 9: The system continuously monitors the operational metrics of the new version of the service.

3. The service mirror differential update method based on edge weak network environment according to claim 2, characterized in that: The system described in step 9 continuously monitors the operating metrics of the new version service. If an anomaly is detected, it automatically triggers a rollback process and re-enabled the old version image container.

4. The service mirror differential update method based on edge weak network environment according to claim 2, characterized in that: The build process described in step 5 employs a multi-stage build mechanism, including a compilation stage and a runtime stage; The compilation stage involves calling a pre-built compilation image to compile the differential code and generate intermediate products. The runtime phase involves copying only the necessary artifacts and minimal dependencies, while removing the compilation toolchain and temporary files.

5. The service mirror differential update method based on edge weak network environment according to claim 3 or 4, characterized in that: The transmission process described in step 2 supports breakpoint resumption and data verification retransmission.

6. The service mirror differential update method based on edge weak network environment according to claim 5, characterized in that: Step 4 describes reusing the local cached image when loading the compilation environment image from the local compilation environment image library.

7. The service mirror differential update method based on edge weak network environment according to claim 5, characterized in that: The differential file package described in step 1 includes the modified source code, configuration fragments, and dependency description files.

8. The service mirror differential update method based on edge weak network environment according to claim 5, characterized in that: The image metadata includes image tags, dependency versions, and build time.

Images