Batch upgrade of devices based on packet scheduling and state closed-loop tracing method

Abstract

The application discloses a kind of equipment batch upgrade and state closed loop tracing method based on grouping scheduling, applied to cloud platform.First, the equipment to be upgraded is divided into multiple groups, an independent upgrade subqueue is created for each group, and a weight value is configured, and the total concurrent resources of the system are allocated in proportion to realize grouping level flow limiting scheduling.The device ID obtained by the issuing authority is uniformly mapped to multiple consumer nodes by hash calculation, and the upgrade instruction is issued in parallel by each node.The device end reports the state in real time according to the preset state machine model through the reliable message protocol.Monitor the residence time of each state, and if it is timed out, it is judged to be failed and triggers differentiated backoff retry: retryable failed equipment re-queues to wait for the next round of scheduling, and non-retryable terminates.Finally, update the scheduling engine after the final state is reported, complete the whole process tracing and management.The application effectively balances the system load, avoids resource contention, significantly improves the throughput and success rate of large-scale equipment upgrade, and ensures that the upgrade process is controllable and traceable.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) device manufacturing technology, and in particular to a method for batch device upgrades and closed-loop status tracing based on group scheduling. Background Technology

[0002] In the manufacturing process of IoT devices, firmware upgrades are a crucial step before products leave the factory. Currently, production line equipment upgrades primarily rely on manual, one-by-one connection operations. Operators use tools such as serial cables, USB data cables, or JTAG debuggers to connect to each device to be upgraded, manually triggering firmware burning or upgrade commands via host computer software, waiting for completion, disconnecting, and switching to the next device. This method allows one operator to handle only one device at a time, with each device taking several minutes. When production batches reach thousands or even tens of thousands of units, the upgrade process becomes a bottleneck in production capacity. Furthermore, the large amount of repetitive, mechanical labor leads to high labor costs and is prone to human errors such as missed upgrades, incorrect upgrades, and version incompatibility. Upgrade records are also fragmented, making it difficult to meet quality traceability and compliance audit requirements.

[0003] Some existing solutions attempt to upgrade multiple devices simultaneously via LAN broadcast, but the devices must be on the same LAN segment, and the broadcast process lacks a two-way confirmation mechanism, meaning the platform cannot know the actual upgrade result for each device. Another solution involves remotely upgrading already sold devices via a cloud-based OTA platform; however, this type of solution is primarily for devices already delivered to end users. In high-concurrency scenarios where upgrades are performed immediately upon batch power-on on production lines, it suffers from drawbacks such as devices not yet having completed user registration and thus unable to access the system, lack of batch task orchestration, and inability to integrate with MES (Manufacturing Execution System). Summary of the Invention

[0004] To address at least one of the aforementioned technical problems, this invention provides a method for batch device upgrades and closed-loop status tracing based on group scheduling.

[0005] In a first aspect, the present invention provides a method for batch device upgrades and closed-loop status tracing based on group scheduling, applied to a cloud platform, the method comprising:

[0006] Multiple devices to be upgraded are divided into several device groups according to preset dimensions, and an independent upgrade task sub-queue is created for each device group; the preset dimensions include production line, production batch or device model;

[0007] A weight value is configured for each device group, and the total concurrent resources of the system are allocated proportionally according to the weight value of each group. The devices in each group are subject to rate limiting and scheduling, so that the devices in each group can obtain the authority to issue upgrade tasks.

[0008] The device ID that has obtained the permission to issue the upgrade command is hashed, and the device is mapped to one of multiple consumer nodes based on the hash value. Each consumer node then issues the upgrade command to the corresponding device.

[0009] The receiving device reports the current upgrade status according to a preset state machine model via a reliable message transmission protocol; the state machine model includes at least the following states: pending upgrade, downloading, verification, upgrade, success, and failure.

[0010] The device upgrade process is monitored in a closed loop according to the state machine model. When the device stays in any state for a longer time than the preset timeout threshold, the upgrade is determined to have failed and failure processing is triggered.

[0011] Based on the failure type, a differentiated backoff and retry strategy is implemented, re-entering retryable failed devices into the tail of the corresponding group queue to wait for the next round of scheduling, and marking non-retryable failed devices as terminated.

[0012] After receiving the final status reported by the receiving device, update the scheduling engine status.

[0013] Preferably, the rate limiting scheduling based on group weights specifically includes: assigning weights to each group, and the distribution rate of each group is the product of the proportion of the assigned weight and the total weight and the upper limit of the system's total rate.

[0014] Preferably, the step of hashing the device ID that has obtained the permission to issue the permission, and mapping the device to one of multiple consumer nodes based on the hash value, includes:

[0015] Obtain the device ID of the device to which the upgrade command is to be issued;

[0016] Calculate the hash value of the device ID, perform a modulo operation with the total number of consumer nodes N as the modulus, and route the device to the consumer node corresponding to the operation result;

[0017] The consumer node sends an upgrade command to the corresponding device;

[0018] In this system, the hash value of the same device ID is fixed, and all upgrade instructions for the same device are routed to the same consumer node and executed sequentially within that consumer node.

[0019] Preferably, the reliable message transmission protocol is the QoS=2 level of the MQTT protocol.

[0020] Preferably, the transition conditions of the state machine model are as follows:

[0021] When the status is pending upgrade, an upgrade command is sent to the device, and the device responds to the upgrade command and migrates to the downloading process;

[0022] When the state is downloading, if a notification is received from the device that the firmware package download is complete and the MD5 verification is successful, the device will be moved to the verification state. If the duration of the device in the downloading state exceeds a first preset threshold, a timeout failure process will be triggered, and the device's state will be moved to failure.

[0023] When the status is "verification in progress", if a notification that the firmware verification has passed is received from the device, the device will be moved to "upgrade in progress"; if the duration of the device in the "verification in progress" status exceeds the second preset threshold, a timeout failure process will be triggered, and the device will be moved to the "failure" status.

[0024] When the status is "Upgrading", if a notification is received from the device that the new firmware has started successfully after a reboot, the process will proceed to "Success". If a notification is received from the device that the startup has failed, or if the duration of the device in the "Upgrading" status exceeds a third preset threshold, the process will proceed to "Failure".

[0025] Preferably, the method further includes a zero-disruption rollback step during the firmware upgrade process, specifically:

[0026] Obtain the firmware package MD5 verification pass information reported by the device;

[0027] In response to the MD5 verification pass information, a boot partition switching command is sent to the device. The boot partition switching command is used to instruct the device to switch the boot flag from the primary partition to the backup partition and perform a reboot operation to load the new version firmware from the backup partition.

[0028] Receive the heartbeat signal periodically reported by the device before the watchdog timer expires;

[0029] If no heartbeat signal is received from the device within the preset heartbeat monitoring time, a rollback command is sent to the device. The rollback command instructs the device to switch the boot flag back from the spare partition to the main partition and perform a reboot operation to load the original firmware from the main partition and resume operation.

[0030] Preferably, the step of implementing a differentiated backoff and retry strategy based on the failure type specifically includes:

[0031] Obtain the failure code corresponding to the device upgrade failure, and determine the failure type based on the failure code;

[0032] When the failure type is a network failure, the maximum number of retries is configured to be 3, and the backoff interval between two adjacent retries increases exponentially. The backoff interval for the nth retry is T0×2n, where T0 is the preset base backoff time, and n=0,1,2.

[0033] When the failure type is a verification failure, the maximum number of retries is configured to be 1, and the backoff interval is T0.

[0034] When the failure type is a storage failure, the maximum number of retries is set to 0, retries are abandoned and the failure is marked as unrecoverable.

[0035] When the failure type does not belong to any of the above categories, the maximum number of retries is configured to be 1, the backoff interval is T0, and if the failure still occurs after retries, a manual processing prompt message is generated.

[0036] Secondly, the present invention also provides a batch upgrade and status closed-loop traceability system for devices based on group scheduling, applied to a cloud platform, the system comprising:

[0037] The equipment group management module is used to divide multiple devices to be upgraded into several equipment groups according to preset dimensions, and to create an independent upgrade task sub-queue for each equipment group; the preset dimensions include production line, production batch or equipment model;

[0038] The concurrent scheduling engine is used to configure weight values ​​for each device group, allocate the total concurrent resources of the system proportionally according to the weight values ​​of each group, perform rate limiting scheduling on the devices in each group, enable the devices in each group to obtain the authority to issue upgrade tasks, and perform hash calculation on the device ID that has obtained the authority to issue tasks, map the device to one of multiple consumer nodes according to the hash value, and have each consumer node issue upgrade instructions to the corresponding device.

[0039] The upgrade status monitoring module is used to receive the current upgrade status reported by the device according to a preset state machine model through a reliable message transmission protocol; the state machine model includes at least the states of pending upgrade, downloading, verification, upgrade, success, and failure; and to perform closed-loop monitoring of the device upgrade process according to the state machine model. When the device stays in any state for a longer time than a preset timeout threshold, the upgrade is determined to have failed and failure processing is triggered.

[0040] The failure handling and retry module is used to execute differentiated backoff and retry strategies based on the failure type, put retryable failed devices back into the tail of the corresponding group queue to wait for the next round of scheduling, and mark non-retryable failed devices as terminated.

[0041] The status closed-loop management module is used to update the scheduling engine status after receiving the final status reported by the device.

[0042] Thirdly, the present invention also provides an electronic device including a processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein when the processor executes the computer instructions, the electronic device performs the method as described in the first aspect above and any possible implementation thereof.

[0043] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor of an electronic device, cause the processor to perform a method as described in the first aspect above and any possible implementation thereof.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] This invention combines group-weighted token bucket rate limiting scheduling with device hash partitioning consumption to achieve on-demand allocation of system concurrent resources according to production line / batch / model, effectively avoiding resource crowding by a single group, supporting high-concurrency upgrades of tens of thousands of devices, and significantly improving production line upgrade throughput. Through MQTT QoS=2 reliable message transmission and state machine timeout closed-loop monitoring, it ensures that status messages for each device are not lost or duplicated; any state timeout automatically results in failure, achieving full-link controllable traceability of the upgrade process. By differentiating backoff retries based on failure code type, different retry counts and backoff intervals are configured for network, verification, and storage failures, avoiding wasted resources from ineffective retries and reducing ineffective scheduling overhead while improving upgrade success rate. Compared to manual upgrades one device at a time, this invention can save a significant amount of manpower, improve upgrade efficiency by tens of times, and automatically generate upgrade reports containing root cause attribution results, meeting quality audit requirements.

[0046] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the background art, the accompanying drawings used in the embodiments of the present invention or the background art will be described below.

[0048] Figure 1 A flowchart illustrating a method for batch device upgrades and status closed-loop tracing based on group scheduling, provided in an embodiment of the present invention;

[0049] Figure 2 for Figure 1 A flowchart illustrating the sub-steps of step S30;

[0050] Figure 3 This is a schematic diagram of a device batch upgrade and status closed-loop traceability system based on group scheduling, provided in an embodiment of the present invention. Detailed Implementation

[0051] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0053] In existing production line firmware upgrade solutions, manual one-by-one connection upgrades are inefficient and prone to human error, while LAN broadcast upgrades lack a two-way confirmation mechanism, resulting in unpredictable upgrade results. Existing OTA solutions lack batch task orchestration and state closed-loop traceability capabilities in high-concurrency scenarios, failing to meet the manufacturing enterprises' needs for efficient, controllable, and auditable upgrades of large-scale equipment. To address these issues, this invention groups devices by production line, batch, or model and configures weights, combining this with a token bucket algorithm to achieve group-level proportional rate limiting, preventing a single group from consuming system resources. Upgrade instructions are evenly distributed to multiple consumer nodes and sent in parallel through device ID hash partitioning, while ensuring that instructions for the same device are sent sequentially. Furthermore, a state closed loop is constructed using MQTT QoS=2 reliable transmission and a state machine timeout transition mechanism, ensuring that device states are not lost or duplicated, and automatically triggering failure handling if any link times out. Finally, differentiated backoff and retries are executed based on the failure code type, forming a complete closed-loop link of "scheduling → sending → state transition → result feedback → scheduling adjustment." The invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for batch device upgrades and status closed-loop tracing based on group scheduling, provided in an embodiment of the present invention. The method is applied to a cloud platform and includes the following steps:

[0055] S10. Divide multiple devices to be upgraded into several device groups according to preset dimensions, and create an independent upgrade task sub-queue for each device group; the preset dimensions include production line, production batch or device model;

[0056] Specifically, the cloud platform obtains the attribute data of the devices to be upgraded from the MES system or device registration information. Maintenance personnel select the categorization dimensions through the platform interface (e.g., selecting "Production Line" groups devices by their production line, "Production Batch" groups them by batch number, and "Equipment Model" groups them by product model). The platform then aggregates the devices into several groups Gg={G1,G2,...,G...} based on the selected dimensions. m After grouping, the platform creates a separate first-in-first-out (FIFO) upgrade task sub-queue Qg for each group Gg. This queue only stores the device IDs of the devices to be upgraded within that group. The group queues are isolated from each other, and the same device belongs to only one group queue at any given time. The device IDs in the queue are sorted by addition time to ensure FIFO. This division into independent sub-queues lays the foundation for data isolation in subsequent differentiated rate-limiting scheduling based on grouping.

[0057] S20. Configure a weight value for each device group, allocate the total concurrent resources of the system according to the weight value of each group, and perform flow-limiting scheduling on the devices in each group so that the devices in each group can obtain the authority to issue upgrade tasks.

[0058] Preferably, the rate limiting scheduling based on group weights specifically includes: assigning weights to each group, and the distribution rate of each group is the product of the proportion of the assigned weight and the total weight and the upper limit of the system's total rate.

[0059] In this step, operations and maintenance personnel configure a weight value wg (positive integer, default value is 1) for each group Gg based on the production capacity priority of each production line, the urgency of delivery of each batch, or the business importance of each model. The cloud platform's concurrent scheduling engine has a built-in token bucket rate limiter, with a total system rate limit of R (unit: tokens / second, which can be configured according to platform performance, e.g., R=200). The token issuance rate rg for group Gg = R×wg / Σw means that each group shares the total system throughput according to its weight. Groups with higher weights can obtain more tokens per unit time, thus allowing more devices to enter the issuance process; groups with a weight of 0 do not obtain tokens and scheduling is suspended.

[0060] The token bucket continuously generates tokens at a rate rg and stores them in the bucket of group Gg. The upper limit of the bucket capacity can be set to twice rg to cope with sudden traffic surges. The scheduling engine polls the head device of each group's queue: if there are available tokens in the group's token bucket, a token is taken out for the device, the device gains the right to issue tokens, is removed from the head of the queue, and proceeds to step S30; if there are no tokens in the bucket, the group is skipped, and the next group is polled, thus achieving the allocation of concurrent resources among groups according to weight ratios. During the rate limiting process, the platform records the number of tokens consumed by each group in real time. When a device in a group completes its upgrade, the consumed tokens are not "returned" but are continuously replenished by the token bucket over time. Therefore, over a long period of time, the actual concurrency of each group strictly converges to the weight ratio. The core effect of this mechanism is to prevent a single group (such as a large production line) from crowding out all system resources, causing other group devices to be unable to be scheduled for a long time. At the same time, the weight configuration gives maintenance personnel the ability to flexibly allocate resources according to business priorities, realizing the controllable and predictable allocation of system resources among groups, and supporting the scheduling fairness and flexibility in scenarios with tens of thousands of concurrent device upgrades.

[0061] S30. Calculate the hash of the device ID that has obtained the permission to issue the upgrade command, and map the device to one of the multiple consumer nodes according to the hash value. Each consumer node then issues the upgrade command to the corresponding device.

[0062] For devices that obtain distribution permissions in step S20, the platform obtains their unique device identifier ID (such as device serial number or MAC address), calls a consistent hash algorithm or standard hash function (such as MD5, SHA-1, or Java String.hashCode()) to calculate the hash value h = hash(deviceID), and performs a modulo operation using the total number of online consumer nodes N as the modulus: partition = h mod N, routing the device to the consumer node numbered partition. Each consumer node is an independent service instance deployed in a distributed cluster, with no shared state between nodes. Each consumer node maintains its own distribution thread pool. Upon receiving the routed device upgrade instruction, it sends an upgrade instruction message to the device's MQTT topic. This message includes parameters such as the firmware download URL, target version number, and MD5 checksum.

[0063] It is important to emphasize that, because the hash value of the same device ID is fixed, all upgrade instructions for the same device are always routed to the same consumer node. Within this node, they are executed serially in the order they are received, thus strictly ensuring the orderliness of instructions for a single device and avoiding state corruption or duplicate upgrades caused by concurrent distribution. Since different devices have different IDs, the hash results are evenly distributed across the consumer nodes, enabling parallel distribution across devices. When the number of consumer nodes changes (e.g., node expansion or contraction), the consistent hashing algorithm minimizes the number of remapped devices, affecting only the routing targets of a small number of devices. This step allows the system to horizontally scale consumer nodes while maintaining the orderliness of instructions for a single device, avoiding the performance overhead caused by distributed locks or global sequence number mechanisms introduced to maintain order in a distributed environment.

[0064] See Figure 2 In one specific implementation, step S30, which involves hashing the device ID for which the issuance permission has been obtained and mapping the device to one of multiple consumer nodes based on the hash value, includes the following sub-steps:

[0065] S301. Obtain the device ID of the device for which an upgrade command is to be issued;

[0066] After the device obtains the token through step S20 and enters the distribution process, the scheduling engine retrieves the device ID from the group queue as the target device identifier to be distributed. The device ID is a unique identifier for the device and can be a device serial number (SN), MAC address, or a globally unique ID assigned by the platform (such as UUID). This ID is bound to the device when it is first registered and remains unchanged throughout its life.

[0067] S302. Calculate the hash value of the device ID, perform a modulo operation with the total number of consumer nodes N as the modulus, and route the device to the consumer node corresponding to the operation result;

[0068] The platform obtains a list of currently online consumer nodes and counts the total number of nodes N (N≥1, typically 3~10 in a deployment). It calculates the hash value of each device ID using a hash function, including but not limited to MD5, SHA-1, SHA-256, Java's String.hashCode(), and C++'s std::hash. Using the hash value as input, it performs a modulo operation on N: partition = hash(deviceID) mod N, obtaining an integer value between 0 and N-1. The platform maintains a consumer node routing table, recording the IP address and port number of the consumer node instances numbered 0 to N-1, and routes devices to the corresponding consumer nodes based on the calculation result. When the consumer node cluster expands or shrinks (e.g., adding a node or a node crashing and going offline), the value of N changes, and the platform dynamically updates the routing table and recalculates the routing allocation for subsequent devices. For devices already undergoing upgrades, their original routing nodes remain unchanged until the upgrade is complete, avoiding inconsistencies caused by mid-upgrade switching.

[0069] Furthermore, in a preferred embodiment, the platform employs a consistent hashing algorithm instead of direct modulo operation: the IP address or unique identifier of the consumer node is mapped to multiple virtual node positions on the hash ring, and the hash value of the device ID is also mapped to the hash ring. The nearest virtual node is searched in a clockwise direction, and the consumer node to which this virtual node belongs is the target node. Consistent hashing only affects a small number of devices in adjacent intervals on the hash ring when the number of nodes changes. Compared to direct modulo operation, which may cause large-scale device remapping, this significantly reduces the impact of cluster scaling on ongoing upgrade tasks.

[0070] S303, The consumer node sends an upgrade command to the corresponding device;

[0071] In this system, the hash value of the same device ID is fixed, and all upgrade instructions for the same device are routed to the same consumer node and executed sequentially within that consumer node.

[0072] After receiving the device ID and upgrade task parameters (including firmware download URL, target version number, MD5 checksum, upgrade strategy, etc.) from the routing, the target consumer node sends an upgrade command message to the device-side topic via the MQTT protocol. Internally, the consumer node maintains a distribution thread pool that can process distribution requests from multiple devices in parallel. Each device's distribution operation is encapsulated as an independent task and submitted to the thread pool for execution. After distribution is complete, the consumer node records a distribution log (including distribution time, distribution result, etc.) for subsequent auditing.

[0073] Because the hash function always produces the same output for the same input (device ID), and the modulo operation rule is constant for the same device ID, each upgrade instruction for the same device in multiple rounds of scheduling (such as re-queuing after an initial upgrade failure and regaining the right to issue commands again) is routed to the same consumer node. Within this consumer node, for multiple instructions with the same device ID (such as initial issuance instructions and retry issuance instructions), the consumer node places them into a bounded queue sorted by reception time. A single thread processes them sequentially in FIFO order. Each subsequent instruction must wait for the previous instruction to complete processing (i.e., receive confirmation from the device or time out) before execution begins. This ensures strict serialization and sequential consistency among multiple instructions from the same device, preventing state overwriting or duplicate upgrades caused by concurrent issuance. Different device IDs, due to their different hash values, are evenly distributed across different consumer nodes and even different processing threads within the same consumer node, achieving full parallelism across devices.

[0074] Through the above implementation method, on the one hand, by using hash modulo to evenly distribute devices across multiple consumer nodes, the system's concurrency capacity expands linearly with the number of nodes, supporting tens of thousands of devices simultaneously undergoing upgrade processes. On the other hand, by using fixed hashing to ensure that the same device always lands on the same node and executes serially, complex coordination mechanisms such as distributed locks or global sequence numbers are avoided to maintain the order of instructions for a single device, reducing the complexity of system implementation and runtime overhead. In contrast, if random allocation or round-robin allocation is used, retry instructions for the same device may land on different nodes, causing multiple nodes to issue instructions to the same device simultaneously, leading to state confusion and repeated upgrades; if a global lock is used to maintain order, it becomes a performance bottleneck under high concurrency. This implementation method solves both load balancing and instruction ordering problems simultaneously with lightweight hash routing, combining scalability and correctness.

[0075] S40. The receiving device reports the current upgrade status according to a preset state machine model via a reliable message transmission protocol; the state machine model includes at least the states of pending upgrade, downloading, verification, upgrading, success, and failure.

[0076] Preferably, the reliable message transmission protocol is the QoS=2 level of the MQTT protocol.

[0077] In this step, after receiving the upgrade command, the device's DMC initiates the upgrade process and reports its current status to the cloud according to a preset state machine model. The state machine defines six core states: PENDING (initial state, indicating the device is queued and awaiting upgrade), DOWNLOADING (downloading firmware package via HTTP / HTTPS from a specified URL), VERIFYING (verifying firmware integrity using MD5), UPGRADING (performing firmware replacement and system reboot), SUCCESS (upgrade complete and new firmware boots normally), and FAILED (any stage of the upgrade process fails). The state transition path is: PENDING → DOWNLOADING → VERIFYING → UPGRADING → SUCCESS, and any state can transition to FAILED.

[0078] Furthermore, the device reports its status via the MQTT protocol. The message subject format is / device / {deviceID} / upgrade / state, and the message body is in JSON format, containing the following fields: device ID, current status, timestamp, failure code (only carried in case of failure), and progress percentage (optional). The MQTT transmission protocol uses QoS=2 (Exactly Once) to ensure that each status message is delivered only once between the device and the cloud, without loss or duplication. The cloud maintains an in-memory status view for each device, storing it with the device ID as the key and the current status and dwell start time as the values, and persists it to the database. This step, with QoS=2, ensures the reliability of status reporting, providing an accurate data foundation for subsequent closed-loop monitoring and preventing the platform from misjudging the device status due to message loss.

[0079] S50. The device upgrade process is monitored in a closed loop according to the state machine model. When the device stays in any state for a longer time than the preset timeout threshold, the upgrade is determined to fail and failure processing is triggered.

[0080] Preferably, the transition conditions of the state machine model are as follows:

[0081] When the status is pending upgrade, an upgrade command is sent to the device, and the device responds to the upgrade command and migrates to the downloading process;

[0082] When the state is downloading, if a notification is received from the device that the firmware package download is complete and the MD5 verification is successful, the device will be moved to the verification state. If the duration of the device in the downloading state exceeds a first preset threshold, a timeout failure process will be triggered, and the device's state will be moved to failure.

[0083] When the status is "verification in progress", if a notification that the firmware verification has passed is received from the device, the device will be moved to "upgrade in progress"; if the duration of the device in the "verification in progress" status exceeds the second preset threshold, a timeout failure process will be triggered, and the device will be moved to the "failure" status.

[0084] When the status is "Upgrading", if a notification is received from the device that the new firmware has started successfully after a reboot, the process will proceed to "Success". If a notification is received from the device that the startup has failed, or if the duration of the device in the "Upgrading" status exceeds a third preset threshold, the process will proceed to "Failure".

[0085] Step S50 performs closed-loop monitoring of the device upgrade process based on the state machine model. When the device's dwell time in any state exceeds a preset timeout threshold, the upgrade is deemed to have failed, and the state machine transition conditions involved in the failure handling are as follows: The cloud platform maintains a state machine instance for each device, recording the device's current state, the timestamp of entering the current state, and the state transition history. At each key node of the upgrade process, the device-side DMC actively reports the status to the cloud according to the predefined state transition path. The complete set of states defined by the state machine model is: PENDING, DOWNLOADING, VERIFYING, UPGRADING, SUCCESS, and FAILED. Among them, PENDING, DOWNLOADING, VERIFYING, and UPGRADING are intermediate states, and SUCCESS and FAILED are final states. The specific transition conditions between each state are as follows:

[0086] 1) The migration conditions for PENDING→DOWNLOADING.

[0087] When the device status is PENDING, it indicates that the device has entered the upgrade task sub-queue and is waiting for scheduling. After the platform completes token acquisition and hash routing through steps S20 and S30, the consumer node sends the upgrade command to the device. Upon receiving the command message sent to the ` / device / {deviceID} / upgrade / command` topic, the device's DMC parses the firmware download URL, target version number, MD5 checksum, and other parameters in the message body, then starts the download process and reports the current status as DOWNLOADING to the cloud via MQTT QoS=2. Upon receiving this status message, the cloud transitions the device's state machine from PENDING to DOWNLOADING and records the timestamp T_start_download for entering the DOWNLOADING state. If the device fails to enter the download process after receiving the command due to an anomaly (such as network failure or parsing failure), it directly reports FAILED along with the corresponding failure code.

[0088] 2) Transition conditions for the DOWNLOADING state.

[0089] When the device status is DOWNLOADING, the device is downloading a firmware package using the HTTP / HTTPS protocol, and supports resuming interrupted downloads. This status transition has two paths: successful transition and failed transition.

[0090] Successful migration path: After the device completes the download of all bytes of the firmware package, it immediately calculates the MD5 hash value of the downloaded binary file and compares the result with the expected MD5 checksum sent from the cloud. If they match, it indicates that the firmware package has not been damaged or tampered with during transmission. The device reports the status as VERIFYING to the cloud via MQTT QoS=2, and includes a "verification passed" flag in the message body. Upon receiving this notification, the cloud migrates the device status from DOWNLOADING to VERIFYING and records the timestamp T_start_verify for entering the VERIFYING status.

[0091] Failure Migration Path (Timeout Failure): The cloud maintains a resident timer for each device in the DOWNLOADING state, starting from T_start_download. The platform presets a first preset threshold T1 (default value is 10 minutes, which can be configured within the range of 1 to 30 minutes in actual deployment based on factors such as firmware package size and network bandwidth). The closed-loop monitoring thread periodically scans all devices in the DOWNLOADING state and calculates the difference Δt_download between the current time and T_start_download. If Δt_download > T1 and no VERIFYING status is reported from the device, the download is determined to have timed out. The cloud forcibly migrates the device state to FAILED, generates failure code 1010 (download timeout), records the failure reason as "download timeout, exceeding the preset threshold T1", and triggers the failure handling logic in step S60. The timeout judgment takes precedence over the status reported by the device later. That is, if the device reports VERIFYING after the timeout, the cloud is already in the FAILED state and will no longer accept the migration from FAILED to VERIFYING, ensuring the irreversibility of the state migration and the consistency of the platform's judgment.

[0092] 3) Transition conditions for the VERIFYING state.

[0093] When the device status is VERIFYING, the device is performing an MD5 check (if the check is completed synchronously after downloading, this status is transient, and the device will immediately proceed to the next process after reporting; if the check takes a long time, it will remain in this status). The transition to this status also has two paths: success and failure.

[0094] Successful migration path: After the device completes the MD5 verification and the result matches, it reports the VERIFYING status to the cloud (in some implementations, if the verification is completed during the DOWNLOADING phase, the UPGRADING status is reported directly). After receiving the notification that the verification is successful, the cloud migrates the device status from VERIFYING to UPGRADING and records the timestamp T_start_upgrade for entering the UPGRADING status.

[0095] Failure migration path (timeout failure): The cloud presets a second preset threshold T2 (default value is 2 minutes, which can be configured within the range of 1 to 5 minutes according to the device's computing power). The closed-loop monitoring thread scans the dwell time of devices in the VERIFYING state. If Δt_verify > T2 and no UPGRADING state is reported from the device, the cloud forces the device state to migrate to FAILED, generates failure code 1021 (verification timeout), and triggers failure handling.

[0096] 4) Transition conditions for the UPGRADING state.

[0097] When the device status is UPGRADING, the device is performing firmware replacement and system reboot operations. This state transition has two paths: success and failure.

[0098] Successful migration path: After the device completes the firmware writing to the spare partition, boot flag switching, and system reboot, the new firmware boots normally. The device's DMC performs a self-check after the new firmware boots, confirming that system services are running normally, and reports a SUCCESS status via MQTT, with the message body containing the new firmware version number and boot time. Upon receiving the SUCCESS status, the cloud migrates the device status from UPGRADING to SUCCESS, records the upgrade completion timestamp, updates the success count in the scheduling engine, and the device upgrade process loop is complete.

[0099] Failure Migration Path 1 (Startup Failure Notification): When the device experiences an abnormal startup after a reboot with new firmware (such as a kernel crash or failure to load critical services), the DMC captures the startup failure event and reports the FAILED status via MQTT, carrying the failure code 2010 (startup failure). Upon receiving this notification, the cloud migrates the device status from UPGRADING to FAILED, records the reason for the failure, and triggers failure handling.

[0100] Failure Migration Path Two (Upgrade Timeout): The cloud presets a third threshold T3 (default value is 5 minutes, configurable within the range of 3-15 minutes depending on device restart time). The closed-loop monitoring thread scans the dwell time of devices in the UPGRADING state. If Δt_upgrade > T3 and no SUCCESS or FAILED status report is received from the device, the cloud forcibly migrates the device status to FAILED, generating failure code 1031 (upgrade timeout) and triggering failure handling.

[0101] It should be noted that the three timeout thresholds T1, T2, and T3 mentioned above are configurable parameters. Maintenance personnel can adjust them by group or globally through the cloud platform's management interface to adapt to differences in computing power among different device models, network environments across different production lines, and firmware package sizes. When a device migrates from any intermediate state to FAILED, the platform records the current state, dwell time, failure code, and triggering method (device-initiated reporting or platform-mandated timeout determination). This data serves as the source for subsequent S60 differentiated retry steps and audit reports.

[0102] Therefore, the above method breaks down the device upgrade process into a finite number of discrete states and sets independent timeout thresholds and clear migration conditions for each intermediate state, transforming the traditional "issue command and ignore it" black-box upgrade into a white-box state machine management that is observable and controllable across the entire link. The timeout forced judgment mechanism solves the "silent suspension" problem when the device cannot actively report failures due to anomalies such as device crashes or network disconnections, ensuring that any abnormal situation can be captured by the platform and trigger compensation actions, thereby achieving highly reliable closed-loop traceability of the upgrade process.

[0103] S60. Execute a differentiated backoff and retry strategy according to the failure type, put retryable failed devices back into the tail of the corresponding group queue to wait for the next round of scheduling, and mark non-retryable failed devices as terminated.

[0104] Preferably, the step of implementing a differentiated backoff and retry strategy based on the failure type specifically includes:

[0105] Obtain the failure code corresponding to the device upgrade failure, and determine the failure type based on the failure code;

[0106] When the failure type is a network failure, the maximum number of retries is configured to be 3, and the backoff interval between two adjacent retries increases exponentially. The backoff interval for the nth retry is T0×2n, where T0 is the preset base backoff time, and n=0,1,2.

[0107] When the failure type is a verification failure, the maximum number of retries is configured to be 1, and the backoff interval is T0.

[0108] When the failure type is a storage failure, the maximum number of retries is set to 0, retries are abandoned and the failure is marked as unrecoverable.

[0109] When the failure type does not belong to any of the above categories, the maximum number of retries is configured to be 1, the backoff interval is T0, and if the failure still occurs after retries, a manual processing prompt message is generated.

[0110] When a device actively reports a FAILED status with a failure code, or when the cloud-based closed-loop monitoring module forcibly determines a failure due to timeout and generates a failure code, the platform first obtains the failure code c_i. The platform has a pre-set failure code enumeration table, classifying failure codes into several root cause categories based on code segments: 101x network segments (1010~1019) correspond to network-related failures, including but not limited to connection timeout (1010), download interruption (1011), network unreachable (1012), TCP connection failure (1013), etc. These failures are usually caused by instantaneous network fluctuations, temporary server overload, or unstable device signals; 102x network segments (1020~1029) correspond to verification-related failures, including but not limited to MD5 mismatch (1020), file integrity verification failure (1021), firmware package signature verification failure (1022), etc. 22) and similar failures indicate that bit-level data corruption occurred during the download or storage of the firmware package; 103x network segments (1030~1039) correspond to storage-related failures, including but not limited to insufficient device storage space (1030), Flash write failure (1031), and partition table corruption (1032), which reflect unrecoverable anomalies in the device's hardware storage resources; 201x network segments (2010~2019) correspond to device anomaly-related failures, including but not limited to reboot failure (2010), new firmware startup failure (2011), and DMC process crash (2012), which are unknown types of anomalies. The platform maps the first segment of the failure code to the corresponding failure type label by parsing the first segment of the failure code. If the failure code does not match any defined code segment, it is classified as "unknown".

[0111] If step S601 determines the failure type to be network-related, the platform generates a retry record for the device, initializing the number of retries retry_count = 0 and the maximum number of retries max_retries = 3. The device's first retry (the 0th retry, i.e., n=0) requires a backoff interval T0 (default value is 60 seconds, configurable within the range of 30~180 seconds depending on the production line network environment). That is, the device waits for T0 seconds after being marked as failed before being reinserted into the group queue. If the first retry fails again, a second retry (n=1) is performed, with a waiting interval of T0×2=120 seconds. If the second retry also fails, a third retry is performed (n=2), with a waiting interval of T0×4=240 seconds. After three failed retries, the device stops retrying, is marked as terminated, and the final failure code is recorded for manual troubleshooting. The backoff interval is implemented as follows: the platform writes a "next schedulable time" field into the device's failure record, with the value being the current timestamp plus the backoff interval value; when polling the group queue, the scheduling engine only considers devices whose current time is greater than or equal to the "next schedulable time" as schedulable, and devices whose time has not yet arrived do not participate in token competition. The core technical effect of this exponential backoff mechanism is that it disperses retry requests along the timeline, avoiding network congestion and server receive queue overflow caused by a large number of devices retrying at the same time, while gradually lengthening the interval as the number of retries increases, allowing sufficient time for network fault recovery.

[0112] If the failure type is verification-related, the platform is configured with max_retries=1, and the backoff interval is fixed at T0 (60 seconds). After a failure, the device waits for T0 hours and then re-enters the queue for the next round of scheduling. If the device still fails after re-downloading and verification (i.e., a verification-related failure code is reported again), it will not be retried, will be marked as terminated, and an operation and maintenance prompt will be generated stating "The firmware package may be corrupted; it is recommended to re-upload the firmware package." The technical consideration for giving only one retry for verification-related failures is that the root cause of MD5 verification failures is usually a corrupted firmware package in the cloud, bad blocks in the device's Flash storage unit, or data corruption due to a high network error rate during the download process. These causes have a limited probability of being resolved through repeated downloads (except for network errors). Giving one retry can eliminate occasional verification failures caused by instantaneous network fluctuations. If the device still fails after one retry, continuing to retry is almost impossible to change the result and will only waste system scheduling resources and device power. Therefore, timely termination and prompting for manual intervention is the optimal strategy.

[0113] If the failure type is storage-related, the platform configures `max_retries=0`, meaning no retry operations are performed. The device is directly marked as terminated, its record in the group queue is removed, and it will no longer participate in any subsequent scheduling. The platform simultaneously generates a device anomaly alarm, alerting maintenance personnel that the device's storage hardware has malfunctioned (e.g., insufficient Flash space or write failure), requiring manual intervention (e.g., replacing the storage chip or repartitioning). The technical effect of not retrying storage-related failures is that insufficient storage space, Flash write failures, and other hardware-level unrecoverable anomalies not only fail to solve the problem with repeated retries but also repeatedly execute time-consuming download and write operations, further consuming the device's limited storage resources and power. It may even accelerate hardware aging due to frequent Flash writes. Therefore, immediate termination is the optimal resource protection strategy.

[0114] If the failure code does not fall into any of the defined code segments 101x, 102x, 103x, or 201x, or falls into the 201x device anomaly class (although defined, its root cause is complex, neither purely network nor verification or storage issues), the platform classifies it as "Unknown," configures max_retries=1, and sets the backoff interval to T0. After a failure, the device waits for T0 hours and re-enters the queue for one retry. If the retry still fails, the device is marked as terminated, and the platform generates a "Unknown anomaly, manual intervention required" message, including the device ID, failure code, failure time, and complete state transition log, for maintenance personnel to perform root cause analysis in conjunction with the device's logs.

[0115] Except for storage-related failures (max_retries=0) which terminate directly, other types of failures, if the retry count has not reached the upper limit, undergo a unified enqueue operation: the platform re-adds the device ID to the tail of the upgrade task sub-queue Q_g of its group Gg, resets the device state machine to PENDING (pending upgrade), and clears the device's "occupied token" record in the scheduling engine. The device will then participate again as a newly enqueued device in the token competition of step S20 and the hash partition distribution of step S30. If the device's retry count has reached the maximum retry count for the corresponding type, it will no longer be enqueued, will be marked as TERMINATED, and its final failure status will be recorded. All operations in the entire retry process (failure code, failure type, number of retryes, enqueue time and dequeue time for each retry) are written to the database audit table for traceability when exporting subsequent upgrade reports.

[0116] This strategy achieves a fine balance between the benefits and costs of retrying. Tests have shown that, compared to a uniform retrying strategy of 3 times, the differentiated strategy can significantly improve the upgrade success rate with the same retry resource consumption, while reducing the system overhead caused by invalid retrying. This significantly improves resource utilization efficiency and the final upgrade success rate in large-scale upgrade scenarios.

[0117] S70: After receiving the final status reported by the receiving device, update the scheduling engine status.

[0118] After the upgrade process is completed (whether successfully or ultimately unsuccessfully), the device reports its final status as SUCCESS or FAILED to the cloud. Upon receiving this status message, the cloud updates the device's status record in the scheduling engine and removes the device from the active upgrade list. If it is SUCCESS, the scheduling engine adds it to the success count for that group and releases the scheduling resources occupied by the device (such as the distribution thread context of the consumer node). If it is FAILED and the device has been marked as terminated (not retryable), it is added to the failure count and the failure details are archived. Based on this, the scheduling engine updates the real-time progress metrics for each group (such as group success rate and the number of remaining devices to be upgraded) and drives the token scheduling decisions for subsequent devices.

[0119] At this point, the complete closed-loop process for a single device, from queuing, obtaining a token, issuing instructions, reporting status, monitoring timeouts, retrying on failure, to finally transmitting the status back, has been completed. This closed-loop process ensures that the status changes of each device throughout the entire upgrade process are within the observable and controllable range of the platform, achieving closed-loop management of "scheduling → issuing instructions → status transition → result transmission → scheduling adjustment." This fundamentally solves the problem of traditional solutions where "instructions are issued but the results are ignored," leading to unknowable upgrade status and neglect of failed devices.

[0120] In one embodiment, the method further includes a zero-disruption rollback step during the firmware upgrade process, specifically:

[0121] Obtain the firmware package MD5 verification pass information reported by the device;

[0122] In response to the MD5 verification pass information, a boot partition switching command is sent to the device. The boot partition switching command is used to instruct the device to switch the boot flag from the primary partition to the backup partition and perform a reboot operation to load the new version firmware from the backup partition.

[0123] Receive the heartbeat signal periodically reported by the device before the watchdog timer expires;

[0124] If no heartbeat signal is received from the device within the preset heartbeat monitoring time, a rollback command is sent to the device. The rollback command instructs the device to switch the boot flag back from the spare partition to the main partition and perform a reboot operation to load the original firmware from the main partition and resume operation.

[0125] In this embodiment, after the device's DMC completes the firmware package download and performs an MD5 hash operation on the downloaded binary file, it compares the calculation result with the expected MD5 value sent from the cloud. If the comparison matches, the device reports a verification pass message to the cloud via MQTT. This message includes the device ID, target version number, verification result (passed), and verification time. Upon receiving the verification pass notification, the cloud confirms that the integrity and correctness of the firmware package meet the upgrade conditions and proceeds to the subsequent partition switching process.

[0126] After receiving the successful MD5 checksum, the cloud sends a boot partition switch command to the device via MQTT in JSON format. The device's storage medium (such as NOR Flash, NAND Flash, or eMMC) is pre-divided into two independent partitions: primary partition A and backup partition B. Primary partition A stores the current firmware version running on the device, while backup partition B has already been written with the new firmware version that passed MD5 checksum during the previous download step. The device changes the boot flag (boot_flag) in the bootloader from the default value A to B. This boot flag is stored in a non-volatile memory area (such as a specific sector of EEPROM or Flash) to ensure that the correct boot partition can still be read after a power outage and reboot. Subsequently, the device performs a system soft reboot (by calling the kernel's reboot system call or triggering a reset via a hardware watchdog). During startup, the bootloader reads the boot flag, finds its value to be B, and then loads and boots the new firmware version from backup partition B. It is important to emphasize that during the firmware download and MD5 verification phases (i.e., the DOWNLOADING and VERIFYING states of the state machine), the device always runs on the original firmware version of the main partition A, and business functions are completely unaffected. The device continues to collect data, report heartbeats, and respond to cloud commands normally, achieving "zero business interruption" during the download and verification phase. The device only goes offline briefly when the restart operation is performed in step S802 (usually lasting several seconds to tens of seconds), and the business resumes immediately after the restart is completed.

[0127] After the device boots with the new firmware, the DMC immediately starts a watchdog timer. The timeout duration of this timer is preset to T_watchdog (default value 90 seconds, which can be configured within the range of 60~180 seconds depending on the device boot time). Before the watchdog timeout, the DMC completes system service initialization, network connection establishment, MQTT reconnection, and other operations in the critical stages after boot. During this period, it reports heartbeat signals to the cloud heartbeat reception topic at fixed intervals (e.g., every 15 seconds). The heartbeat message body includes the device ID, current firmware version number, system uptime, and critical service status. The cloud maintains a heartbeat reception record table, recording the latest heartbeat reception timestamp for each device in the UPGRADING state.

[0128] The cloud-based heartbeat monitoring duration, T_heartbeat, is configured (default value is 120 seconds, which is T_watchdog plus a redundancy margin to ensure the cloud's judgment occurs later than the device's watchdog timeout, avoiding false judgments). The cloud-based heartbeat monitoring thread periodically scans all devices in the UPGRADING state and calculates the difference Δt_heartbeat between the current time and the latest heartbeat reception time. If Δt_heartbeat > T_heartbeat, the cloud determines that the new firmware boot has failed (possible reasons include new firmware kernel crash, incompatible critical drivers causing system hangs, DMC process not starting properly, etc.), and immediately sends a rollback command to the device. Upon receiving the rollback command, the device's bootloader switches the boot flag from B back to A and performs another system reboot. After rebooting, the bootloader reads the boot flag as A, loads the original firmware from the main partition A, and the device returns to its pre-upgrade state, thus preventing the device from becoming bricked due to the new firmware failing to boot properly. After the rollback is completed, the device reports the FAILED status to the cloud via MQTT, carrying the failure code 2011 (new firmware failed to start, rollback has been completed). The cloud then migrates the device status to FAILED and triggers the failure handling in step S60.

[0129] Furthermore, in a preferred embodiment of this implementation, the device-side watchdog timer itself also serves as an independent security defense: if the DMC process crashes and fails to report heartbeats after the new firmware boots, the device-side watchdog timer will automatically trigger a hardware-level reboot after timeout, and the Bootloader is configured to prioritize attempting primary partition A (i.e., default rollback) during the first boot after the watchdog reset, thereby achieving local autonomous rollback without relying on cloud commands. Cloud heartbeat monitoring serves as a second line of defense, completing recovery by issuing rollback commands if the device-side watchdog fails to trigger a rollback (e.g., the watchdog driver is not initialized correctly).

[0130] This embodiment achieves triple protection during firmware upgrades through a collaborative mechanism of A / B dual-partition design and watchdog heartbeat monitoring: zero service interruption during the download verification phase, automatic rollback if the new firmware fails to boot after an upgrade restart, and proactive rollback command if no heartbeat is received from the cloud within a timeout period. This reduces the risk of device bricking to near zero. Compared to the traditional single-partition overwrite upgrade method (where the device becomes bricked if the new firmware fails to write or boots abnormally, requiring factory disassembly and re-flashing), this implementation reduces the damage caused by upgrade failure from "irreversible hardware-level failure" to "automatically recoverable software-level anomaly." This significantly improves fault tolerance and operational efficiency in large-scale production line upgrade scenarios, while also protecting hardware assets from being scrapped due to upgrade failure.

[0131] See Figure 3 In one embodiment, the present invention also provides a batch device upgrade and status closed-loop traceability system based on group scheduling, applied to a cloud platform, the system comprising:

[0132] The equipment group management module 100 is used to divide multiple devices to be upgraded into several equipment groups according to preset dimensions, and to create an independent upgrade task sub-queue for each equipment group; the preset dimensions include production line, production batch or equipment model;

[0133] The concurrent scheduling engine 200 is used to configure weight values ​​for each device group, allocate the total concurrent resources of the system proportionally according to the weight values ​​of each group, perform rate limiting scheduling on the devices in each group, enable the devices in each group to obtain the authority to issue upgrade tasks, and perform hash calculation on the device ID that has obtained the authority to issue tasks, map the device to one of multiple consumer nodes according to the hash value, and have each consumer node issue upgrade instructions to the corresponding device.

[0134] The upgrade status monitoring module 300 is used to receive the current upgrade status reported by the device according to a preset state machine model through a reliable message transmission protocol; the state machine model includes at least the states of pending upgrade, downloading, verification, upgrading, success, and failure; and to perform closed-loop monitoring of the device upgrade process according to the state machine model. When the device stays in any state for a longer time than a preset timeout threshold, the upgrade is determined to have failed and failure processing is triggered.

[0135] The failure handling and retry module 400 is used to execute differentiated backoff and retry strategies according to the failure type, put retryable failed devices back into the tail of the corresponding group queue to wait for the next round of scheduling, and mark non-retryable failed devices as terminated.

[0136] The status closed-loop management module 500 is used to update the scheduling engine status after receiving the final status reported by the device.

[0137] It is understood that the system provided in this embodiment has functions or includes modules that can be used to execute the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0138] The present invention also provides an electronic device including a processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein when the processor executes the computer instructions, the electronic device performs a method as described in any of the above possible implementations.

[0139] The present invention also provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor of an electronic device, cause the processor to perform a method as described in any of the above possible implementations.

[0140] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

Claims

1. A method for batch device upgrades and closed-loop status tracing based on group scheduling, characterized in that, Applied to a cloud platform, the method includes: Multiple devices to be upgraded are divided into several device groups according to preset dimensions, and an independent upgrade task sub-queue is created for each device group; the preset dimensions include production line, production batch or device model; A weight value is configured for each device group, and the total concurrent resources of the system are allocated proportionally according to the weight value of each group. The devices in each group are subject to rate limiting and scheduling, so that the devices in each group can obtain the authority to issue upgrade tasks. The device ID that has obtained the permission to issue the upgrade command is hashed, and the device is mapped to one of multiple consumer nodes based on the hash value. Each consumer node then issues the upgrade command to the corresponding device. The receiving device reports the current upgrade status according to a preset state machine model via a reliable message transmission protocol; the state machine model includes at least the following states: pending upgrade, downloading, verification, upgrade, success, and failure. The device upgrade process is monitored in a closed loop according to the state machine model. When the device stays in any state for a longer time than the preset timeout threshold, the upgrade is determined to have failed and failure processing is triggered. Based on the failure type, a differentiated backoff and retry strategy is implemented, re-entering retryable failed devices into the tail of the corresponding group queue to wait for the next round of scheduling, and marking non-retryable failed devices as terminated. After receiving the final status reported by the receiving device, update the scheduling engine status.

2. The method for batch device upgrade and status closed-loop traceability based on group scheduling according to claim 1, characterized in that, The rate limiting scheduling based on group weights specifically includes: assigning weights to each group, and the distribution rate of each group is the product of the assigned weight, the proportion of the total weight, and the upper limit of the system's total rate.

3. The method for batch device upgrade and status closed-loop traceability based on group scheduling according to claim 1, characterized in that, The step of hashing the device ID that has obtained the permission to issue commands, and mapping the device to one of multiple consumer nodes based on the hash value, includes: Obtain the device ID of the device to which the upgrade command is to be issued; Calculate the hash value of the device ID, perform a modulo operation with the total number of consumer nodes N as the modulus, and route the device to the consumer node corresponding to the operation result; The consumer node sends an upgrade command to the corresponding device; In this system, the hash value of the same device ID is fixed, and all upgrade instructions for the same device are routed to the same consumer node and executed sequentially within that consumer node.

4. The method for batch device upgrade and status closed-loop traceability based on group scheduling according to claim 1, characterized in that, The reliable message transmission protocol is the MQTT protocol with QoS=2 level.

5. The method for batch device upgrade and status closed-loop traceability based on group scheduling according to claim 1, characterized in that, The specific transition conditions of the state machine model are as follows: When the status is pending upgrade, an upgrade command is sent to the device, and the device responds to the upgrade command and migrates to the downloading process; When the status is "downloading", if a notification is received from the device that the firmware package download is complete and the MD5 verification is successful, then the process will be moved to "verification". If the duration of the device in the download state exceeds a first preset threshold, a timeout failure process is triggered, and the device state is transitioned to failure. When the status is "verification in progress", if a notification is received from the device that the firmware verification has passed, then the process will proceed to "upgrade in progress". If the dwell time of the device in the verification state exceeds the second preset threshold, timeout failure processing is triggered, and the state of the device is transitioned to failure. When the status is "Upgrading", if a notification is received from the device that the new firmware has started successfully after a reboot, the process will proceed to "Success". If a notification is received from the device that the startup has failed, or if the duration of the device in the "Upgrading" status exceeds a third preset threshold, the process will proceed to "Failure".

6. The method for batch device upgrade and status closed-loop traceability based on group scheduling according to claim 1, characterized in that, The method also includes a zero-disruption rollback step during the firmware upgrade process, specifically: Obtain the firmware package MD5 verification pass information reported by the device; In response to the MD5 verification pass information, a boot partition switching command is sent to the device. The boot partition switching command is used to instruct the device to switch the boot flag from the primary partition to the backup partition and perform a reboot operation to load the new version firmware from the backup partition. Receive the heartbeat signal periodically reported by the device before the watchdog timer expires; If no heartbeat signal is received from the device within the preset heartbeat monitoring time, a rollback command is sent to the device. The rollback command instructs the device to switch the boot flag back from the spare partition to the main partition and perform a reboot operation to load the original firmware from the main partition and resume operation.

7. The method for batch device upgrade and status closed-loop traceability based on group scheduling according to claim 1, characterized in that, The differentiated backoff and retry strategy based on the failure type is specifically as follows: Obtain the failure code corresponding to the device upgrade failure, and determine the failure type based on the failure code; When the failure type is a network failure, the maximum number of retries is configured to be 3, and the backoff interval between two adjacent retries increases exponentially, with the backoff interval for the nth retry being T0×2. n Where T0 is the preset basic retreat time, n=0,1,2; When the failure type is a verification failure, the maximum number of retries is configured to be 1, and the backoff interval is T0. When the failure type is a storage failure, the maximum number of retries is set to 0, retries are abandoned and the failure is marked as unrecoverable. When the failure type does not belong to any of the above categories, the maximum number of retries is configured to be 1, the backoff interval is T0, and if the failure still occurs after retries, a manual processing prompt message is generated.

8. A batch upgrade and status closed-loop traceability system for equipment based on group scheduling, characterized in that, The system, applied to a cloud platform, includes: The equipment group management module is used to divide multiple devices to be upgraded into several equipment groups according to preset dimensions, and to create an independent upgrade task sub-queue for each equipment group; the preset dimensions include production line, production batch or equipment model; The concurrent scheduling engine is used to configure weight values ​​for each device group, allocate the total concurrent resources of the system proportionally according to the weight values ​​of each group, perform rate limiting scheduling on the devices in each group, enable the devices in each group to obtain the authority to issue upgrade tasks, and perform hash calculation on the device ID that has obtained the authority to issue tasks, map the device to one of multiple consumer nodes according to the hash value, and have each consumer node issue upgrade instructions to the corresponding device. The upgrade status monitoring module is used to receive the current upgrade status reported by the device according to a preset state machine model through a reliable message transmission protocol; the state machine model includes at least the states of pending upgrade, downloading, verification, upgrade, success, and failure; and to perform closed-loop monitoring of the device upgrade process according to the state machine model. When the device stays in any state for a longer time than a preset timeout threshold, the upgrade is determined to have failed and failure processing is triggered. The failure handling and retry module is used to execute differentiated backoff and retry strategies based on the failure type, put retryable failed devices back into the tail of the corresponding group queue to wait for the next round of scheduling, and mark non-retryable failed devices as terminated. The status closed-loop management module is used to update the scheduling engine status after receiving the final status reported by the device.

9. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein when the processor executes the computer instructions, the electronic device performs the device batch upgrade and status closed-loop traceability method based on group scheduling as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which includes program instructions that, when executed by a processor of an electronic device, cause the processor to perform the device batch upgrade and status closed-loop traceability method based on group scheduling as described in any one of claims 1 to 7.

Images