A disaster recovery method for industrial equipment software OTA process

Abstract

The present invention discloses a disaster recovery method during the OTA process of industrial equipment software, which belongs to the field of data processing technology and includes the following steps: S1: obtaining the currently running program and generating a backup program, and executing the OTA update; S2: detecting the OTA update status and the device startup status. If it is detected that the OTA update fails or the device startup fails, the network quality level of the current industrial equipment is detected; if the network quality level of the current industrial equipment is higher than a set threshold, step S3 is executed; if the network quality level of the current industrial equipment is lower than or equal to the set threshold, step S4 is executed; S3: re-execute the OTA update; S4: select the backup program to restore the device operation. The disaster recovery method during the OTA process of industrial equipment software solves the problem that the existing OTA update process is prone to the risk of equipment failure caused by update failure, thereby resulting in low reliability of the OTA update.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a disaster recovery method in an industrial equipment software OTA process. Background Art

[0002] OTA (Over-the-Air) remote updates are a critical step in industrial software maintenance, but their practical application presents a complex technical challenge: any failure during the update process can cause the device to malfunction, resulting in serious user experience issues and potential financial losses. Update failures can arise from a variety of factors, including network outages, insufficient storage space, hardware compatibility issues, and software conflicts. Furthermore, if an update fails, the device may become unbootable, making remote intervention extremely difficult.

[0003] This creates a technical contradiction: on the one hand, we need frequent OTA updates to fix vulnerabilities and add features; on the other hand, each update potentially increases the risk of device failure. Furthermore, the different types of devices and diverse usage environments further complicate the issue, making it difficult to ensure the reliability of OTA updates. Summary of the Invention

[0004] In order to overcome the defects of the prior art, the present invention provides a disaster recovery method during the OTA process of industrial equipment software to solve the above problems.

[0005] The technical solution adopted by the present invention to solve the technical problem is: a disaster recovery method during the OTA process of industrial equipment software, comprising the following steps:

[0006] S1: Get the current running program and generate a backup program, and perform an OTA update to replace the current running program;

[0007] S2: Detecting the OTA update status and device startup status. If the OTA update failure or device startup failure is detected, the network quality level of the current industrial device is detected; if the network quality level of the current industrial device is higher than the set threshold, step S3 is executed; if the network quality level of the current industrial device is lower than or equal to the set threshold, step S4 is executed;

[0008] S3: Re-execute OTA update;

[0009] S4: Select the backup program to restore the device.

[0010] It is worth noting that in step S2, the preset status file is detected through the recovery process or the exit status code of the OTA update process is detected through the recovery process. If the status file indicates that the update is not completed or the exit status code is a value corresponding to the update failure, the OTA update is judged to have failed.

[0011] It is worth noting that in step S2, whether the device is successfully started is determined by detecting the startup flag file through the recovery process or detecting the running status of the device's key services through the recovery process.

[0012] It is worth noting that, in step S2, the step of detecting the preset status file through the recovery process includes:

[0013] When the OTA update starts, a status file is preset; after the update is successfully completed, the status file is deleted and a first success flag file is created;

[0014] When the recovery process detects the status file, if the status file exists and the first success flag file does not exist, it is determined that the OTA update has failed; if the first success flag file exists, it is determined that the OTA update has succeeded.

[0015] Specifically, in step S2, the step of detecting the exit status code of the OTA update process through the recovery process includes: in the OTA update program, assigning values ​​corresponding to update success and update failure to the exit status code; the recovery process obtains the exit status code of the OTA update process; if the exit status code is a value corresponding to update success, it is judged that the OTA update is successful; if the exit status code is a value corresponding to update failure, it is judged that the OTA update has failed.

[0016] Preferably, in step S2, the step of detecting the startup flag file through the recovery process includes:

[0017] When the system starts, a startup flag file is created; after the system is completely started, the startup flag file is deleted and a second success flag file is created;

[0018] When the recovery process detects the startup flag file, if the startup flag file exists and the second success flag file does not exist, it is determined that the device startup fails; if the second success flag file exists, it is determined that the device startup is successful.

[0019] It is worth noting that in step S2, when the running status of the device key service is detected through the recovery process, if the device key service is not running, it is judged that the device startup has failed; if the device key service is running, it is judged that the device startup has succeeded.

[0020] Optionally, in step S2, whether the device is started is detected by a hardware watchdog timer. If the hardware watchdog timer is not reset regularly, it is determined that the device startup has failed.

[0021] It is worth noting that in step S1, before executing the OTA update, the program file and related configuration files are obtained from the path where the currently running program is located, a backup program is generated and stored in the specified backup path, and the storage location of the backup program is recorded after the storage is completed.

[0022] It is worth noting that in step S2, the step of re-executing the OTA update is: calling the OTA update tool through the recovery process to re-obtain the update package and execute the OTA update; in the process of executing step S3, the number of times the OTA update is executed is detected. When the number of times the OTA update is executed exceeds the update threshold, it is detected that the OTA update fails or the device fails to start, and the backup program is selected to restore the device operation.

[0023] The beneficial effect of the present invention is that in the disaster recovery method during the industrial equipment software OTA process, the update result is judged by monitoring the OTA update status and the device startup status. If an OTA update failure or a device startup failure is detected, the network quality level of the current industrial equipment is detected to determine whether the network environment is stable. If the network quality level of the current industrial equipment is higher than the set threshold, it indicates that the network environment is stable and the OTA update will be re-triggered. If the network quality level of the current industrial equipment is lower than or equal to the set threshold, it indicates that the network environment is unstable and the device operation will be restored through the backup program. This solution provides a more comprehensive disaster recovery mechanism by combining the re-execution of OTA updates with the recovery of backup programs to ensure reliability during OTA updates. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Flowchart of a disaster recovery method during an OTA process for industrial equipment software according to an embodiment of the present invention;

[0025] Figure 2 The flowchart of detecting the exit status code of the OTA update process through the recovery process in one embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following is a further description of specific embodiments of the present invention in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is intended to facilitate understanding of the present invention and does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0027] like Figure 1 and 2 As shown, a disaster recovery method during the OTA process of industrial equipment software includes the following steps:

[0028] S1: Get the current running program and generate a backup program, and perform an OTA update to replace the current running program;

[0029] S2: Detecting the OTA update status and device startup status. If the OTA update failure or device startup failure is detected, the network quality level of the current industrial device is detected; if the network quality level of the current industrial device is higher than the set threshold, step S3 is executed; if the network quality level of the current industrial device is lower than or equal to the set threshold, step S4 is executed;

[0030] S3: Re-execute OTA update;

[0031] S4: Select the backup program to restore the device.

[0032] In the disaster recovery method during the industrial equipment software OTA process, the update result is determined by monitoring the OTA update status and device startup status. If an OTA update failure or device startup failure is detected, the network quality level of the current industrial equipment is detected to determine whether the network environment is stable. If the current network quality level of the industrial equipment is higher than the set threshold, it indicates that the network environment is stable and the OTA update will be retriggered. If the current network quality level of the industrial equipment is lower than or equal to the set threshold, it indicates that the network environment is unstable and the device operation will be restored through the backup program. This solution provides a more comprehensive disaster recovery mechanism by combining re-executing the OTA update with the backup program recovery to ensure the reliability of the OTA update.

[0033] In this embodiment, detailed logs are recorded for each OTA update and backup program recovery operation to facilitate subsequent analysis and troubleshooting.

[0034] In this embodiment, the process of generating the network quality level includes:

[0035] Obtain network data streams from the target network device, including packet loss rate (packet loss rate in the past 5 minutes), latency fluctuation (average latency standard deviation), number of connection interruptions (number of disconnections in the past hour), download rate (average rate of the most recent OTA), and upload rate (average rate of device status reports).

[0036] The network stability score is calculated based on the weighted sum of packet loss rate, latency fluctuation, and number of connection interruptions. The weight of packet loss rate is 0.4, the weight of latency fluctuation is 0.3, and the weight of number of connection interruptions is 0.3.

[0037] The network speed score is calculated based on the weighted sum of download rate and upload rate, where the download rate has a weight of 0.6 and the upload rate has a weight of 0.4.

[0038] A comprehensive score is calculated based on the weighted sum of the network stability score and the network speed score, where the network stability score has a weight of 0.7 and the network speed score has a weight of 0.3;

[0039] A corresponding network quality level is set for each preset quality level interval, and the current network quality level is determined according to the mapping relationship between the comprehensive score and the preset quality level interval.

[0040] It is worth noting that in step S2, the preset status file is detected through the recovery process or the exit status code of the OTA update process is detected through the recovery process. If the status file indicates that the update is not completed or the exit status code is a value corresponding to the update failure, the OTA update is judged to have failed.

[0041] In this embodiment, when performing OTA upgrades on multiple industrial devices, multiple servers are set up to locate OTA update failures. The steps include:

[0042] Binary code all industrial equipment into groups, so that each industrial equipment is given a unique binary code;

[0043] In binary encoding, each binary bit's two states (0 or 1) are bound to different servers. For example, the k-th binary bit's state 0 is bound to one server, the k-th binary bit's state 1 is bound to one server, and the m-th binary bit's state 1 is also bound to one server.

[0044] The industrial device communicates with the server corresponding to its own binary code. For example, if the binary code of the industrial device is 101, the industrial device will communicate with the server bound to the first binary bit with a state of 1, the server bound to the second binary bit with a state of 0, and the server bound to the third binary bit with a state of 1. When the OTA update is successful, the industrial device will send a success signal to the server it is communicating with, such as returning 1. When the OTA update fails, the industrial device will send a failure signal to the server it is communicating with, such as returning 0.

[0045] After receiving the failure signal, the server generates a feedback signal. The host computer sorts the servers with feedback signals according to the binary bits bound to the servers themselves, obtains the binary code of the industrial device that failed the OTA update, and locates the industrial device that failed the OTA update. For example, if an industrial device with a binary code of 101 fails the OTA update, it will send a failure signal to the server bound to the server with the first binary bit in the state of 1, the server bound to the second binary bit in the state of 0, and the server bound to the third binary bit in the state of 1. After receiving the failure signal, these three servers generate a feedback signal to the host computer. The host computer traces the feedback signal to the server that sent the feedback signal, obtains the binary bit bound to the server, and obtains the binary bit state (0 or 1). Then, based on the binary bit and the binary bit state, it reconstructs the binary code to obtain the binary code of 101, thereby locating the corresponding industrial device.

[0046] Preferably, the failure of the device to start is usually caused by system crash, kernel error or file system damage. In step S2, the startup flag file is detected by the recovery process or the running status of the device's key services is detected by the recovery process to determine whether the device has started successfully.

[0047] Before the OTA update, the system starts an independent recovery process and keeps it running. The specific process is as follows:

[0048] Write a recovery process program (such as recovery_daemon) in C, C++, or other languages. This program needs to include monitoring logic and OTA re-triggering logic to create an executable file for the recovery process.

[0049] Create a systemd service file in the / etc / systemd / system / directory. The code is as follows:

[0050] [Unit] Description=Recovery Daemon for OTA Updates After=network.target [Service]Type=simple ExecStart= / usr / bin / recovery_daemonRestart=always RestartSec=5 [Install] WantedBy=multi-user.target

[0051] In this embodiment, the recovery process is an independent daemon responsible for monitoring the OTA update status and device boot status. The recovery process has a high priority, ensuring normal operation in the event of device anomalies. This independent recovery process enables automated repair of OTA update failures, reducing manual intervention.

[0052] Specifically, the update failure and device boot failure detection mechanisms are combined to form a comprehensive detection mechanism. The recovery process can run according to the following logic:

[0053] Detection at startup: Check the startup flag file or key services to determine whether the device starts normally;

[0054] Update detection: Check the OTA update status file or exit status code to determine whether the update is successful;

[0055] Recovery logic: Re-trigger the OTA update if update failure is detected or the device cannot boot.

[0056] If the OTA update is successful, the device boots up using the new version of the program.

[0057] Optionally, in step S2, the step of detecting a preset status file through the recovery process includes:

[0058] At the start of an OTA update, a status file (e.g., / var / ota_in_progress) is created to indicate that the update is in progress. After the update is successfully completed, the status file is deleted and a first success flag file (e.g., / var / ota_success) is created. If the update fails, the status file is retained.

[0059] After the OTA update is completed, when the recovery process detects the status file, if the status file exists and the first success flag file does not exist, the OTA update is judged to have failed. If the first success flag file exists, the OTA update is judged to have succeeded.

[0060] The code is as follows:

[0061] #include<stdio.h>

[0062] #include<unistd.h>

[0063] #include<sys / stat.h>

[0064] #define OTA_IN_PROGRESS_FILE " / var / ota_in_progress"

[0065] #define OTA_SUCCESS_FILE " / var / ota_success"

[0066] int is_ota_failed() {

[0067] if(access(OTA_IN_PROGRESS_FILE,F_OK) == 0&&access(OTA_SUCCESS_FILE,F_OK) != 0) {

[0068] return 1; / / OTA failed

[0069] }

[0070] return 0; / / OTA succeeded or not started

[0071] }

[0072] Specifically, in step S2, the step of detecting the exit status code of the OTA update process through the recovery process includes: in the OTA update program, assigning values ​​corresponding to update success and update failure to the exit status code; the recovery process obtains the exit status code of the OTA update process through waitpid or a similar mechanism; if the exit status code is a value corresponding to update success, the OTA update is judged to be successful; if the exit status code is a value corresponding to update failure, the OTA update is judged to be failed.

[0073] The code is as follows:

[0074] #include<stdio.h>

[0075] #include<stdlib.h>

[0076] #include<unistd.h>

[0077] #include<sys / wait.h>

[0078] void run_ota_update() {

[0079] pid_t pid = fork();

[0080] if (pid == 0) {

[0081] / / Child process: run OTA update

[0082] execl(" / usr / bin / ota_update", "ota_update", NULL);

[0083] exit(1); / / If execl fails

[0084] } else if (pid>0) {

[0085] / / Parent process: wait for OTA update to complete

[0086] int status;

[0087] waitpid(pid,&status, 0);

[0088] if (WIFEXITED(status)&&WEXITSTATUS(status) != 0) {

[0089] printf("OTA update failed with status %d\n",WEXITSTATUS(status));

[0090] }

[0091] }

[0092] }

[0093] It is worth noting that, in step S2, the step of detecting the startup flag file through the recovery process includes:

[0094] When the system starts, a startup flag file (such as / var / boot_in_progress) is created. After the system is fully started, the startup flag file is deleted and a second success flag file (such as / var / boot_success) is created.

[0095] After the system starts and a preset time has passed, when the recovery process detects the startup flag file, if the startup flag file exists and the second success flag file does not exist, the device startup is judged to have failed; if the second success flag file exists, the device startup is judged to have succeeded.

[0096] The code is as follows:

[0097] #include<stdio.h>

[0098] #include<unistd.h>

[0099] #include<sys / stat.h>

[0100] #define BOOT_IN_PROGRESS_FILE " / var / boot_in_progress"

[0101] #define BOOT_SUCCESS_FILE " / var / boot_success"

[0102] int is_boot_failed() {

[0103] if (access(BOOT_IN_PROGRESS_FILE,F_OK)==0&&access(BOOT_SUCCESS_FILE, F_OK) != 0) {

[0104] return 1; / / Boot failed

[0105] }

[0106] return 0; / / Boot succeeded or not started

[0107] }

[0108] Preferably, in step S2, when the running status of the device's key services (such as network services or file system services, etc.) is detected through the recovery process, if the device's key services are not running, the device startup is judged to have failed; if the device's key services are running, the device startup is judged to have succeeded.

[0109] The code is as follows:

[0110] #include<stdio.h>

[0111] #include<stdlib.h>

[0112] int is_service_running(const char *service_name) {

[0113] char command

[256] ;

[0114] snprintf(command, sizeof(command), "systemctl is-active --quiet %s", service_name);

[0115] return system(command) == 0;

[0116] }

[0117] int is_boot_failed() {

[0118] If(!is_service_running("network.service")

[0119] || !is_service_running("filesystem.service")) {

[0120] return 1; / / Boot failed

[0121] }

[0122] return 0; / / Boot succeeded

[0123] }

[0124] Optionally, in step S2, whether the device is started is detected by a hardware watchdog timer. If the hardware watchdog timer is not reset regularly, it is determined that the device startup has failed.

[0125] Use the hardware watchdog timer to detect whether the system is running normally. If the system does not reset the watchdog regularly, the hardware watchdog timer will trigger a restart to detect whether the device fails to start. The code is as follows:

[0126] #include<stdio.h>

[0127] #include<unistd.h>

[0128] #include<fcntl.h>

[0129] #define WATCHDOG_DEVICE " / dev / watchdog"

[0130] void feed_watchdog() {

[0131] int fd = open(WATCHDOG_DEVICE, O_WRONLY);

[0132] if (fd>= 0) {

[0133] write(fd, "1", 1); / / Feed the watchdog

[0134] close(fd);

[0135] }

[0136] }

[0137] Specifically, in step S1, before executing the OTA update, the program file and related configuration files are obtained from the path where the currently running program is located, a backup program is generated and stored in a designated backup path, and the storage location of the backup program is recorded after the storage is completed.

[0138] Before an OTA update, the system backs up all currently running programs, configuration files, and dependent libraries to a specified path. The backup path can be local storage (such as a hard drive or flash memory) or remote storage (such as a server). When the device boots, the system checks whether the main program can start normally. If the main program fails to start, the system automatically switches to the program in the backup path.

[0139] Specifically, in step S3, the steps for re-executing the OTA update are: calling the OTA update tool through the recovery process to re-acquire the update package and execute the OTA update; during the execution of step S3, the number of OTA update executions is detected. When the number of OTA update executions exceeds the update threshold, it is detected that the OTA update failed or the device failed to start, and the backup program is selected to restore the device operation. If the new OTA update still fails, the recovery process can be attempted multiple times (up to the update threshold). If multiple attempts fail, the recovery process can switch to the backup program and notify the operation and maintenance personnel. This solution backs up the current program before executing the update, and uses the backup program to automatically restore the device by switching to the backup file after multiple update failures.

[0140] In this embodiment, the recovery process will re-download the update package and verify its integrity. If the update package is complete, the recovery process will re-execute the OTA update process.

[0141] In step S4, the OTA update process status is detected. If the OTA update still fails after being re-triggered multiple times, the number of failures reaching the threshold is recorded and a status identifier is generated. According to the status identifier, the system is switched to the backup path and the program code in the backup file is loaded. The device is started through the backup path, the device operation log is obtained, and it is determined whether the startup is successful. If the device is successfully started, the key parameters in the operation log are extracted to determine the current operating status of the system. The operating status data is used to generate an alarm message, which includes the number of failures and the device identifier. The alarm message is transmitted to the operation and maintenance system through the system communication module to obtain a transmission confirmation signal. If the transmission confirmation signal is not received, the alarm message is retried and the number of retries is recorded. According to the number of retries, the sending frequency is adjusted to determine whether the alarm message is successfully delivered. The feedback data from the operation and maintenance system is obtained, the local log is updated, and it is determined that the alarm processing is completed.

[0142] For example, when monitoring the status of an OTA update, the system uses a polling mechanism to check the update progress every 5 seconds. If an update fails or the device fails to boot three times in a row, the failure threshold is determined to have been reached and the "OTA_FAILED" status indicator is generated. Based on this indicator, the system automatically switches to a predefined backup path, loads the backup program, and verifies its integrity. After extracting key parameters from the log, such as memory usage and CPU load, the system generates an alarm. The communication module uses the MQTT protocol to push the alarm to the operation and maintenance system address. If no ACK response is received within 5 seconds, retries are performed using an exponential backoff algorithm (e.g., intervals of 1s, 2s, or 4s). If the number of retries exceeds three, the call is automatically switched to a TCP persistent connection and retransmitted. When the operation and maintenance system returns the processing result, the timestamp and processing status are appended to the local log file.

[0143] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It is apparent to those skilled in the art that various changes, modifications, substitutions, and variations to these embodiments may be made without departing from the principles and spirit of the present invention, and these changes and modifications still fall within the scope of protection of the present invention.

Claims

1. A disaster recovery method during the OTA process of industrial equipment software, characterized in that: The following steps are involved: S1: Get the current running program and generate a backup program, and perform an OTA update to replace the current running program; S2: Detecting the OTA update status and device startup status. If the OTA update failure or device startup failure is detected, the network quality level of the current industrial device is detected; if the network quality level of the current industrial device is higher than the set threshold, step S3 is executed; if the network quality level of the current industrial device is lower than or equal to the set threshold, step S4 is executed; In step S2, a preset status file is detected by the recovery process, or an exit status code of the OTA update process is detected by the recovery process. If the status file indicates that the update is not completed or the exit status code is a value corresponding to an update failure, it is determined that the OTA update has failed. In step S2, the step of detecting a preset status file through a recovery process includes: When the OTA update starts, a status file is preset; after the update is successfully completed, the status file is deleted and a first success flag file is created; When the recovery process detects the status file, if the status file exists and the first success flag file does not exist, the OTA update is judged to have failed; if the first success flag file exists, the OTA update is judged to have succeeded; Among them, all industrial devices are grouped into binary codes, and each industrial device is assigned a unique binary code. In the binary code, the two states of each binary bit are bound to different servers. The industrial device communicates with the server corresponding to its own binary code. When the OTA update is successful, the industrial device will send a success signal to the server it is communicating with. When the OTA update fails, the industrial device will send a failure signal to the server it is communicating with. After receiving the failure signal, the server will generate a feedback signal. The upper computer sorts the servers with feedback signals according to the binary bits bound to the servers themselves, and obtains the binary codes of the industrial devices that failed the OTA update, so as to locate the industrial devices that failed the OTA update. S3: Re-execute OTA update; S4: Select the backup program to restore the device.

2. The method for disaster recovery during an OTA update of industrial equipment software according to claim 1, characterized in that: In step S2, the startup flag file is detected by the recovery process or the running status of the key service of the device is detected by the recovery process to determine whether the device is successfully started.

3. The method for disaster recovery during an OTA process of industrial equipment software according to claim 1, characterized in that: In step S2, the step of detecting the exit status code of the OTA update process through the recovery process includes: in the OTA update program, assigning values ​​corresponding to update success and update failure to the exit status code; the recovery process obtains the exit status code of the OTA update process; if the exit status code is a value corresponding to update success, it is judged that the OTA update is successful; if the exit status code is a value corresponding to update failure, it is judged that the OTA update has failed.

4. The method for disaster recovery during an OTA process of industrial equipment software according to claim 2, characterized in that: In step S2, the step of detecting the startup flag file through the recovery process includes: When the system starts, a startup flag file is created; after the system is completely started, the startup flag file is deleted and a second success flag file is created; When the recovery process detects the startup flag file, if the startup flag file exists and the second success flag file does not exist, it is determined that the device startup fails; if the second success flag file exists, it is determined that the device startup is successful.

5. The method for disaster recovery during an OTA process of industrial equipment software according to claim 2, characterized in that: In step S2, when detecting the running status of the key service of the device through the recovery process, if the key service of the device is not running, it is determined that the device startup has failed; if the key service of the device is running, it is determined that the device startup has succeeded.

6. The method for disaster recovery during an OTA process of industrial equipment software according to claim 1, characterized in that: In step S2, whether the device is started is detected by a hardware watchdog timer. If the hardware watchdog timer is not reset regularly, it is determined that the device has failed to start.

7. The method for disaster recovery during an OTA process of industrial equipment software according to claim 1, characterized in that: In step S1, before executing the OTA update, the program file and related configuration files are obtained from the path where the currently running program is located, a backup program is generated and stored in the specified backup path, and the storage location of the backup program is recorded after the storage is completed.

8. The method for disaster recovery during an OTA process of industrial equipment software according to claim 1, characterized in that: In step S3, the step of re-performing the OTA update is: calling the OTA update tool through the recovery process to re-acquire the update package and perform the OTA update; During the execution of step S3, the number of times the OTA update is executed is detected. When the number of times the OTA update is executed exceeds the update threshold, it is detected that the OTA update has failed or the device has failed to start, and a backup program is selected to restore the device operation.

Images