Web management and AI inference independent running method based on process isolation

Abstract

The application discloses an independent running method based on process isolation, web management and AI inference, which comprises the following steps: starting a web management service and an AI inference worker process running independently according to a camera channel on an edge computing device, and making the AI inference worker process enter an independent process group; reading a video stream, executing AI inference and generating a heartbeat file, a video frame file and an alarm record according to a channel configuration by the AI inference worker process; recording life cycle information and keeping the AI inference worker process running when the web management service exits; after the web management service is restarted, carrying out process survival, configuration consistency and heartbeat aging verification by combining the heartbeat file, channel configuration data and an operating system process table, re-registering the effective process as a current running process of the corresponding channel, or cleaning up abnormal processes and starting new processes, and recovering page display, video output and alarm reporting processing.

Description

Technical Field

[0001] This application relates to the field of process management technology, specifically a method for independent operation of web management and AI inference based on process isolation. Background Technology

[0002] With the development of edge computing and video intelligent analysis technologies, AI algorithm boxes with local inference capabilities have been widely used in scenarios such as park security, factory inspection, and construction supervision. These devices typically connect multiple RTSP video streams to the same edge computing device, and complete camera configuration, algorithm selection, operation control, image viewing, and alarm querying through a web-based backend. The local AI inference program performs target detection, alarm determination, and alarm record generation on the video frames of each channel. In current implementations, the web management service often simultaneously handles page management, process initiation, runtime status maintenance, and alarm reporting. The AI ​​inference program is usually directly started by the web management service and is affected by its lifecycle.

[0003] When the Web management service exits due to version updates, configuration adjustments, abnormal crashes, or manual restarts, running inference tasks are easily terminated, leading to interrupted video detection, missed alarm records, or loss of page status. Even if some inference processes continue to run in the background, it is difficult to accurately determine whether the original inference processes are still alive, whether they correspond to the current channel configuration, and whether the heartbeat data is valid after the Web management service restarts. This can result in problems such as duplicate streaming, duplicate inference, continued running of processes with old configurations, confusion in channel ownership, and difficulty in tracing the source of alarms. Therefore, there is an urgent need for an operation management mechanism that can isolate the Web management service from AI inference tasks and reliably restore channel ownership and page display status after the Web service restarts. Summary of the Invention

[0004] The purpose of this application is to provide a method and system for independent operation of Web management and AI inference based on process isolation, so as to solve the problems mentioned in the background art.

[0005] According to one aspect of this application, a method for independent operation of web management and AI inference based on process isolation is provided, comprising the following steps: On the edge computing device, a web management service and an AI inference worker process that runs independently according to camera channel are started respectively, and the AI ​​inference worker process is put into an independent process group. According to the channel configuration data, the AI ​​inference worker process is controlled to read the corresponding video stream, execute AI inference, and generate heartbeat files, video frame files, and alarm records. The heartbeat files contain channel running identity information. When the Web management service exits, an exit record is written to the Web lifecycle table, and the AI ​​inference worker process with a valid heartbeat continues to run; After the Web management service is restarted, the heartbeat file, channel configuration data and operating system process table are read, and the AI ​​inference worker process is sequentially checked for process liveness, configuration consistency and heartbeat validity. Based on the verification results, the AI ​​inference worker processes that pass the verification will be re-registered as the current running processes of the corresponding camera channels, or the processes and files that fail the verification will be cleaned up and the corresponding AI inference worker processes will be restarted according to the current channel configuration data. The operation status table and operation attribution table are updated based on the re-registration results, and the page display, video stream output and alarm reporting processing are restored according to the operation status table, video frame files and alarm records.

[0006] Preferably, the channel configuration data includes at least the channel number, RTSP address de-identification digest, algorithm identifier, alarm threshold, activation status, and configuration version number. After saving the camera configuration or algorithm configuration, the Web management service updates the configuration version number and transmits the updated channel configuration data to the corresponding AI inference worker process.

[0007] Preferably, the channel operation identity information includes channel number, AI inference worker process number, AI inference worker process group number, start time, configuration version number, algorithm identifier, RTSP address de-identification digest, lease sequence number and timestamp, and the AI ​​inference worker process increments the lease sequence number each time it writes to the heartbeat file.

[0008] Preferably, the AI ​​inference worker process first writes the heartbeat content to a temporary heartbeat file, and then renames the temporary heartbeat file to the heartbeat file; the AI ​​inference worker process first writes the video frame content to a temporary video frame file, and then renames the temporary video frame file to the video frame file; the Web management service only reads the renamed heartbeat file and the video frame file.

[0009] Preferably, the process liveness verification includes: the Web management service queries the operating system process table based on the AI ​​inference worker process number, reads the command line parameters and process group number of the corresponding process, compares the channel number and the process group number in the command line parameters with the channel number and AI inference worker process group number in the heartbeat file, and determines whether the heartbeat file belongs to the live AI inference worker process of the corresponding camera channel based on the comparison result.

[0010] Preferably, the configuration consistency verification includes: the Web management service comparing the configuration version number, algorithm identifier, and RTSP address anonymization digest in the heartbeat file with the configuration version number, algorithm identifier, and RTSP address anonymization digest in the current channel configuration data item by item; if all three items are consistent, the heartbeat timeliness verification is initiated; if any item is inconsistent, the corresponding independent process group is terminated, and a new AI inference worker process is started according to the current channel configuration data after the heartbeat file stops updating.

[0011] Preferably, the heartbeat validity period verification includes timestamp verification and lease sequence number verification; the timestamp verification determines whether the heartbeat has timed out based on the current time, the timestamp in the heartbeat file, and the heartbeat validity period threshold; the lease sequence number verification determines whether the lease sequence number is incremented by continuously reading the heartbeat file; when both are passed, the corresponding AI inference worker process is selected as a candidate process.

[0012] In another aspect, this application also provides an independently operating system for web management and AI inference based on process isolation, comprising: The process startup module is used to start the Web management service and the AI ​​inference worker process that runs independently by camera channel on the edge computing device, and to put the AI ​​inference worker process into an independent process group. The inference execution module is used to control the AI ​​inference worker process to read the corresponding video stream, execute AI inference, and generate heartbeat files, video frame files, and alarm records according to the channel configuration data. The heartbeat file contains channel operation identity information. The exit retention module is used to write an exit record to the Web lifecycle table when the Web management service exits, and retain the AI ​​inference worker process with a valid heartbeat to continue running; The restart verification module is used to read the heartbeat file, channel configuration data and operating system process table after the Web management service is restarted, and to perform process survival verification, configuration consistency verification and heartbeat validity verification on the AI ​​inference worker process in sequence. The process recovery module is used to re-register the AI ​​inference worker process that has passed the verification as the current running process of the corresponding camera channel based on the verification result, or to clean up the processes and files that have failed the verification and restart the corresponding AI inference worker process according to the current channel configuration data. The status recovery module is used to update the running status table and running attribution table based on the re-registration results, and to restore page display, video stream output and alarm reporting processing according to the running status table, video frame files and alarm records.

[0013] This application also provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the independent operation method of Web management and AI inference based on process isolation as described above.

[0014] In another aspect, this application provides a storage medium storing computer program instructions that can be executed by a processor to implement the process-isolated, independently running method for web management and AI inference as described above.

[0015] This application runs the Web management service and the AI ​​inference worker processes for each camera channel separately, and places the AI ​​inference worker processes into independent process groups. When the Web management service exits or restarts, it does not directly interrupt ongoing video streaming, AI inference, and alarm record generation, thus reducing the risk of detection gaps caused by background maintenance, abnormal exits, or version updates. By writing the channel number, process number, process group number, configuration version number, algorithm identifier, RTSP address anonymized digest, lease sequence number, and timestamp into the heartbeat file, and combining this with the operating system process table and current channel configuration data for process liveness verification, configuration consistency verification, and heartbeat validity verification, it can accurately distinguish between valid running processes, old configuration processes, and expired residual files. After the Web management service restarts, only the verified AI inference worker processes are re-registered as the current running processes for the corresponding channels. Processes and files that fail verification are cleaned up before being restarted with the current configuration. Therefore, it can reduce duplicate streaming, duplicate inference, mismatched alarm sources, and confused channel affiliation, improving the continuity, maintainability, and operational status recovery reliability of the multi-channel video early warning system. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram illustrating an independent operation method for Web management and AI inference based on process isolation, provided in an embodiment of this application.

[0018] Figure 2 This is a schematic diagram of an independently operating system architecture for web management and AI inference based on process isolation, provided as an embodiment of this application.

[0019] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] It should be noted that all user information (including but not limited to user device information, user personal information, object information corresponding to device usage data, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, device usage data, etc.) involved in all embodiments of this application are information and data authorized by the user or fully authorized by all parties.

[0022] This implementation method is applicable to multi-channel video early warning systems deployed on the same edge computing device. The edge computing device can be an AI algorithm box with a processor, neural network acceleration unit, local storage, and web backend. On-site cameras are connected via RTSP video streams, and camera channels are distinguished as CH01 to CH08. The web management service is responsible for camera management, algorithm configuration, operation control, alarm query, reporting configuration, and system information display. The AI ​​inference worker process runs independently for each camera channel, responsible for reading the corresponding video stream, performing AI inference, writing heartbeat files and video frame files, and generating alarm records. Alarm records are stored in a local SQLite database, while video frame files and heartbeat files are stored in a shared memory directory. A browser can view channel images, operating status, and alarm records through the web management service.

[0023] The edge computing device runs a Linux operating system. The Web management service is started in a single-process manner, and the AI ​​inference worker processes are launched by the Web management service according to each channel. The SQLite database includes at least the following tables: camera configuration table, algorithm configuration table, Web lifecycle table, running status table, running attribution table, and alarm record table. The camera configuration table stores the channel number, RTSP address, enabled status, and channel name; the algorithm configuration table stores the algorithm identifier, alarm threshold, and configuration version number; the Web lifecycle table stores the start time, exit time, and exit reason of the Web management service; the running status table stores the channel number, running status, frame rate, inference time, number of alarms, last heartbeat time, AI inference worker process number, and AI inference worker process group number; the running attribution table stores the AI ​​inference worker process currently assigned to each camera channel; and the alarm record table stores the alarm time, channel number, alarm type, alarm image path, reporting status, and the identity information of the AI ​​inference worker process that generated the alarm record.

[0024] The following detailed description, in conjunction with specific embodiments, illustrates the implementation process of the independent operation method for Web management and AI inference based on process isolation described in this application. It should be noted that these embodiments are merely for explaining this application and not for limiting its scope of protection. Any conventional adjustments or substitutions made by those skilled in the art to the steps without departing from the concept of this application should be included within the scope of protection of this application.

[0025] like Figure 1 As shown in the figure, this application discloses a schematic diagram of an independent operation method for Web management and AI inference based on process isolation, including the following method steps: S1, Start the Web management service and the AI ​​inference worker process that runs independently according to the camera channel on the edge computing device, and put the AI ​​inference worker process into an independent process group. S2, according to the channel configuration data, control the AI ​​inference worker process to read the corresponding video stream, execute AI inference and generate heartbeat file, video frame file and alarm record, the heartbeat file contains channel running identity information; S3, when the Web management service exits, write an exit record to the Web lifecycle table, and retain the AI ​​inference worker process with a valid heartbeat to continue running; S4. After the Web management service is restarted, read the heartbeat file, channel configuration data and operating system process table, and perform process survival verification, configuration consistency verification and heartbeat timeliness verification on the AI ​​inference worker process in sequence. S5. Based on the verification result, the AI ​​inference worker process that passed the verification is re-registered as the current running process of the corresponding camera channel, or the processes and files that failed the verification are cleaned up and the corresponding AI inference worker process is restarted according to the current channel configuration data. S6. Update the running status table and running attribution table based on the re-registration results, and restore page display, video stream output and alarm reporting processing according to the running status table, video frame file and alarm record.

[0026] In some embodiments, for step S1, in one implementation, the system service management program first starts the Web management service. After completing the database connection, camera configuration table reading, and algorithm configuration table reading, the Web management service filters camera channels whose activation status is "enabled". The "enabled status" refers to the channel switch value saved by the maintenance personnel on the operation control page or algorithm configuration page. The Web management service only starts the AI ​​inference worker process for camera channels whose activation status is "enabled" and whose algorithm identifier is not empty. The AI ​​inference worker process enters an independent process group. An independent process group refers to a collection of processes with an independent process group number. When the Web management service exits, the termination signal sent by the operating system to the Web management service process group will not be automatically transmitted to that independent process group.

[0027] When the Web management service starts the AI ​​inference worker process, it passes the channel configuration data to the corresponding process. The channel configuration data includes the channel number, RTSP address anonymized digest, algorithm identifier, alarm threshold, enabled status, and configuration version number. The RTSP address anonymized digest is a string obtained by digesting the camera IP address, port number, and video stream path after removing the username and password from the RTSP address. The digest calculation can use the SHA-256 digest algorithm. For example, if the original RTSP address contains the username, password, camera IP address, port number, and stream path, the Web management service first removes the username and password fields, then performs a SHA-256 digest on the remaining string to obtain a fixed-length digest value, which is then written to the camera configuration table. This digest value participates in subsequent configuration consistency checks and is not displayed to the page with the original account information.

[0028] The configuration version number refers to the version identifier of the channel configuration data. Specifically, after each time the Web management service saves the camera configuration or algorithm configuration, it increments the configuration version number of the corresponding camera channel by 1 and saves the incremented configuration version number along with the channel number, RTSP address anonymized digest, algorithm identifier, alarm threshold, and activation status. For example, when CH01 is first activated, its configuration version number is 1. After the operations and maintenance personnel change the algorithm identifier of CH01 from safety helmet detection to open flame and smoke detection, the configuration version number is updated to 2. When the AI ​​inference worker process starts, it reads the configuration version number as 2, and subsequently writes configuration version number 2 to the heartbeat file. The configuration version number does not participate in alarm determination independently, but it participates in the configuration consistency verification after the Web management service restarts.

[0029] In some embodiments, for step S2, after receiving the channel configuration data, the AI ​​inference worker process establishes a running directory according to the channel number, connects to the corresponding RTSP video stream, and selects the output category of the AI ​​detection model according to the algorithm identifier. The algorithm identifier includes safety helmet detection, reflective clothing detection, open flame and smoke detection, personnel gathering detection, or disabled. The alarm threshold refers to the lower limit of the target confidence required to generate an alarm record, which is obtained from the validation data statistics; for example, after collecting the corresponding on-site images and completing manual annotation, the precision, recall, and F1 score are calculated on the validation set with different target confidence thresholds, and the threshold with a higher F1 score and a number of false alarms that meet the on-site management requirements is selected and written into the algorithm configuration table. Optionally, when there is no on-site historical data, the initial alarm threshold can be obtained by using a validation set of similar scenarios first, and then updated weekly according to the results of manual review after the device is running; the channel restart process is still triggered by configuring the version number during the update.

[0030] In one embodiment, the AI ​​detection model employs a single-stage target detection model, exemplarily using the YOLOv5s target detection network. The model's total input consists of video frames decoded from the RTSP video stream by the AI ​​inference worker process and then scaled and padded. The data is a 3×640×640 three-dimensional tensor, where 3 represents the three color channels: red, green, and blue, and pixel values ​​are normalized to 0 to 1. The model's total output is a set of detection results, in N×6 format, where N represents the number of targets retained in the current frame after non-maximum suppression. Each row sequentially represents the x-coordinate of the top-left corner of the target bounding box, the y-coordinate of the top-left corner of the target bounding box, the x-coordinate of the bottom-right corner of the target bounding box, the y-coordinate of the bottom-right corner of the target bounding box, the category identifier, and the target confidence score. The category identifiers include: person not wearing a safety helmet, person not wearing reflective clothing, flame, smoke, and person.

[0031] The YOLOv5s object detection network consists of a backbone network, a feature fusion network, and a detection head. The backbone network is arranged in series along the data flow direction, including a Focus module, a convolutional module, a C3 residual module, and a SPPF module. The Focus module slices the input video frames at spatial intervals and then concatenates them along the channel dimension, with an output channel number of, for example, 32. The convolutional module uses a 3×3 convolutional kernel, a stride of 2, and padding of 1, with output channel numbers of, for example, 64, 128, 256, and 512. The C3 residual module includes a main branch and a side branch. The main branch contains a bottleneck convolutional layer, and the side branch retains the input features. The two features are concatenated along the channel dimension and then output through a 1×1 convolution. The SPPF module uses 5×5 max pooling layers stacked in series and concatenates the features before and after pooling along the channel dimension. The feature fusion network employs a path aggregation network, comprising upsampling, feature concatenation, and a C3 residual module, outputting feature maps at three scales: 80×80, 40×40, and 20×20. The detection head performs convolutional prediction on these three scale feature maps to obtain bounding boxes, class labels, and target confidence scores. The target appearance, contour, color, and local texture in the input video frame, after passing through convolutional layers and multi-scale feature fusion, correspond to the target bounding boxes and class probabilities of the detection head. Therefore, there is a spatial correspondence between the model output and the actual target region in the input video frame.

[0032] The training process uses the same data format as the inference process. Training images are sourced from camera footage of construction sites, factory entrances and exits, warehouse areas, and server rooms. Annotations include bounding boxes and category labels. The loss function comprises bounding box loss, target confidence loss, and category loss, combined with weighted coefficients. Each weighted coefficient represents the proportion of each loss term in the total loss, determined by normalizing the average value of each loss term in the initial training phase, and ranges from 0 to 1. The sum of the three weighted coefficients is 1. The AdamW optimizer can be used during training, with an initial learning rate of, for example, 0.001, adjusted using cosine annealing. Batch sizes range from 8 to 32, and the number of training epochs ranges from 100 to 300. Training images can be randomly scaled, horizontally flipped, have varying brightness, and be enhanced with mosaic effects. Training stops when the validation set loss fails to decrease for several consecutive epochs.

[0033] The loss function can be expressed as:

[0034] in, Indicates the total loss; Indicates the bounding box loss; Indicates the target confidence loss; Indicates category loss; , , These represent the weight coefficients for the bounding box loss, target confidence loss, and class loss, respectively. Each loss term is calculated from normalized coordinates, probabilities, or class labels. The bounding box loss can use the complete intersection-union loss, the target confidence loss can use the binary cross-entropy loss, and the class loss can use the multi-class cross-entropy loss. As an example, if the average values ​​of the bounding box loss, target confidence loss, and class loss in several batches during the initial training phase are 0.08, 0.04, and 0.04, respectively, then the reciprocals of each average value can be taken first, and then the reciprocals can be normalized by summing them to obtain the three weight coefficients, which are then written into the training configuration file.

[0035] After obtaining the set of detection results, the AI ​​inference worker process determines the alarm based on the algorithm identifier. For helmet detection, the AI ​​inference worker process filters detection results categorized as individuals not wearing helmets and whose target confidence reaches the alarm threshold; for reflective vest detection, it filters results categorized as individuals not wearing reflective vests and whose target confidence reaches the alarm threshold; for open flame and smoke detection, it filters results categorized as flames or smoke and whose target confidence reaches the alarm threshold; for crowd detection, it filters results categorized as individuals and whose target confidence reaches the alarm threshold, and generates an alarm record when the number of filtered individuals reaches the crowd gathering threshold. The crowd gathering threshold is determined by on-site management rules, for example, it can be configured to 2 people, and the configuration version number is updated synchronously when this threshold is modified.

[0036] In one embodiment, the AI ​​inference worker process also generates a heartbeat file at a fixed write cycle. Channel operation identity information is written to the heartbeat file, including the channel number, AI inference worker process number, AI inference worker process group number, start time, configuration version number, algorithm identifier, RTSP address anonymized digest, lease sequence number, and timestamp. The lease sequence number is an integer that increments each time the AI ​​inference worker process writes to the heartbeat file; for example, 1 for the first write and 2 for the next. The timestamp is the system time at the time of writing to the heartbeat file, in seconds. The heartbeat file uses JSON format, and the web management service obtains the operating status of the corresponding camera channel by reading this heartbeat file.

[0037] In practice, the AI ​​inference worker process first writes the heartbeat content to a temporary heartbeat file, then refreshes the write buffer and renames the temporary heartbeat file to the official heartbeat file. Video frame files undergo the same processing: the AI ​​inference worker process first encodes the video frames with detection boxes into JPEG format and writes them to a temporary video frame file, then renames the temporary video frame file to the official video frame file. The web management service only reads the official heartbeat file and the official video frame file.

[0038] As an example, when the AI ​​inference worker process of CH01 writes the temporary heartbeat file, it includes the channel number CH01, AI inference worker process number 2451, AI inference worker process group number 2451, configuration version number 3, algorithm identifier FIRE, lease sequence number 105, and timestamp. After renaming, the Web management service reads the official heartbeat file and updates the running status table. Temporary file writing and renaming are common atomic update methods used in file systems, and the Web management service will not read heartbeat content with incomplete fields.

[0039] When an alarm record is generated, the AI ​​inference worker process writes the alarm time, channel number, alarm type, alarm image path, reporting status, AI inference worker process number, AI inference worker process group number, configuration version number, and algorithm identifier into the alarm record table. The alarm image path is generated according to the channel number and date; for example, the alarm image for CH01 on a certain day is stored in the directory corresponding to that channel's date. The reporting status is initially set to "pending reporting," and the alarm reporting service reads this field and sends the alarm data to the business server.

[0040] In some embodiments, for step S3, after receiving a restart command or process exit signal, the Web management service first stops receiving new page requests, and then writes an exit record to the Web lifecycle table. The exit record includes the Web management service exit time, exit reason, and Web instance number. The Web instance number refers to a local number generated each time the Web management service starts, which can be obtained by combining the startup time and process ID, and is used to distinguish different startup cycles. After writing the exit record, the Web management service reads the camera channels in the running status table and checks the timestamp and lease number of the corresponding heartbeat file; for AI inference worker processes whose heartbeat files are still valid, the Web management service does not send a termination signal to their independent process group.

[0041] The heartbeat validity period threshold refers to the upper limit in seconds for determining whether a heartbeat file still represents an active AI inference worker process. This threshold is determined by the heartbeat write cycle, heartbeat scan cycle, system scheduling jitter limit, and file write and renaming time limit, and can be obtained by the following formula:

[0042] in, This represents the heartbeat validity period threshold, in seconds. This indicates the heartbeat write cycle of the AI ​​inference worker process, in seconds. This indicates the heartbeat scan cycle of the Web management service, in seconds. This indicates the upper limit of system scheduling jitter, in seconds; This indicates the maximum time limit for writing and renaming the heartbeat file, in seconds. and The calibration can be performed when the device is first deployed. The calibration method is to continuously record the heartbeat write interval and file renaming time for a number of consecutive times, and take the 95th percentile value as the corresponding parameter. For example, when the heartbeat write cycle is 1 second, the heartbeat scan cycle is 3 seconds, the system scheduling jitter limit is 0.5 seconds, and the file write and renaming time limit is 0.2 seconds, the heartbeat validity threshold is taken as the larger of 3.7 seconds and 3 seconds according to the above rules, i.e., 3.7 seconds.

[0043] During the period following the exit of the Web management service, the AI ​​inference worker process continues to read the corresponding RTSP video stream, continues to execute AI inference, continues to update the heartbeat file and video frame file according to the aforementioned temporary file writing and renaming method, and continues to write alarm records to the alarm log table. This approach addresses the on-site issue that when maintenance personnel restart the web backend or the web backend exits abnormally, the safety helmet detection, reflective vest detection, open flame and smoke detection, and personnel gathering detection of the camera channel are not simultaneously stopped due to the exit of the web backend process; since the alarm log table is directly written by the AI ​​inference worker process, alarm records generated during the exit period still retain the channel number, alarm type, and AI inference worker process identity information.

[0044] In some embodiments, for step S4, after the Web management service restarts, it first establishes an SQLite database connection and reads the current channel configuration data from the camera configuration table and algorithm configuration table; then it scans the heartbeat files in the shared memory directory and matches the heartbeat files with the camera channels whose activation status is enabled according to the channel number. If an enabled channel does not have a corresponding heartbeat file, the Web management service adds the camera channel to the set to be reactivated; if a heartbeat file exists but lacks any field in the channel running identity information, the Web management service treats the heartbeat file as an invalid file and deletes the corresponding video frame file.

[0045] The process liveness verification is performed by the Web management service. The Web management service queries the operating system process table based on the AI ​​inference worker process ID in the heartbeat file, reads the command-line parameters and process group ID of the process, and compares the channel number in the command-line parameters and the process group ID in the operating system process table with the channel number and AI inference worker process group ID in the heartbeat file, respectively.

[0046] For example, the heartbeat file records the channel number CH02, the AI ​​inference worker process number 2510, and the AI ​​inference worker process group number 2510. After the Web management service reads the operating system process table, if process number 2510 exists, the command line parameters include CH02, and the process group number is also 2510, then the heartbeat file passes the process liveness check. If the process number does not exist, it means that the heartbeat file comes from an AI inference worker process that has exited. The Web management service deletes the heartbeat file and the corresponding video frame file, and adds CH02 to the set to be restarted.

[0047] The configuration consistency check is performed after the process liveness check passes. The Web management service compares the configuration version number, algorithm identifier, and RTSP address anonymized digest in the heartbeat file with the configuration version number, algorithm identifier, and RTSP address anonymized digest in the current channel configuration data item by item. If all three are consistent, it means that the configuration of the AI ​​inference worker process is consistent with the current configuration in the database, and the Web management service enters the heartbeat validity check. If any item is inconsistent, the Web management service sends a termination signal to the corresponding independent process group and waits for the corresponding heartbeat file to stop updating. Then, it starts a new AI inference worker process according to the current channel configuration data. For example, if the configuration version number in the CH03 heartbeat file is 4, while the configuration version number of CH03 in the algorithm configuration table is 5, it means that the channel configuration has been modified but the old AI inference worker process is still running. The Web management service first terminates the old AI inference worker process and then starts a new AI inference worker process according to configuration version number 5.

[0048] Heartbeat validity verification includes timestamp verification and lease sequence number verification. Timestamp verification determines whether the heartbeat has timed out based on the current time, the timestamp in the heartbeat file, and the heartbeat validity threshold; lease sequence number verification determines whether the lease sequence number increments by continuously reading the heartbeat file.

[0049] Specifically, the Web management service reads the heartbeat file for the first time to obtain the lease sequence number. After waiting for one heartbeat write cycle, it reads the same heartbeat file again. If the lease sequence number read the second time is greater than the lease sequence number read the first time, and the timestamp of the second read still does not exceed the heartbeat validity period threshold, then the corresponding AI inference worker process passes the heartbeat validity period verification.

[0050] For example, if the lease sequence number is 220 when read for the first time, and then 221 is read again after waiting for 1 second, and the difference between the current time and the timestamp in the heartbeat file is 0.4 seconds, which is less than the heartbeat validity period threshold of 3.7 seconds, then the AI ​​inference worker process can be used as a candidate process. If the second read is still 220, the Web management service will wait for another heartbeat write cycle and then check again. If the check still does not increase, the camera channel will be added to the set to be restarted.

[0051] When multiple candidate processes exist for the same camera channel, the Web management service selects the currently running process in a specific order. First, it compares the lease numbers of each candidate process, selecting the one with the later lease number as the currently running process. If the lease numbers are the same, it compares the startup times, selecting the one with the later startup time as the currently running process. Independent process groups containing unselected candidate processes are cleaned up by the Web management service sending a termination signal.

[0052] In some embodiments, for step S5, the AI ​​inference worker process that has passed process liveness verification, configuration consistency verification, and heartbeat timeliness verification is re-registered by the Web management service as the currently running process of the corresponding camera channel. During re-registration, the Web management service writes the channel number, AI inference worker process number, AI inference worker process group number, start time, most recent heartbeat time, configuration version number, and algorithm identifier into the running status table, and writes the correspondence between the channel number and the AI ​​inference worker process number and AI inference worker process group number into the running attribution table. The Web management service's memory running management structure also registers the same set of data; subsequent channel shutdowns, configuration modifications, or reading of running status are all based on the running attribution table and the running status table.

[0053] Cases failing verification are handled according to the reason for failure. When process liveness verification fails, the Web management service deletes the heartbeat file and video frame file, and starts a new AI inference worker process based on the current channel configuration data. When configuration consistency verification fails, the Web management service first terminates the independent process group containing the old AI inference worker process, waits for the old heartbeat file to stop updating, and then starts a new AI inference worker process based on the current channel configuration data. When heartbeat expiration verification fails, the Web management service deletes expired heartbeat files and video frame files, and restarts the corresponding AI inference worker process. As an example, if the CH04 heartbeat file exists but the corresponding process is no longer in the operating system process table, after deleting the CH04 heartbeat file and video frame file, the Web management service starts a new CH04 AI inference worker process based on the RTSP address de-identification digest in the camera configuration table, the algorithm identifier in the algorithm configuration table, and the alarm threshold.

[0054] Channel termination and configuration modification are also handled according to the running attribution table. When the Web management service receives a channel termination command, it first reads the AI ​​inference worker process group number of the corresponding camera channel from the running attribution table, then sends a termination signal to the corresponding independent process group, and updates the running status table after the heartbeat file stops updating. When the Web management service receives a channel configuration modification command, it first saves the new channel configuration data and updates the configuration version number, then terminates the old AI inference worker process, and starts a new AI inference worker process according to the new channel configuration data after the old heartbeat file stops updating. This sequence ensures that the database configuration, running attribution table, and actual process status change in the same control order, reducing the possibility of both old and new configuration processes existing simultaneously for the same camera channel.

[0055] The association between alarm records and the current channel's running identity information is performed after re-registration. The Web management service reads the channel number, configuration version number, and algorithm identifier from the alarm record table and associates them with the current channel's running identity information in the running status table. For alarm records generated during Web management service exit, since the alarm record table stores the AI ​​inference worker process number, AI inference worker process group number, configuration version number, and algorithm identifier, the Web management service can confirm the camera channel and running process corresponding to the alarm record after restarting. The alarm reporting service reads records with a reporting status of pending reporting or failed and awaiting retry from the alarm record table and uploads them by device ID, server address, and authentication key. For duplicate pending reporting records with the same channel number, alarm type, and alarm image path, the alarm reporting service retains the record with the earliest creation time and adds it to the reporting queue.

[0056] In some embodiments, for step S6, after the re-registration result is written to the running status table and the running attribution table, the overview page, running control page, system information page, and alarm record page of the Web management service read the same database status. The overview page displays the corresponding channel screen area according to the channel number and running status in the running status table; the running control page determines the process group that sent the signal when stopping or restarting the channel according to the running attribution table; the system information page reads the frame rate, inference time, number of alarms, and last heartbeat time; the alarm record page reads the alarm record table and filters it according to the channel number, alarm type, and time range.

[0057] When outputting the video stream, the Web management service first reads the running status table to confirm that a current process exists in the camera channel, and then reads the corresponding video frame file. The validity of the video frame file is determined based on the current time, the last modification time of the video frame file, and the heartbeat validity period threshold. Optionally, the time difference of the video frame file is obtained by the following formula:

[0058] in, This indicates the time difference between video frame files, in seconds. This represents the system time, in seconds, when the Web management service reads the video frame file. This indicates the last modification time of the video frame file, in seconds. The Web management service compares the video frame file time difference with the sum of the heartbeat validity period threshold and the video frame encoding write margin. The video frame encoding write margin refers to the time remaining from when the AI ​​inference worker process completes AI inference for one frame until the JPEG file is renamed, in seconds. It can be determined by statistically analyzing the 95th percentile value of this time.

[0059] As an example, if the current time is 100.0 seconds, the last modification time of the video frame file is 98.8 seconds, the time difference of the video frame file is 1.2 seconds, the heartbeat validity threshold is 3.7 seconds, and the video frame encoding write margin is 0.3 seconds, then the Web management service reads the video frame file and outputs it to the MJPEG video stream interface; if the time difference of the video frame file exceeds the corresponding upper limit, but the heartbeat file still passes the heartbeat validity check, the Web management service waits for the next frame update and records the video frame update delay in the operation log.

[0060] The alarm reporting service starts along with the Web management service after the Web management service restarts, and reads alarm records from the alarm record table whose reporting status is "pending reporting" or "failed and awaiting retry". For alarm records that fail to be reported, the retry interval increases with the number of failures and is subject to the maximum retry interval limit.

[0061] in, Indicates the first The retry interval after a failure is in seconds. This indicates the maximum retry interval, in seconds. Indicates the initial retry interval, in seconds; Indicates the number of failures, without unit. This can be determined by statistical analysis of the server response time through heartbeat connectivity testing, for example, by taking several times the 95th percentile of the server response time. Set the alarm delay limit according to the business server's allowed limit.

[0062] In one embodiment, the edge computing device also reads CPU utilization, memory utilization, disk utilization, chip temperature, and neural network acceleration unit load, and writes the resource data to the system status cache. Before starting a new AI inference worker process, the Web management service determines whether to start it based on the number of enabled channels, the neural network acceleration unit load, and the remaining memory. If resources are insufficient, the Web management service does not start a new camera channel and records the corresponding channel number and resource data in the operation log. For camera channels that are already running, the number of inference tasks can be reduced by lowering the frame sampling frequency. The frame sampling frequency refers to the frequency at which the AI ​​inference worker process selects image frames from the RTSP video stream and sends them to the AI ​​detection model, measured in frames per second. This parameter can be determined during stress testing, for example, by enabling camera channels one by one, recording the neural network acceleration unit load, inference time, and chip temperature, and selecting the frame sampling frequency with no abnormal logs during continuous operation as the initial value.

[0063] In the above implementation process, the exit record of the Web management service, the channel running identity information in the heartbeat file, the process status in the operating system process table, the current channel configuration data, and the process identity field in the alarm record table all participate in the operation recovery of the same camera channel. During the Web management service restart, the AI ​​inference worker process continues to process video streams and alarm records; after the Web management service restarts, it distinguishes between surviving processes, old configuration processes, and invalid files through process liveness verification, configuration consistency verification, and heartbeat timeliness verification, and then re-registers the AI ​​inference worker processes that pass the verification to the running status table and running attribution table. Page display, video stream output, channel stopping, configuration modification, and alarm reporting all read the re-registered local status, and the conditions for duplicate streaming, duplicate inference, and alarm source mismatch are reduced accordingly.

[0064] It should be noted that although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0065] Please see Figure 2 , Figure 2 This application provides an embodiment of a stand-alone system for web management and AI inference based on process isolation. This system embodiment is similar to... Figure 1 Corresponding to the illustrated method embodiments, this system can be specifically applied to various electronic devices. The system specifically includes: The process startup module 301 is used to start the Web management service and the AI ​​inference worker process that runs independently by camera channel on the edge computing device, and to make the AI ​​inference worker process enter the independent process group. The inference execution module 302 is used to control the AI ​​inference worker process to read the corresponding video stream, execute AI inference, and generate a heartbeat file, video frame file, and alarm record according to the channel configuration data. The heartbeat file contains channel operation identity information. The exit retention module 303 is used to write an exit record to the Web lifecycle table when the Web management service exits, and retain the AI ​​inference worker process with a valid heartbeat to continue running; The restart verification module 304 is used to read the heartbeat file, channel configuration data and operating system process table after the Web management service is restarted, and to perform process survival verification, configuration consistency verification and heartbeat timeliness verification on the AI ​​inference worker process in sequence. The process recovery module 305 is used to re-register the AI ​​inference worker process that has passed the verification as the current running process of the corresponding camera channel according to the verification result, or to clean up the processes and files that have failed the verification and restart the corresponding AI inference worker process according to the current channel configuration data. The status recovery module 306 is used to update the running status table and running attribution table based on the re-registration results, and to restore page display, video stream output and alarm reporting processing according to the running status table, video frame file and alarm record.

[0066] Each processing unit and / or module in the embodiments of this application can be implemented by an analog circuit that implements the functions described in the embodiments of this application, or by software that executes the functions described in the embodiments of this application.

[0067] Based on the same inventive concept, this application also provides an electronic device. The method corresponding to the electronic device can be the method in the foregoing embodiments, and its problem-solving principle is similar to that method. The electronic device provided in this application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the foregoing embodiments of this application.

[0068] Figure 3The diagram illustrates the structure of an electronic device suitable for implementing the methods described in the embodiments of this application. The electronic device includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 402 or a program loaded from a storage section 408 into a random access memory (RAM) 403. The RAM 403 also stores various programs and data required for system operation. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input section 406, an output section 407, a communication section 409, and an input / output (I / O) interface 405 are also connected to the bus 404.

[0069] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a storage medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 401, it performs the functions defined in the methods of this application.

[0070] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0071] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of electronic devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0072] The above description represents the preferred embodiments of the present invention. It should be noted that, for those skilled in the art, various improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for independent operation of Web management and AI inference based on process isolation, characterized in that, include: On the edge computing device, a web management service and an AI inference worker process that runs independently according to the camera channel are started respectively, and the AI ​​inference worker process is put into an independent process group. According to the channel configuration data, the AI ​​inference worker process is controlled to read the corresponding video stream, execute AI inference, and generate heartbeat files, video frame files, and alarm records. The heartbeat files contain channel running identity information. When the Web management service exits, an exit record is written to the Web lifecycle table, and the AI ​​inference worker process with a valid heartbeat continues to run; After the Web management service is restarted, the heartbeat file, channel configuration data and operating system process table are read, and the AI ​​inference worker process is sequentially checked for process liveness, configuration consistency and heartbeat validity. Based on the verification results, the AI ​​inference worker processes that pass the verification will be re-registered as the current running processes of the corresponding camera channels, or the processes and files that fail the verification will be cleaned up and the corresponding AI inference worker processes will be restarted according to the current channel configuration data. The operation status table and operation attribution table are updated based on the re-registration results, and the page display, video stream output and alarm reporting processing are restored according to the operation status table, video frame files and alarm records.

2. The method according to claim 1, characterized in that, The channel configuration data includes at least the channel number, RTSP address de-identification digest, algorithm identifier, alarm threshold, activation status, and configuration version number. After saving the camera configuration or algorithm configuration, the Web management service updates the configuration version number and transmits the updated channel configuration data to the corresponding AI inference worker process.

3. The method according to claim 2, characterized in that, The channel operation identity information includes channel number, AI inference worker process number, AI inference worker process group number, start time, configuration version number, algorithm identifier, RTSP address de-identification digest, lease number and timestamp. The AI ​​inference worker process increments the lease number each time it writes to the heartbeat file.

4. The method according to claim 3, characterized in that, The AI ​​inference worker process first writes the heartbeat content to a temporary heartbeat file, and then renames the temporary heartbeat file to the heartbeat file; the AI ​​inference worker process first writes the video frame content to a temporary video frame file, and then renames the temporary video frame file to the video frame file; the Web management service only reads the renamed heartbeat file and the video frame file.

5. The method according to claim 3, characterized in that, The process liveness verification includes: the Web management service queries the operating system process table based on the AI ​​inference worker process number, reads the command line parameters and process group number of the corresponding process, compares the channel number and process group number in the command line parameters with the channel number and AI inference worker process group number in the heartbeat file, and determines whether the heartbeat file belongs to the live AI inference worker process of the corresponding camera channel based on the comparison result.

6. The method according to claim 5, characterized in that, The configuration consistency verification includes: the Web management service compares the configuration version number, algorithm identifier, and RTSP address anonymization digest in the heartbeat file with the configuration version number, algorithm identifier, and RTSP address anonymization digest in the current channel configuration data item by item; if all three items are consistent, the heartbeat timeliness verification is initiated; if any item is inconsistent, the corresponding independent process group is terminated, and a new AI inference worker process is started according to the current channel configuration data after the heartbeat file stops updating.

7. The method according to claim 6, characterized in that, The heartbeat validity period verification includes timestamp verification and lease number verification. The timestamp verification determines whether the heartbeat has timed out based on the current time, the timestamp in the heartbeat file, and the heartbeat validity period threshold. The lease number verification determines whether the lease number is incremented by continuously reading the heartbeat file. If both are passed, the corresponding AI inference worker process is selected as a candidate process.

8. A stand-alone operating system for Web management and AI inference based on process isolation, characterized in that, include: The process startup module is used to start the Web management service and the AI ​​inference worker process that runs independently by camera channel on the edge computing device, and to put the AI ​​inference worker process into an independent process group. The inference execution module is used to control the AI ​​inference worker process to read the corresponding video stream, execute AI inference, and generate heartbeat files, video frame files, and alarm records according to the channel configuration data. The heartbeat file contains channel operation identity information. The exit retention module is used to write an exit record to the Web lifecycle table when the Web management service exits, and retain the AI ​​inference worker process with a valid heartbeat to continue running; The restart verification module is used to read the heartbeat file, channel configuration data and operating system process table after the Web management service is restarted, and to perform process survival verification, configuration consistency verification and heartbeat validity verification on the AI ​​inference worker process in sequence. The process recovery module is used to re-register the AI ​​inference worker process that has passed the verification as the current running process of the corresponding camera channel based on the verification result, or to clean up the processes and files that have failed the verification and restart the corresponding AI inference worker process according to the current channel configuration data. The status recovery module is used to update the running status table and running attribution table based on the re-registration results, and to restore page display, video stream output and alarm reporting processing according to the running status table, video frame files and alarm records.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A storage medium storing computer program instructions thereon, characterized in that, The computer program instructions can be executed by a processor to implement the method as described in any one of claims 1-7.

Images