Robot safety alerting method and system based on edge computing
By acquiring image and pose information through edge computing technology, predicting target motion trends, and dynamically scheduling resources, the problems of lag and slow recovery in target recognition and tracking in robot systems are solved, achieving stable and fast target tracking results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 武汉船舶职业技术学院
- Filing Date
- 2026-04-15
- Publication Date
- 2026-07-14
Smart Images
Figure CN122391673A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent control technology, specifically a robot safety monitoring method and system based on edge computing. Background Technology
[0002] As robots become increasingly prevalent in inspection, security, and collaborative operations, the requirements for continuous target perception, real-time response, and stable control are constantly rising. Existing solutions largely rely on central-side computation or static rule-based control, resulting in long processing chains, delayed responses, and slow recovery after partial lock-out. Especially when object recognition results are affected by occlusion, motion blur, and posture disturbances, relying solely on single-frame recognition or fixed threshold control easily leads to tracking interruptions, accumulated control deviations, and uneven resource allocation. While edge computing can shorten processing paths, current technologies generally lack a mechanism to create a closed-loop collaboration between object recognition, motion prediction, gimbal control, state determination, and resource reallocation, making it difficult to balance real-time performance and stability. Summary of the Invention
[0003] This application provides a robot safety monitoring method and system based on edge computing, which is used to at least solve the problem of how to achieve stable target tracking and rapid recovery after loss of lock under limited computing power.
[0004] Firstly, this application provides a robot safety monitoring method based on edge computing, the method comprising: Acquire images and pose information of the warning scene, determine target location information, and establish a historical frame feature cache; Based on the target positioning information and historical frame feature cache, the short-term motion trend and predicted position of the target are determined, and based on the offset between the predicted position of the target and the center of the field of view, combined with the attitude information, gimbal adjustment commands are generated. Execute gimbal adjustment commands to obtain updated images, and determine target locking stability indicators based on the proportion of the target visible area in the updated images and the historical frame feature cache. If the target locking stability index is lower than the preset stability threshold, edge computing resources are scheduled, and the target is recaptured based on the updated image and historical frame feature cache, and the target positioning information and historical frame feature cache are updated. Based on the updated target positioning information, output stable tracking results, and feed back the updated target positioning information to the next round of target short-term motion trend determination and gimbal adjustment command generation.
[0005] In one possible implementation, the posture information includes robot posture information and gimbal posture information. Determining the target localization information includes: performing noise reduction and contrast enhancement processing on the warning scene image; extracting target candidate regions from the processed warning scene image; combining the robot posture information and gimbal posture information to determine the target center coordinates and target bounding box corresponding to the target candidate regions; and determining the target localization information based on the target center coordinates and target bounding box.
[0006] In one possible implementation, establishing a historical frame feature cache includes: extracting the current frame appearance features of the target candidate region; assigning a target identifier to the target candidate region; and associating the current frame appearance features with the target identifier and storing them in the historical frame feature cache.
[0007] In one possible implementation, determining the short-term motion trend and predicted position of the target includes: determining the target displacement and scale change based on the target center coordinates and target bounding box in consecutive frames; performing temporal filtering on the target displacement and scale change using historical frame feature buffers to obtain a motion state sequence; determining the target motion direction and predicted velocity based on the motion state sequence; and determining the target short-term motion trend and predicted position based on the target motion direction and predicted velocity.
[0008] In one possible implementation, generating gimbal adjustment instructions includes: determining the target angular displacement based on the offset between the predicted target position and the center of the field of view; determining the gimbal rotation direction based on the target angular displacement; determining the base gimbal rotation speed based on the target angular displacement, robot posture information, and gimbal posture information; increasing the base gimbal rotation speed if the offset is greater than a preset offset threshold; and generating gimbal adjustment instructions based on the gimbal rotation direction and the base gimbal rotation speed.
[0009] In one possible implementation, executing a gimbal adjustment command to acquire an updated image and determining the target locking stability index based on the target visible area ratio in the updated image and the historical frame feature cache includes: driving the gimbal to rotate according to the gimbal adjustment command and acquiring an image after the gimbal rotation as the updated image; determining the target region in the updated image and determining the target visible area ratio based on the target region; extracting target texture features and local contrast features from the target region; matching the target texture features with historical features in the historical frame feature cache to obtain a feature matching result; and determining the target locking stability index based on the target visible area ratio, local contrast features, and the feature matching result.
[0010] In one possible implementation, scheduling edge computing resources includes: determining the processing priority of the image processing task corresponding to the target region based on the target locking stability index; allocating edge computing resources from low-priority tasks to the image processing task corresponding to the target region; and establishing a high-priority processing thread based on the reallocated edge computing resources.
[0011] In one possible implementation, recapturing the target based on the updated image and historical frame feature cache, and updating the target location information and historical frame feature cache include: extracting updated frame appearance features from the target region in the updated image using a high-priority processing thread; matching the updated frame appearance features with historical features in the historical frame feature cache to obtain a target matching result; determining the matching target identifier corresponding to the target region if the target matching result meets preset matching conditions; determining the updated target center coordinates and the updated target bounding box based on the target region, and updating the target location information based on the updated target center coordinates and the updated target bounding box; and writing the updated frame appearance features into the historical feature record corresponding to the matching target identifier in the historical frame feature cache.
[0012] In one possible implementation, outputting stable tracking results and feeding back updated target positioning information to the next round of target short-term motion trend determination and gimbal adjustment command generation includes: generating a target position sequence based on target positioning information at consecutive times; performing temporal smoothing on the target position sequence to obtain stable tracking results; and using the target positioning information at the current time as input for the next round of target short-term motion trend determination and gimbal adjustment command generation.
[0013] Secondly, this application provides a robot safety monitoring system based on edge computing for implementing a robot safety monitoring method based on edge computing. The system includes: The positioning and database building module is used to acquire images and pose information of the warning scene, determine the target positioning information, and establish a historical frame feature cache; The trend control module is used to determine the short-term motion trend and predicted position of the target based on the target positioning information and historical frame feature cache, and to generate gimbal adjustment commands based on the offset between the predicted position of the target and the center of the field of view, combined with attitude information. The status determination module is used to execute gimbal adjustment commands to obtain updated images, and determine the target locking stability index based on the proportion of the target visible area in the updated image and the historical frame feature cache. The recapture module is used to schedule edge computing resources and recapture the target based on the updated image and historical frame feature cache when the target locking stability index is lower than the preset stability threshold, and update the target positioning information and historical frame feature cache. The feedback output module is used to output stable tracking results based on the updated target positioning information, and to feed back the updated target positioning information to the next round of target short-term motion trend determination and gimbal adjustment command generation.
[0014] Compared with existing technologies, the advantages and beneficial effects of this application are as follows:
[0015] By combining target positioning information with historical frame feature caching for joint analysis, short-term target motion trend prediction and predicted position generation are achieved, enabling early guidance and control. Gimbal control technology, which couples predicted position offset with attitude information, enables gimbal orientation correction, reducing tracking lag. A joint determination technique, combining the target's visible area ratio with historical feature matching, enables target locking stability assessment, accurately identifying unstable states. Dynamic scheduling of edge computing resources and target re-acquisition technology enable rapid recovery of tracking after lock loss, improving continuity. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating the method described in this application;
[0017] Figure 2 This is a block diagram of the module composition of the system in this application. Detailed Implementation
[0018] To enable those skilled in the art to better understand the technical solution, the present invention will be described in detail below with reference to embodiments. The description in this part is only exemplary and explanatory, and should not be used to limit the scope of protection of the present invention in any way.
[0019] Edge computing is a technology that moves data processing, analysis, judgment, and control execution as close to the device or near-end node as possible. Its core lies in shortening the transmission path of perceived data from acquisition to processing and then to feedback control. In processing tasks involving object recognition, edge computing enables image parsing, state determination, and control response to be completed continuously locally, reducing reliance on remote computing resources. This makes it more suitable for technology systems with high requirements for latency, continuity, and stability. Furthermore, by co-designing object recognition results with target motion analysis, posture control, and state feedback, an integrated processing mechanism for continuous tracking and dynamic control can be formed. Based on this, this application proposes a robot safety monitoring method and system based on edge computing.
[0020] like Figure 1 As shown, a robot safety monitoring method based on edge computing is proposed, which includes:
[0021] Acquire images and pose information of the warning scene, determine target location information, and establish a historical frame feature cache;
[0022] In one embodiment, after the robot enters a safety alert state, the edge computing unit first receives the alert scene image acquired by the forward imaging component, and simultaneously receives the robot's body posture and gimbal posture. After completing the initial target discovery in the current frame, the edge computing unit determines the target's spatial position in the image plane, forming target localization information. The target localization information includes at least the target's center position, target boundary range, and the target's corresponding identification marker. Subsequently, the edge computing unit extracts image features that reflect appearance differences around the current target and establishes a historical frame feature cache. The historical frame feature cache is used to continuously store the target's appearance records and identification associations at adjacent times, providing a unified data foundation for subsequent motion trend judgment, target lock status determination, and target re-acquisition.
[0023] The attitude information includes robot attitude information and gimbal attitude information. Determining target localization information includes: denoising and contrast enhancement processing of the warning scene image; extracting target candidate regions from the processed warning scene image; combining robot attitude information and gimbal attitude information to determine the target center coordinates and target bounding box corresponding to the target candidate regions; and determining target localization information based on the target center coordinates and target bounding box.
[0024] In one embodiment, the attitude information is obtained through a layered acquisition method. The robot attitude information is output by an inertial measurement unit, a wheel speed odometer, or a chassis encoder, and is used to characterize the robot's pitch, roll, yaw, and displacement change trends at the current sampling moment. The gimbal attitude information is output by a gimbal angle sensor or a gimbal motor feedback unit, and is used to characterize the rotation state of the camera direction relative to the robot body.
[0025] After receiving the alert scene image, the edge computing unit first performs denoising and contrast enhancement processing on the current image. Denoising can employ median filtering, mean filtering, or bilateral filtering to remove photosensitive noise and slight compression noise, preventing background texture fragments from being misidentified as target edges. Contrast enhancement can use grayscale stretching, local contrast enhancement, or brightness equalization to increase the grayscale difference between dark areas and target edges, making the target contour easier to separate from the background. After preprocessing, the edge computing unit extracts target candidate regions from the processed image. Target candidate regions can be obtained through motion region detection, salient region extraction, background subtraction, or candidate box outputs based on a trained recognition network.
[0026] To avoid an excessive number of candidate results impacting subsequent processing speed, candidate regions are typically filtered based on area range, aspect ratio, and edge continuity, retaining only those regions that meet the requirements of the alert scenario. Subsequently, the edge computing unit combines robot and gimbal posture information to perform posture compensation on the image coordinates of the candidate regions. The purpose of posture compensation is not to change the target's original position in the image, but to eliminate the instantaneous deviations in target localization caused by robot body swaying and gimbal deflection, making the positional changes of the same target in consecutive frames closer to the actual motion trajectory. After posture compensation, the edge computing unit calculates the center position of the target candidate region and generates the minimum bounding rectangle enclosing the target region, forming the target bounding box. The target center coordinates represent the target's primary position in the current frame, and the target bounding box defines the target's spatial range and size.
[0027] The edge computing unit then generates target localization information based on the target center coordinates and the target bounding box. To ensure direct access in subsequent steps, the target localization information preferably includes the current frame number, target center coordinates, target bounding box, and a status marker indicating whether the target is within the effective field of view. This processing ensures that the target localization information reflects the target's current position and provides a stable input boundary for subsequent short-term motion trend judgment. This implementation, by linking image preprocessing, candidate extraction, and pose compensation, makes the target localization result independent of the instantaneous state of a single image, making it particularly suitable for robot surveillance scenarios involving inspection, turning, or slight vibrations.
[0028] Establishing a historical frame feature cache includes: extracting the current frame appearance features of the target candidate region; assigning a target identifier to the target candidate region; and associating the current frame appearance features with the target identifier and storing them in the historical frame feature cache.
[0029] In one embodiment, the historical frame feature cache is generated synchronously with the target location information to store the target's appearance record and identification association in consecutive frames. Here, the current frame appearance features refer to image description information extracted from the target candidate region that reflects the target's surface texture, color distribution, contour structure, or local contrast differences.
[0030] After generating target localization information, the edge computing unit performs feature extraction on the target candidate region. Feature extraction can employ one or more combinations of color histograms, texture direction distribution, local block grayscale statistics, or lightweight convolutional features. Considering that this application is geared towards edge computing scenarios, feature extraction preferably uses a description method with low computational cost and a certain tolerance for scale changes, ensuring that the cache building process does not crowd out processing resources for subsequent tracking and control tasks. When assigning target identifiers to target candidate regions, a sequential numbering method can be used, or an identifier generation method can be used, which combines the camera channel number, the first appearance time, and the target sequence number. The principle for setting target identifiers is that the same target remains unique during short-term continuous tracking, and different targets can be stably distinguished. After completing the identifier allocation, the edge computing unit associates the current frame appearance features with the target identifiers and writes them into the historical frame feature cache.
[0031] Historical frame feature caching can be implemented using a circular queue, a linked cache table, or an index table built according to target identifiers. If a circular queue is used, the cache capacity can be set based on the available storage space of the edge device and the maximum continuous tracking duration of the target, for example, storing appearance records of the most recent few frames. If an index table is used, each target identifier corresponds to an independent recording unit, and the unit stores features from multiple frames in chronological order. When the cache is established, in addition to writing the appearance features and target identifier of the current frame, the target bounding box size and center position can also be recorded simultaneously for spatial consistency verification during subsequent target recapture stages. This setup is because relying solely on the appearance of a single frame is easily affected by changes in lighting, partial occlusion, or motion blur. By organizing continuous frame features by identifier, temporal continuity can be used in subsequent steps to determine whether the current target is still the same object.
[0032] When the edge computing unit detects that the position and appearance of the same target in a new frame both meet the continuity condition, the newly extracted appearance features can be written to the corresponding cache. When the target is detected to have disappeared, been replaced, or is significantly inconsistent with historical records, the overwriting of old records can be paused, and the cache can be updated only after the recapture result is confirmed. Through this method, the historical frame feature cache not only serves as a data storage function but also as a means of maintaining target continuity. It provides a continuous observation basis for subsequent short-term motion trend judgment, a historical comparison basis for target lock status determination, and a reliable appearance reference for recapture after target lock loss.
[0033] Based on the target positioning information and historical frame feature cache, the short-term motion trend and predicted position of the target are determined, and based on the offset between the predicted position of the target and the center of the field of view, combined with the attitude information, gimbal adjustment commands are generated.
[0034] In one embodiment, after obtaining target positioning information and historical frame feature cache, the edge computing unit first performs correlation analysis on the changes in the target's spatial position and range over consecutive time intervals to form a motion description reflecting the target's short-term movement state, and then obtains the target's predicted position for the next time interval based on this description. Subsequently, the edge computing unit compares the predicted target position with the current field of view center to obtain the target's offset relative to the field of view center, and generates a gimbal adjustment command by combining the robot's current posture and the gimbal's current posture. The gimbal adjustment command includes at least a rotation direction and rotation speed, used to drive the gimbal to realign the shooting direction with the area where the target is expected to appear. By adopting a processing path of prediction followed by control, the gimbal can avoid passively correcting itself around the current frame position, thereby reducing tracking lag caused by rapid target movement or image delay.
[0035] Determining the short-term motion trend and predicted position of the target includes: determining the target displacement and scale change based on the target center coordinates and target bounding box in consecutive frames; performing temporal filtering on the target displacement and scale change by combining historical frame feature cache to obtain a motion state sequence; determining the target motion direction and predicted velocity based on the motion state sequence; and determining the target short-term motion trend and predicted position based on the target motion direction and predicted velocity.
[0036] In one embodiment, the short-term motion trend of the target and the predicted position of the target are jointly determined based on the changes in position and boundary range across consecutive frames. The edge computing unit first reads multiple consecutive frames corresponding to the current target identifier from the historical frame feature cache, and extracts the target center coordinates and the target bounding box from each frame. The target center coordinates are used to represent the main position of the target in the image plane, and the target bounding box is used to represent the spatial range currently occupied by the target.
[0037] Based on the difference in target center coordinates between adjacent frames, the edge computing unit calculates the target displacement; based on the changes in the target bounding box width and height between adjacent frames, the edge computing unit calculates the scale change. The target displacement mainly reflects the target's movement amplitude and direction within the frame, while the scale change mainly reflects changes in distance between the target and the camera, size changes when the target enters or leaves the field of view, and local scaling fluctuations caused by attitude disturbances. To avoid single-frame abnormal fluctuations directly affecting the prediction results, the edge computing unit performs temporal filtering on the target displacement and scale change. Temporal filtering can be implemented using moving average, weighted average, or median correction. If a weighted average is used, frames closer to the current moment are assigned higher weights, and frames from earlier moments are assigned lower weights, ensuring that the prediction results are more sensitive to current motion changes. After temporal filtering, a motion state sequence is obtained. The motion state sequence can be understood as a continuous record of the target's motion within the frame over a short period, including at least the direction of position change, changes in movement amplitude, and scale change trends. The edge computing unit then determines the target's motion direction and predicted velocity based on the motion state sequence.
[0038] In this embodiment, the short-term movement trend of the target can be represented by the short-term movement vector of the target. The predicted position of the target can be extrapolated based on the current center position of the target according to the short-term movement vector of the target, which can be specifically expressed as:
[0039]
[0040]
[0041] in, This is the short-term motion vector of the target, used to characterize the short-term motion trend of the target at the current moment; For predicting speed; The unit direction vector corresponding to the target's direction of motion. The current target center position; Predict the target position for the next moment; This is the prediction time interval corresponding to the current processing cycle. The edge computing unit determines the unit direction vector based on the target's motion direction, determines the target's position change within the prediction time interval based on the predicted velocity, and obtains the target's predicted position by combining this with the target's center position at the current moment.
[0042] The target's motion direction can be determined by the main direction of the continuous frame displacement, and the predicted velocity can be determined by the average rate of change of the continuous frame displacement. When the scale change increases continuously in a short period of time, it can be determined that the target is moving towards the camera direction. When the scale change decreases continuously, it can be determined that the target is moving away from the camera direction.
[0043] After determining the target's motion direction and predicted velocity, the edge computing unit extrapolates a short-term predicted position along the target's motion direction based on the current target center position, thus obtaining the target's predicted position. This short-term prediction is based on the current frame processing cycle, typically corresponding to the next sampling time or the next control cycle. The time span should not be too long to ensure the predicted position remains within the range that the gimbal can correct in a timely manner. If the target's appearance in consecutive frames differs significantly from the corresponding record in the historical frame feature cache, the edge computing unit reduces the participation of earlier frame data in the motion state sequence to avoid erroneous predictions caused by target replacement, partial occlusion, or short-term false detections. Through this processing method, the target's short-term motion trend is not directly inferred from a single frame position, but is determined by changes in target position, size, and historical continuity. Therefore, it is more suitable for short-cycle, real-time tracking control in robot surveillance scenarios.
[0044] The process of generating gimbal adjustment instructions includes: determining the target angular displacement based on the offset between the predicted target position and the center of the field of view; determining the gimbal rotation direction based on the target angular displacement; determining the basic gimbal rotation speed based on the target angular displacement, robot posture information, and gimbal posture information; increasing the basic gimbal rotation speed if the offset is greater than a preset offset threshold; and generating gimbal adjustment instructions based on the gimbal rotation direction and the basic gimbal rotation speed.
[0045] In one embodiment, the gimbal adjustment command is established around the offset between the predicted target position and the center of the field of view. After obtaining the predicted target position, the edge computing unit first calculates the offset of the predicted target position relative to the center of the field of view. The offset can be calculated separately for lateral and longitudinal offsets, or it can be further converted into a combined offset. The center of the field of view is the center position of the current image frame, used to characterize the ideal alignment direction of the gimbal. The target angular displacement is determined based on the offset, and its meaning is the amount of angular correction that the gimbal needs to perform to bring the target back from the predicted position to the vicinity of the center of the field of view.
[0046] The edge computing unit determines the gimbal's rotation direction by judging whether it should rotate left, right, up, or down based on the target's angular displacement. Then, the edge computing unit combines the target's angular displacement, robot posture information, and gimbal posture information to determine the basic gimbal rotation speed. Robot posture information reflects whether the robot is turning, pitching, or swaying at the current moment; gimbal posture information reflects the current angle of deflection and actual pointing of the gimbal. If the robot shows a clear turning tendency, or the gimbal's current posture is close to the mechanical rotation boundary, the basic gimbal rotation speed is prioritized according to the smooth correction principle to avoid excessively fast rotation causing image jitter and target loss again. If the robot's posture is relatively stable and the gimbal still has sufficient rotation margin, the basic gimbal rotation speed can be set according to the fast following principle to shorten the time it takes for the target to return to the center area. A preset offset threshold is used to distinguish between normal correction and enhanced correction control states; the setting principle can be determined based on the camera's field of view, image resolution, and the allowable distance of the target from the center. For example, in scenarios where the target occupies a small proportion of the screen and is easily lost once it deviates from the center, the preset offset threshold is set relatively small; in scenarios with a wide field of view where the target can remain clearly visible even after a short period of deviation, the preset offset threshold can be appropriately widened.
[0047] When the offset exceeds a preset offset threshold, the edge computing unit increases the rotation speed of the base gimbal to quickly correct the gimbal towards the predicted target position. When the offset is below the preset threshold, the edge computing unit maintains the rotation speed of the base gimbal to avoid frequent acceleration that could cause control jitter. Finally, the edge computing unit generates a gimbal adjustment command based on the gimbal rotation direction and the base gimbal rotation speed, and sends the command to the gimbal execution unit. Upon receiving the command, the gimbal execution unit drives the motor to rotate in the corresponding direction and speed. By generating control commands based on predicted position, attitude state, and offset threshold, gimbal control no longer relies solely on the target's instantaneous position in the current frame. Instead, it can proactively correct for the target's potential position in the next moment, thus maintaining good continuous tracking capability even when the target moves rapidly, the robot's posture changes, or processing delays occur.
[0048] Execute gimbal adjustment commands to obtain updated images, and determine target locking stability indicators based on the proportion of the target visible area in the updated images and the historical frame feature cache.
[0049] In one embodiment, after generating a gimbal adjustment command, the edge computing unit sends the command to the gimbal execution unit, driving the camera component to rotate in the direction corresponding to the predicted target position. After the rotation ends or stabilizes, the edge computing unit captures the current image as an updated image. Subsequently, the edge computing unit re-determines the target region in the updated image, calculates the proportion of the target's visible area in the current frame, and combines this with the appearance records stored in the historical frame feature cache to determine whether the current target maintains a continuous and stable locked state. The target locking stability index characterizes whether the target has a basis for continuous tracking in the current frame. This index simultaneously reflects the degree of target occlusion, the clarity of the target region, and the consistency between the current target and historical targets, providing a direct basis for subsequent edge computing resource scheduling and target re-acquisition.
[0050] The process of executing gimbal adjustment commands to acquire updated images and determining target locking stability metrics based on the target visible area ratio in the updated image and historical frame feature cache includes: driving the gimbal to rotate according to the gimbal adjustment commands and acquiring images after the gimbal rotation as updated images; identifying the target region in the updated image and determining the target visible area ratio based on the target region; extracting target texture features and local contrast features from the target region; matching the target texture features with historical features in the historical frame feature cache to obtain feature matching results; and determining target locking stability metrics based on the target visible area ratio, local contrast features, and feature matching results.
[0051] In one embodiment, the target locking stability index is obtained in the following order: "execute gimbal rotation—acquire and update image—determine target area—extract current features—compare historical features—form comprehensive judgment." After receiving the gimbal adjustment command, the gimbal execution unit drives the motor to move according to the rotation direction and speed in the command, so that the camera direction moves closer to the area where the target is predicted to be located.
[0052] Considering the slight mechanical vibrations that may occur during the gimbal's startup, deceleration, and shutdown, the updated image is preferably the first stable frame after the gimbal has completed its rotation. Alternatively, the image with the highest clarity among several consecutive frames after rotation can be selected. After obtaining the updated image, the edge computing unit first determines the target region within that image. The method for determining the target region is consistent with the previous localization stage. The predicted target location can continue to be used as the search center, and a local search window can be set near the predicted target location. Then, the target region in the current frame is determined by combining edge contours, region connectivity, and appearance similarity. The reason for this approach is that the gimbal has already completed directional correction according to the predicted location, and the target is highly likely to appear near the predicted location. Using a local search window can reduce false detections caused by full-image search and reduce the processing burden on edge devices.
[0053] After determining the target region, the edge computing unit calculates the proportion of the actual visible portion of the target region to the total area of the target bounding box, obtaining the target visible region percentage. This target visible region percentage describes the completeness of the target in the current image. If the target region has continuous edges and the main body is not occluded, the target visible region percentage is high; if the target is partially occluded by foreground objects, enters the edge of the image, or is partially missing due to rotational blur, the target visible region percentage decreases. To ensure the operability of this percentage, areas within the target bounding box that satisfy texture continuity, natural grayscale transitions, and clear distinction from the background are considered valid visible portions. Subsequently, the edge computing unit extracts target texture features and local contrast features from the target region. Target texture features describe the texture distribution, edge organization, and local structural differences of the target surface, and can be extracted using one or a combination of grayscale co-occurrence statistics, local block orientation statistics, or lightweight convolutional features. Local contrast features describe the brightness difference and edge sharpness between the target region and the surrounding background, and can be obtained using neighborhood grayscale difference, local mean difference, or local variance statistics.
[0054] The purpose of introducing local contrast features is to avoid situations where the target, although roughly in the correct position, still lacks reliable tracking conditions due to backlighting, low light, or fogging. After completing the feature extraction of the current frame, the edge computing unit reads the historical feature record corresponding to the current target identifier from the historical frame feature cache and matches the currently extracted target texture features with the historical features to obtain the feature matching result. The feature matching result is used to represent the degree of consistency between the current target area and the historically continuously tracked targets in appearance. The higher the consistency, the more likely the currently detected target area is still the same target. If the matching result drops significantly, there may be target replacement, severe target occlusion, or the target has deviated from the expected position. Finally, the edge computing unit determines the target locking stability index based on the proportion of the target's visible area, local contrast features, and feature matching results. The target locking stability index can be set using a graded judgment method, such as dividing it into three levels: stable locking, weakly stable locking, and unstable waiting for re-capture; or it can be set using a normalized comprehensive score method.
[0055] If a comprehensive scoring method is adopted, the principle is set as follows: the higher the proportion of the visible target area, the more consistent the target texture with historical records, and the more obvious the local contrast between the target area and the background, the higher the target locking stability index. Correspondingly, when the proportion of the visible target area is low, the local contrast is insufficient, or the feature matching result decreases, the target locking stability index decreases. To facilitate subsequent resource scheduling, the threshold of the target locking stability index can be preset according to the lighting conditions of the warning scene, the average target size, and the processing capability of the edge device. For scenes with stable lighting and simple backgrounds, the threshold can be set relatively strictly; for scenes with more background interference and targets that are easily partially occluded, the threshold can be appropriately relaxed to avoid frequent triggering of re-capture. Through the above processing, the target locking stability index no longer depends on the result of a single location, but is constrained by the completeness of the target, the recognizability of the image, and the consistency of history. Therefore, it can more realistically reflect the current tracking status and provide a stable basis for whether to schedule edge computing resources in the future.
[0056] If the target locking stability index is lower than the preset stability threshold, edge computing resources are scheduled, and the target is recaptured based on the updated image and historical frame feature cache, and the target positioning information and historical frame feature cache are updated.
[0057] In one embodiment, when the target locking stability index falls below a preset stability threshold, the edge computing unit determines the current tracking state as unstable and enters a resource reallocation and target re-acquisition process. The edge computing unit first suspends low-priority image processing tasks unrelated to the current target, transfers available computing resources to the image processing task corresponding to the current target region, and then uses the reallocated processing power to perform local enhancement analysis on the updated image. Subsequently, the edge computing unit, in conjunction with the historical frame feature cache, re-identifies the target region in the updated image to determine whether the matching target is still the original target; if the identification is successful, the target positioning information and the historical frame feature cache are updated, enabling subsequent tracking to continue based on the new positioning.
[0058] Scheduling edge computing resources includes: determining the processing priority of the image processing task corresponding to the target region based on the target locking stability index; allocating edge computing resources from low-priority tasks to the image processing task corresponding to the target region; and establishing high-priority processing threads based on the reallocated edge computing resources.
[0059] In one embodiment, the scheduling of edge computing resources is directly triggered by the target locking stability index, and the image processing task corresponding to the target area is the primary resource recipient. Low-priority tasks here may include routine background area update tasks, auxiliary analysis tasks for non-critical targets, and periodic recording tasks. These tasks are delayed for a short period without affecting the immediacy of target recapture.
[0060] After detecting that the target locking stability index is lower than a preset stability threshold, the edge computing unit first sorts the currently running tasks according to a pre-defined priority rule. The principle for setting the priority rule is: tasks directly related to the target region and affecting the target re-identification result have the highest priority, while tasks not directly related to the current target or that can be processed later have a lower priority. Based on this, the edge computing unit determines the processing priority of the image processing task corresponding to the target region, and reclaims processor time slices, image cache space, and feature calculation quotas from low-priority tasks, reallocating these edge computing resources to the image processing task corresponding to the target region. The reallocation method can be a fixed proportion allocation or an allocation method that increases progressively according to the degree of instability.
[0061] When the target locking stability index is only slightly below the preset stability threshold, the edge computing unit can increase edge computing resources only to maintain the overall stable operation of the system. When the target locking stability index is significantly below the preset stability threshold, the edge computing unit can further reduce the processing frequency of low-priority tasks, allowing more resources to be concentrated on target area analysis. The preset stability threshold can be determined based on the test results of the warning scenario samples. For example, by statistically analyzing the distribution range of the target locking stability index under continuous stable tracking conditions, the boundary value that can distinguish between stable and unstable states can be selected as the preset stability threshold. After completing the resource transfer, the edge computing unit establishes a high-priority processing thread. The high-priority processing thread is mainly used to perform operations such as target area cropping, updating frame appearance feature extraction, and historical feature matching.
[0062] The reason for this setup is that target recapture has high processing time requirements. If conventional thread sequential execution is still used, the target may deviate from the center of the image again due to queuing. By establishing a high-priority processing thread, the target area in the updated image can be re-analyzed in a shorter time, thereby shortening the interval between instability and recovery. The entire resource scheduling process does not change the basic structure of the original tracking process, but temporarily changes the allocation of edge computing resources during the instability phase, so that limited computing power is prioritized for the target recovery process.
[0063] The target is recaptured based on the updated image and historical frame feature cache. Updating the target location information and historical frame feature cache includes: extracting the appearance features of the updated frame from the target region in the updated image using a high-priority processing thread; matching the appearance features of the updated frame with historical features in the historical frame feature cache to obtain the target matching result; determining the matching target identifier corresponding to the target region if the target matching result meets the preset matching conditions; determining the updated target center coordinates and the updated target bounding box based on the target region, and updating the target location information based on the updated target center coordinates and the updated target bounding box; and writing the updated frame appearance features into the historical feature record corresponding to the matching target identifier in the historical frame feature cache.
[0064] In one embodiment, target recapture is performed after the high-priority processing thread is established. The edge computing unit first extracts the appearance features of the updated frame from the target region in the updated image. The extraction method of the updated frame appearance features is consistent with that of the historical frame feature cache establishment stage to ensure the comparability of the features before and after. Subsequently, the edge computing unit reads the historical feature record corresponding to the current target identifier from the historical frame feature cache and matches the updated frame appearance features with the historical features to obtain the target matching result. The target matching result is used to represent the degree of consistency in appearance between the current target region and the historically continuously tracked targets.
[0065] To avoid misjudgments caused by relying solely on appearance similarity, preset matching conditions can be set based on both appearance consistency and positional continuity. Specifically, when the similarity between the appearance features of the updated frame and historical features reaches a preset matching threshold, and the target region in the updated image is still near the previous predicted position, the edge computing unit determines that the target matching result meets the preset matching conditions. The preset matching threshold can be set based on the feature fluctuation range of the sample target during continuous tracking, ensuring that the same target can still be identified under changes in lighting and slight occlusion, while different targets are not mistaken for the same object. When the target matching result meets the preset matching conditions, the edge computing unit determines the matching target identifier corresponding to the target region and recalculates the updated target center coordinates and the updated target bounding box based on the current target region. The updated target center coordinates represent the latest position of the target in the current frame, and the updated target bounding box represents the latest range of the target in the current frame.
[0066] The edge computing unit then updates the target positioning information based on these two results, ensuring that subsequent short-term motion trend judgments and gimbal control no longer rely on the position records from before instability. Simultaneously, the edge computing unit writes the updated frame appearance features into the historical feature cache corresponding to the matching target identifier in the historical feature record. This writing can be done by overwriting the oldest record or appending records chronologically, ensuring that the historical frame feature cache always reflects the target's recent appearance changes. If the target matching result does not meet the preset matching conditions, the edge computing unit can keep the original historical feature records intact and continue using high-priority processing threads to perform the next round of search and matching on subsequent updated images. Through this processing method, target recapture, target positioning information update, and historical frame feature cache update form a continuous closed loop, ensuring that the recovered target maintains the same identifier as the original target and that the appearance records used for subsequent tracking can promptly reflect the target's current state.
[0067] Based on the updated target positioning information, output stable tracking results, and feed back the updated target positioning information to the next round of target short-term motion trend determination and gimbal adjustment command generation.
[0068] In one embodiment, after completing target recapture and updating target positioning information, the edge computing unit outputs the target position results of consecutive moments in chronological order, forming a stable tracking result for the current tracking cycle. The stable tracking result characterizes the smooth motion state of the target in the continuous frame and serves as the direct input for the next round of tracking control. Simultaneously with outputting the stable tracking result, the edge computing unit writes back the updated target positioning information to the tracking state area for use in the next round of short-term target motion trend judgment and gimbal adjustment command generation. This method of simultaneous result output and state write-back allows the target recovery results of the current round to immediately participate in the next round of prediction and control, avoiding state gaps between processing rounds.
[0069] The process of outputting stable tracking results and feeding back the updated target positioning information to the next round of target short-term motion trend determination and gimbal adjustment command generation includes: generating a target position sequence based on the target positioning information at continuous time intervals; performing time-series smoothing on the target position sequence to obtain stable tracking results; and using the target positioning information at the current time interval as input for the next round of target short-term motion trend determination and gimbal adjustment command generation.
[0070] In one embodiment, the output and feedback of stable tracking results are performed in the order of "generating a position sequence, performing temporal smoothing, forming a stable result, and writing back the input for the next round". After obtaining the updated target positioning information, the edge computing unit first extracts the target center coordinates from the target positioning records of several consecutive time periods, and arranges them in chronological order to generate a target position sequence.
[0071] The target position sequence reflects the continuous movement trajectory of the target over a short period, preserving both the overall direction of movement and the sequence of position changes. The number of consecutive time points selected here can be determined based on the image sampling frequency and the gimbal control cycle. If the sampling frequency is high, the number of consecutive time points can be increased appropriately to obtain a more complete position change process; if the sampling frequency is low, the number of consecutive time points can be reduced appropriately to avoid introducing excessively long time spans that cause trajectory lag. After generating the target position sequence, the edge computing unit performs temporal smoothing on the target position sequence. The purpose of temporal smoothing is to reduce single-frame detection jitter, jumps after local occlusion recovery, and positional abrupt changes caused by slight gimbal swings, making the output tracking results more consistent with the target's true motion state. Temporal smoothing can be implemented using any of the following methods: moving average, weighted average, or median correction.
[0072] If a weighted average is used, the closer the location record is to the current time, the higher the weight, and the farther the location record is from the current time, the lower the weight. This can suppress jitter while maintaining responsiveness to the current motion trend. If an anomaly point appears in the continuous location records that deviates significantly from the trajectory of adjacent positions, the edge computing unit can identify the anomaly point as an instantaneous offset record and reduce its participation in the smoothing process. After completing the temporal smoothing process, the edge computing unit obtains a stable tracking result. The stable tracking result can be represented as the smoothed center position of the current target, or it can also include the smoothed target bounding box change result, which can be used for interface display, trajectory saving, or continued use in subsequent control. In order to ensure that the feedback process forms a closed loop with the previous steps, after obtaining the stable tracking result, the edge computing unit does not just use it as the final output to end the processing, but writes the target positioning information of the current time into the next round of tracking input area, as the basis data for judging the short-term motion trend of the target and generating gimbal adjustment commands in the next round.
[0073] The reason for this approach is that if the next round of prediction and control is still based on the position record before the update, it is easy to cause lag in the prediction direction or accumulation of control deviation. By directly feeding back the target positioning information that has been corrected and smoothed at the current moment, the next round of processing can start from the latest stable state. During feedback, the edge computing unit preferably retains the relationship between the current target identifier and the corresponding position record, so that the continuous tracking link of the same target can still be used when calling data in the next round. If there are still large jumps in position changes for more than two consecutive frames after the current round outputs stable tracking results, the edge computing unit can reduce the participation of earlier historical position records in the next round of prediction, and only retain position records closer to the current moment for calculation, so as to ensure that the feedback data is closer to the current real state. Through this integrated output and feedback processing method, the stable tracking result not only plays the role of result output, but also plays the role of state connection, so that the target positioning information continuously cycles in the path of "update-output-feedback-re-prediction-re-control", thereby ensuring that the entire robot safety warning process can run continuously.
[0074] like Figure 2 As shown, an edge computing-based robot safety monitoring system is used to implement an edge computing-based robot safety monitoring method. The system includes:
[0075] The localization and database construction module is used to acquire images and attitude information of the surveillance scene, determine the target's location information, and establish a historical frame feature cache. This module can be composed of a forward-facing imaging camera, an inertial measurement unit (IMU), a gimbal angle sensor, an edge processor, and local memory. The forward-facing imaging camera acquires images of the surveillance scene; the IMU and gimbal angle sensor output attitude information; the edge processor calculates target location information, assigns target identifiers, and extracts features; and the local memory stores the historical frame feature cache.
[0076] The trend control module determines the short-term motion trend and predicted position of the target based on target positioning information and historical frame feature cache. It then generates gimbal adjustment commands based on the offset between the predicted target position and the center of the field of view, combined with attitude information. The trend control module can consist of an edge processor, a motion analysis unit, a control calculation unit, and a gimbal control interface. The edge processor accesses the historical frame feature cache and target positioning information; the motion analysis unit calculates the short-term motion trend and predicted position of the target; the control calculation unit generates gimbal adjustment commands based on attitude information; and the gimbal control interface sends the control commands to the gimbal actuator.
[0077] The state determination module executes gimbal adjustment commands to acquire updated images and determines the target locking stability index based on the proportion of the target's visible area in the updated image and the historical frame feature cache. The state determination module can consist of a gimbal actuator, an image acquisition unit, an edge processor, and a state evaluation unit. The gimbal actuator rotates according to the gimbal adjustment commands, the image acquisition unit acquires the updated image, the edge processor extracts the target area, target texture features, and local contrast features from the updated image, and the state evaluation unit calculates the target locking stability index in conjunction with the historical frame feature cache.
[0078] The recapture module is used to schedule edge computing resources and recapture the target based on updated images and historical frame feature caches when the target locking stability index is lower than a preset stability threshold. It also updates the target location information and historical frame feature cache. The recapture module can consist of a task scheduling unit, an edge processor, local memory, and a high-priority processing thread management unit. The task scheduling unit adjusts the allocation of edge computing resources when the target locking stability index is lower than the preset stability threshold; the high-priority processing thread management unit establishes a fast processing channel oriented towards the target area; the edge processor completes target recapture, target matching, and location update; and the local memory synchronously updates the historical frame feature cache.
[0079] The feedback output module outputs stable tracking results based on the updated target positioning information and feeds this updated information back to the next round of target short-term motion trend determination and gimbal adjustment command generation. The feedback output module can consist of a result output interface, an edge processor, and a state cache unit. The edge processor generates stable tracking results based on the updated target positioning information. The result output interface sends the tracking results to a display terminal, control terminal, or upper-level management system. The state cache unit stores the updated target positioning information for the current round and feeds it back to the trend control module as input for the next round of motion trend judgment and gimbal adjustment command generation.
[0080] It should be noted that, in this document, the terms "comprising," "including," and any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Specific examples have been used in this document to illustrate the principles and implementation methods of the present invention. These examples are merely for the purpose of helping to understand the method and core ideas of the present invention. The above descriptions are only preferred embodiments of the present invention. It should be pointed out that, due to the limitations of written expression and the objective existence of infinite specific structures, those skilled in the art can make several improvements, modifications, or variations without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, variations, or combinations, or the direct application of the concept and technical solution of the present invention to other situations without modification, should all be considered within the scope of protection of the present invention.
Claims
1. A robot safety monitoring method based on edge computing, characterized in that, The method includes: Acquire images and pose information of the warning scene, determine target location information, and establish a historical frame feature cache; The short-term motion trend and predicted position of the target are determined based on the target positioning information and the historical frame feature cache. Based on the offset between the predicted target position and the center of the field of view, and combined with the attitude information, a gimbal adjustment command is generated. The gimbal adjustment command is executed to obtain an updated image, and the target locking stability index is determined based on the proportion of the target visible area in the updated image and the historical frame feature cache. If the target locking stability index is lower than a preset stability threshold, edge computing resources are scheduled, and the target is recaptured based on the updated image and the historical frame feature cache, and the target positioning information and the historical frame feature cache are updated. Based on the updated target positioning information, output stable tracking results, and feed back the updated target positioning information to the next round of target short-term motion trend determination and gimbal adjustment command generation.
2. The method according to claim 1, characterized in that, The attitude information includes robot attitude information and gimbal attitude information, and the target positioning information includes: The warning scene image is subjected to noise reduction and contrast enhancement processing; Extract target candidate regions from the processed warning scene image; By combining the robot's posture information and the gimbal's posture information, the target center coordinates and target bounding box corresponding to the target candidate region are determined; The target positioning information is determined based on the target center coordinates and the target bounding box.
3. The method according to claim 2, characterized in that, The establishment of the historical frame feature cache includes: Extract the current frame appearance features of the target candidate region; Assign a target identifier to the target candidate region; The current frame appearance features are associated with the target identifier and stored in the historical frame feature cache.
4. The method according to claim 3, characterized in that, Determining the short-term movement trend of the target and the predicted location of the target includes: The target displacement and scale change are determined based on the target center coordinates and the target bounding box in consecutive frames. By combining the historical frame feature cache, temporal filtering is performed on the target displacement and the scale change to obtain a motion state sequence; The target motion direction and predicted velocity are determined based on the motion state sequence; The short-term movement trend of the target and the predicted position of the target are determined based on the target's direction of movement and the predicted speed.
5. The method according to claim 4, characterized in that, The generated gimbal adjustment command includes: The target angular displacement is determined based on the offset between the predicted target position and the center of the field of view. The gimbal rotation direction is determined based on the target angular displacement; The rotation speed of the basic gimbal is determined based on the target angular displacement, the robot posture information, and the gimbal posture information. If the offset is greater than a preset offset threshold, increase the rotation speed of the basic gimbal. The gimbal adjustment command is generated based on the gimbal rotation direction and the base gimbal rotation speed.
6. The method according to claim 5, characterized in that, The step of executing the gimbal adjustment command to obtain an updated image, and determining the target locking stability index based on the proportion of the target visible area in the updated image and the historical frame feature cache, includes: The gimbal is driven to rotate according to the gimbal adjustment command, and the image after the gimbal rotation is acquired is used as the updated image; In the updated image, a target region is identified, and the proportion of the visible area of the target region is determined based on the target region. Extract target texture features and local contrast features from the target region; The target texture features are matched with historical features in the historical frame feature cache to obtain feature matching results; The target locking stability index is determined based on the proportion of the visible area of the target, the local contrast features, and the feature matching results.
7. The method according to claim 6, characterized in that, The scheduled edge computing resources include: The processing priority of the image processing task corresponding to the target region is determined based on the target locking stability index; The edge computing resources are allocated from low-priority tasks to the image processing tasks corresponding to the target region. A high-priority processing thread is established based on the reallocated edge computing resources.
8. The method according to claim 7, characterized in that, The step of recapturing the target based on the updated image and the historical frame feature cache, and updating the target positioning information and the historical frame feature cache, includes: The high-priority processing thread is used to extract the appearance features of the updated frame from the target region in the updated image; The updated frame appearance features are matched with the historical features in the historical frame feature cache to obtain the target matching result; If the target matching result meets the preset matching conditions, determine the matching target identifier corresponding to the target region; The updated target center coordinates and the updated target bounding box are determined based on the target area, and the target positioning information is updated based on the updated target center coordinates and the updated target bounding box. The updated frame appearance features are written into the historical feature record corresponding to the matching target identifier in the historical frame feature cache.
9. The method according to claim 1, characterized in that, The process of outputting stable tracking results and feeding back the updated target positioning information to the next round of target short-term motion trend determination and gimbal adjustment command generation includes: Generate a target location sequence based on the target location information at consecutive time points; The target position sequence is subjected to temporal smoothing processing to obtain the stable tracking result; The target positioning information at the current moment is used as the input for determining the target's short-term motion trend and generating the gimbal adjustment command in the next round.
10. A robot safety monitoring system based on edge computing, used to implement the robot safety monitoring method based on edge computing as described in any one of claims 1-9, characterized in that, The system includes: The positioning and database building module is used to acquire images and pose information of the warning scene, determine the target positioning information, and establish a historical frame feature cache; The trend control module is used to determine the short-term motion trend and predicted position of the target based on the target positioning information and the historical frame feature cache, and to generate gimbal adjustment commands based on the offset between the predicted target position and the center of the field of view, combined with the attitude information. The status determination module is used to execute the gimbal adjustment command to obtain an updated image, and determine the target locking stability index based on the proportion of the target visible area in the updated image and the historical frame feature cache. The recapture module is used to schedule edge computing resources and recapture the target based on the updated image and the historical frame feature cache when the target locking stability index is lower than a preset stability threshold, and update the target positioning information and the historical frame feature cache. The feedback output module is used to output a stable tracking result based on the updated target positioning information, and to feed back the updated target positioning information to the next round of target short-term motion trend determination and gimbal adjustment command generation.