A method and system for intelligent inspection of security robots for multiple scenarios

Abstract

This invention discloses an intelligent inspection method and system for security robots in multiple scenarios, relating to the field of intelligent security. The method includes: in environments with good visibility, collecting vibration data generated by the robot's interaction with the ground to establish a standard fingerprint strip and historical images; in low-visibility environments, the robot collects blurred visual images and vibration data in real time, generating the current inspection fingerprint and matching it with the standard fingerprint strip; when a deviation is detected, the system calls upon a visual memory library and uses a deep learning visual matching network to determine the closest reference landmark, thereby adjusting the robot's turning angle to bring it back to the center path of the fingerprint strip. This invention can be applied to industrial control software and deployed on various models of security robots to achieve process control during inspections. It solves the problem of unstable positioning of security robots during inspections in low-visibility environments, realizing path self-correction and intelligent inspection control based on vibration fingerprint and deep visual matching.

Description

Technical Field

[0001] This invention relates to the field of intelligent security, and more specifically, to an intelligent inspection method and system for security robots for multiple scenarios. Background Technology

[0002] The widespread application of intelligent security systems has led to security robots gradually replacing manual inspections, undertaking security inspection tasks in complex environments such as factories, warehouses, subway stations, tunnels, and underground parking garages. These scenarios often feature complex terrain, numerous obstacles, and poor lighting conditions; some areas even experience reduced visibility due to smoke, dust, or smog. Traditional security inspection robots primarily rely on lidar or visual sensors for path recognition and obstacle avoidance. However, when the environment is dimly lit, reflective, or obstructed, the accuracy of visual recognition significantly decreases, easily leading to problems such as path deviation, lost positioning, or repeated inspections.

[0003] On the other hand, most existing path localization algorithms rely on geometric coordinates or inertial navigation. When optical features decay, tires slip, or ground materials change, their accumulated errors are large, making it difficult to achieve highly stable inspection control. Especially in low-visibility areas such as warehouses, tunnels, or underground spaces, robots cannot rely on visual features for reliable localization.

[0004] Security robots, when deployed across multiple models and platforms, still suffer from strong algorithm dependence and poor environmental adaptability, making it difficult to apply the same inspection strategy across different environments. Improving the robustness of inspection and positioning in complex, low-visibility scenarios, reducing the impact of visual degradation, and maintaining system portability and consistency have become key technical challenges that need to be addressed in the field of intelligent inspection. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for intelligent inspection of security robots for multiple scenarios, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for intelligent inspection of security robots for multiple scenarios includes the following steps:

[0008] In environments with good visibility, when performing standard inspection tasks, the robot collects body vibration data and historical visual images generated by the interaction between the robot and the ground, and establishes a standard fingerprint strip on the inspection path based on the body vibration data. The standard fingerprint strip consists of a central fingerprint sequence and allowable offset ranges on both sides. A visual memory bank is constructed based on the historical visual images.

[0009] In low-visibility environments, the robot collects blurred visual images and body vibration data over a period of time in real time, generates a current inspection fingerprint based on the body vibration data, and matches the current inspection fingerprint with the standard fingerprint strip.

[0010] When the current inspection fingerprint is detected to be outside the allowable offset range on both sides, the visual memory library with pre-established reference landmarks is invoked. Combined with the currently acquired blurry visual image, the reference landmark closest to the current position is determined by a deep learning-based visual matching network. The deviation direction of the robot is determined by the reference landmark in the current image and the image when the robot successfully matches the center position of the standard fingerprint strip in the history.

[0011] Adjust the steering angle according to the correction direction, so that the robot gradually approaches along the center direction of the standard fingerprint strip until the currently inspected fingerprint is located within the standard fingerprint strip;

[0012] The method is applied to industrial control software and deployed on various models of security robots to achieve process control during inspections.

[0013] In some embodiments, the process of establishing a standard fingerprint strip on the inspection path based on fuselage vibration data includes:

[0014] During multiple cycles of the robot's operation along the predetermined inspection path, data on the robot's body vibration generated by its interaction with the ground is collected.

[0015] Multiple vibration signals at the same path location are preprocessed and feature extracted, and the features include vibration amplitude sequence, spectral distribution and time-domain waveform features;

[0016] The vibration features are clustered and fused to extract a representative central vibration feature sequence as a central fingerprint;

[0017] The allowable offset range is determined based on the range of vibration characteristics, forming a standard fingerprint band around the central fingerprint.

[0018] In some embodiments, the process of matching the current inspection fingerprint with the standard fingerprint band includes:

[0019] Calculate the feature similarity with the currently inspected fingerprint within a sliding window along the path of the standard fingerprint strip;

[0020] The path position of the sliding window with the highest feature similarity score is selected as the current matching inspection position, and the standard fingerprint band area at the sliding window position with the highest similarity score is used as the comparison benchmark. If the current inspection fingerprint is within the allowable offset range of the comparison benchmark, the robot's driving state is determined to be normal; otherwise, the current fingerprint feature is determined to have exceeded the standard fingerprint band range and correction is triggered.

[0021] In some embodiments, the method further includes a dual location confirmation step, specifically including:

[0022] The current inspection location is confirmed by matching the current inspection fingerprint with the standard fingerprint strip.

[0023] The deep learning-based visual matching network determines the nearest reference landmark to the current location;

[0024] When the distance between the current matching inspection position and the reference landmark is less than or equal to a preset range, the robot positioning result is determined to be consistent and a normal status is output; otherwise, the positioning result is determined to be abnormal, and the current inspection fingerprint is re-matched with the standard fingerprint strip or the reference landmark is re-determined.

[0025] In some embodiments, the process of determining the nearest reference landmark includes: using a convolutional neural network or a feature pyramid network to perform multi-scale feature extraction on the currently acquired blurred visual image to obtain a current visual feature vector containing low-frequency contours and light intensity distribution.

[0026] Retrieve visual feature vectors of each landmark pre-stored under good visibility conditions from the visual memory bank;

[0027] A deep learning matching network is used to align the current visual feature vector with the visual feature vectors of each landmark in the memory and generate matching confidence.

[0028] The landmark with the highest matching confidence level is selected as the closest reference landmark.

[0029] In some embodiments, the process of calculating the correction direction includes:

[0030] Obtain the image coordinates of the reference landmark in the current frame; retrieve the image coordinates of the reference landmark from the history when the robot successfully matched the center of the standard fingerprint strip at the current matching inspection position as the reference point;

[0031] The correction direction is obtained based on the difference in horizontal and vertical displacement between the image coordinates of the reference landmark in the current frame and the reference point, so that when the robot moves toward the correction direction, the image coordinates of the reference landmark in the current frame gradually move toward the position of the reference point.

[0032] In some embodiments, the criteria for distinguishing between environments with good visibility and environments with low visibility include:

[0033] The ambient light sensor or vision module installed on the robot detects ambient brightness and image clarity in real time.

[0034] When the ambient brightness is higher than a preset threshold and the image clarity index is higher than a preset threshold, it is determined to be an environment with good visibility; otherwise, it is determined to be an environment with low visibility.

[0035] In some embodiments, the image sharpness index is a structural similarity index between the current image and a reference sharp image.

[0036] This invention also discloses an intelligent inspection system for security robots in multiple scenarios, comprising:

[0037] The vibration acquisition module is used to collect data on the robot's body vibration and historical visual images generated by the robot's interaction with the ground in environments with good visibility.

[0038] The fingerprint strip construction module is used to establish a standard fingerprint strip on the inspection path based on the body vibration data. The standard fingerprint strip consists of a central fingerprint sequence and allowable offset ranges on both sides, which are used to characterize the vibration characteristic range of the robot under normal inspection conditions.

[0039] The fingerprint matching module is used to acquire blurred visual images and fuselage vibration data in real time in low visibility environments, generate the current inspection fingerprint, and match the current inspection fingerprint with the standard fingerprint strip.

[0040] The visual matching module is used to call a pre-established visual memory library with reference landmarks when the current inspection fingerprint is detected to be outside the allowable offset range on both sides. Combined with the currently acquired blurry visual image, the module uses a deep learning-based visual matching network to determine the reference landmark closest to the current position. Based on the deviation direction of the reference landmark in the current image and the historical image when the robot successfully matched to the center position of the standard fingerprint strip, the module determines the correction direction required by the robot.

[0041] The control module is used to adjust the steering angle according to the correction direction, so that the robot gradually approaches along the center direction of the standard fingerprint strip until the currently inspected fingerprint is once again located within the standard fingerprint strip.

[0042] In some embodiments, the system is embedded in industrial control software for deployment on various models of security robots to achieve process control during inspections.

[0043] The advantage of this invention over existing technologies lies in its pioneering use of robot vibration signals generated by interaction with the ground as the core identification feature. Unlike traditional visual or radar signals, this vibration signal does not rely on external lighting or surface reflection, thus enabling continuous acquisition and construction of stable path representations in low-visibility environments such as smoke, haze, and insufficient light. Through multi-cycle sampling and deep learning clustering analysis, the system establishes a standard fingerprint band consisting of central vibration features and permissible offset ranges on both sides, allowing the robot to achieve path recognition and offset detection based on its own body feature signals.

[0044] When environmental visual features degrade, the system calls upon a pre-established visual memory library and uses a deep learning visual matching network to perform multi-scale feature extraction and fuzzy matching on the blurred image. Based on the deviation relationship between reference landmarks in the current and historical images, the system determines the robot's correction direction, achieving automatic path correction. This invention does not rely on precise direction determination by the vision system during the correction phase; it only needs to obtain the approximate deviation direction based on the fuzzy image matching results. Combined with vibration fingerprint matching, it can determine whether the robot has returned to the standard inspection path, thus achieving robust correction and path maintenance in low-visibility environments.

[0045] Furthermore, the system of the present invention uses a sliding window matching algorithm to dynamically search for the most similar vibration feature window on the standard fingerprint strip to accurately determine the current path position of the robot; at the same time, it combines a dual position confirmation mechanism to compare the vibration fingerprint positioning and visual landmark positioning results by distance, ensuring that the two are consistent within a preset tolerance range, thereby significantly improving the robustness and safety of inspection positioning.

[0046] The algorithm module of this invention can be embedded in industrial control software, possessing excellent portability and adaptability to various models of security robots, enabling a unified inspection control strategy across multiple platforms. By fusing robot-specific interactive signals such as body vibration with deep visual memory, this invention maintains high-precision path control and positioning stability in complex terrain, varying lighting conditions, and low visibility environments, demonstrating significant engineering practical value and promotional significance.

[0047] In summary, this invention not only maintains stable inspection path control in complex terrain and low visibility environments, but also improves the system's cross-device availability and environmental adaptability, demonstrating significant engineering practical value and promotion potential. Attached Figure Description

[0048] Figure 1 This is an overall schematic diagram of the invention;

[0049] Figure 2 This is a schematic diagram of the fingerprint strip construction of the present invention;

[0050] Figure 3This is a schematic diagram of the low visibility matching and correction method of the present invention;

[0051] Figure 4 This is a schematic diagram of the visual memory and dual position confirmation of the present invention. Detailed Implementation

[0052] The specific embodiments of the present invention will now be described with reference to the accompanying drawings.

[0053] To achieve intelligent inspection of security robots in multiple scenarios, such as Figure 1 As shown, the present invention includes the following steps:

[0054] In environments with good visibility, when performing standard inspection tasks, the robot collects body vibration data and historical visual images generated by the interaction between the robot and the ground, and establishes a standard fingerprint strip on the inspection path based on the body vibration data. The standard fingerprint strip consists of a central fingerprint sequence and allowable offset ranges on both sides, and a visual memory bank is built based on historical visual images.

[0055] In low-visibility environments, the robot collects blurry visual images and body vibration data over a period of time in real time, generates the current inspection fingerprint based on the body vibration data, and matches the current inspection fingerprint with the standard fingerprint strip.

[0056] When the current inspection fingerprint is detected to be outside the allowable offset range on both sides, the visual memory library with pre-established and labeled reference landmarks is called. Combined with the currently acquired blurry visual image, the visual matching network based on deep learning is used to determine the reference landmark closest to the current position. The deviation direction of the robot is determined by the image when the robot successfully matches the center position of the standard fingerprint strip in the current image and the history.

[0057] Adjust the steering angle according to the correction direction, so that the robot gradually approaches the center of the standard fingerprint strip until the current fingerprint being inspected is located within the standard fingerprint strip;

[0058] The method of this invention can be applied to industrial control software and deployed on various models of security robots to achieve process control of inspection.

[0059] In a specific embodiment, firstly, in an environment with good visibility, the security robot performs a standard inspection task along a predetermined inspection path. This process requires multiple operating cycles to ensure sufficient and reliable data. The robot is typically equipped with accelerometers, gyroscopes, ground contact pressure sensors, and ambient light sensors to capture its dynamic interaction characteristics with the ground. Vibration signals mainly refer to the micro-vibration acceleration signals caused by ground materials, obstacles, and road undulations during the robot's chassis or wheelset movement, originating from a three-axis accelerometer installed on the chassis or body. These vibration signals exhibit periodic or random acceleration fluctuations in the time domain and reflect the energy distribution characteristics of different ground contact states in the frequency domain. By analyzing the vibration amplitude, spectral morphology, and temporal characteristics, a unique fingerprint can be formed. This fingerprint reflects the robot's contact consistency along the inspection path and is the key data foundation for establishing a standard fingerprint band and determining the direction of deviation in this invention. Meanwhile, the high-resolution camera installed at the front of the robot will simultaneously collect historical visual images. These images cover building walls, equipment, ground markings, pipeline layouts and other fixed features along the path, ensuring that each location point is associated with clear visual information.

[0060] The collected vibration data of the robot body undergoes preliminary filtering to remove high-frequency noise and sudden interference. It is then segmented and stored according to the path location, establishing a correspondence between timestamps and position coordinates. For multiple vibration signals collected at the same path location within different periods, preprocessing is performed, including normalization to eliminate the influence of minor differences in robot load or speed, before key features are extracted.

[0061] In a further embodiment, in order to establish a stable and reliable standard fingerprint strip on the inspection path, the system performs statistical analysis and feature extraction based on the body vibration data collected by the robot during multiple cycles, and extracts a representative central vibration feature sequence through a simple and efficient similarity analysis method.

[0062] First, as the robot repeatedly runs along a predetermined inspection path for multiple cycles, the system continuously collects vibration signals generated by the robot's interaction with the ground. The vibration data is acquired by accelerometers mounted on the chassis or wheels, reflecting the minute impacts experienced by the robot during movement and the characteristics of ground feedback. To ensure data comparability, the data from each inspection cycle is synchronized and aligned according to path distance or time segment.

[0063] Secondly, multiple vibration signals from the same path location are preprocessed. Preprocessing includes removing abnormal spikes, mean normalization, and bandpass filtering to eliminate motor noise and environmental interference. Subsequently, three types of basic features are extracted: vibration amplitude sequence, reflecting the ground roughness at that location; spectral distribution characteristics, obtained through Fast Fourier Transform (FFT) to determine the distribution of vibration energy across different frequency bands, used to distinguish ground materials or hardness; and time-domain waveform characteristics, recording the continuous pattern of vibration changes and reflecting the stability of the ground contact mode.

[0064] After feature extraction, in one embodiment, a clustering method based on feature similarity is used to identify stable features. Specifically, in multiple vibration feature sets at the same path location, the cosine similarity or Euclidean distance between any two sets of features is calculated, and the average similarity of each feature with other samples is statistically analyzed. The set of features with the highest similarity is considered the most representative central feature at that location, i.e., the central fingerprint. By sequentially extracting the central fingerprints at each location along the inspection path, a complete central fingerprint sequence is formed.

[0065] To reflect subtle changes that may occur during normal inspections, the system determines the permissible offset range for each central fingerprint based on the maximum and minimum deviations of historical characteristics. All central fingerprints and their offset ranges together constitute a standard fingerprint band. The standard fingerprint band is continuously distributed along the path, with the central fingerprint representing the ideal inspection state and the offset range defining the acceptable normal fluctuation range for the robot.

[0066] In another embodiment, such as Figure 2 As shown, a deep learning model is used for clustering and feature fusion. Specifically, an architecture combining a variational autoencoder and a long short-term memory network is employed. First, the vibration feature sequence of each cycle is input into the encoder to generate low-dimensional latent representations, which capture the essential distributional differences of the vibration signals. Next, a Gaussian mixture model is used to cluster all latent representations, identifying the densest cluster as the core group for normal inspection states. The mean sequence from this cluster is calculated as the center fingerprint, and this sequence is continuously arranged along the path to form the center fingerprint sequence. The training process is performed offline after collecting at least ten cycles of data, using mean squared error as the reconstruction loss, combined with KL divergence regularization to ensure smoothness of the latent space. The initial learning rate is set to 0.001, and the Adam optimizer is used for two thousand iterations until convergence. Training data augmentation includes adding Gaussian noise to simulate sensor drift and ensure model robustness.

[0067] In a further embodiment, historical visual images undergo preprocessing during the construction of the visual memory bank, including distortion correction, illumination normalization, and resolution unification. Significant and stable features along the path, such as pillars, corners, or fixed ground objects, are selected as reference landmarks. These landmarks are annotated with bounding boxes and keypoint coordinates in the images, and multi-scale visual feature vectors are extracted using a feature pyramid network. The network architecture includes a ResNet50 as the backbone, with a feature pyramid module added at the top to fuse shallow high-resolution features with deep semantic features, outputting a 512-dimensional descriptor. The landmark feature vectors of all historical images, along with location labels and the image coordinates at the center of the corresponding standard fingerprint band, are stored in the visual memory bank.

[0068] In a further embodiment, when the environment switches to low visibility, such as in foggy, nighttime, or smoky scenes, the robot measures the illuminance value using an ambient light sensor and simultaneously calculates a structural similarity index for the current image. The structural similarity index is calculated by comparing the current image with a pre-stored clear reference image, calculating the mean difference in brightness, the standard deviation ratio of contrast, and the structural correlation coefficient, and then taking a weighted average of these three values ​​to obtain a value between 0 and 1. In some embodiments, if the brightness is below 50 lux or the structural similarity is below 0.6, it is determined to be a low visibility environment, and the robot switches to a positioning mode that integrates vibration and vision.

[0069] like Figure 3 As shown, in low-visibility environments, the robot continues to collect body vibration data over a period of time, such as the signal within the sliding window of the most recent 30 seconds. It also performs preprocessing and feature extraction to generate the current inspection fingerprint.

[0070] In a further embodiment, a sliding window matching mechanism is used to compare the similarity between the current inspected fingerprint and each local fingerprint segment in the standard fingerprint band, thereby determining the robot's current position and running status on the inspection path.

[0071] Specifically, a sliding window is preset on the path sequence of the standard fingerprint strip, and the window length corresponds to the duration of the vibration signal currently collected by the robot. The similarity between the current feature vector and the reference feature vectors corresponding to each sliding window in the standard fingerprint strip is calculated sequentially. In this embodiment, a cosine similarity algorithm is preferably used, which measures the degree of similarity between two feature vectors in high-dimensional space by calculating the cosine value of the angle between them. The closer the cosine value is to 1, the more consistent the feature directions of the two vectors are, meaning the robot's current vibration mode is closer to the standard state.

[0072] The system calculates the similarity score of all windows within the entire sliding window range and selects the window with the highest similarity as the current matching inspection position. The standard fingerprint strip area corresponding to this window is determined as the comparison benchmark. Next, the system further compares whether the difference between the current feature and the comparison benchmark is within the allowable deviation range. If the similarity score is high and the deviation is within the allowable range, the robot's driving state is determined to be normal, and it continues to run according to the predetermined path; if the similarity drops significantly or the deviation exceeds the threshold, it indicates that the robot may have deviated from the inspection path. At this time, the system will trigger a correction process, and the control module will adjust the wheel speed difference or steering angle according to the deviation direction, so that the robot gradually returns to the center area of ​​the standard fingerprint strip.

[0073] In the specific correction process, it is necessary to first determine the closest reference landmark. This invention first utilizes a convolutional neural network or feature pyramid network to extract multi-scale features from the blurred visual image captured by the current camera. Because images in low-visibility environments often suffer from reduced contrast, blurred edges, and uneven local illumination, traditional single-scale features are insufficient to accurately capture the structural information of the scene. The multi-scale feature extraction mechanism of this invention extracts visual features at different resolution levels, simultaneously focusing on local texture and global contours, with particular emphasis on the contour morphology and light intensity distribution of low-frequency components. This allows the system to extract stable scene features from blurred images even under conditions such as smoke, haze, or low lighting.

[0074] After feature extraction, the system retrieves pre-stored landmark features from the visual memory database under good visibility conditions. Each landmark corresponds to a set of visual feature vectors extracted by a deep learning model. These vectors record core information such as the landmark's shape, grayscale gradient, and illumination distribution under clear conditions. To ensure robust matching, the visual memory database is typically built by learning from images of the same landmark from multiple angles under different lighting conditions, giving the landmark features a certain degree of generalization ability and enabling them to adapt to environmental changes.

[0075] During the matching phase, the system employs a deep learning matching network to align the feature vector of the current blurred image with the feature vectors of each landmark in the memory database. This network learns the distribution patterns of landmarks in the feature space, automatically capturing the differences between different landmarks and calculating their similarity. The matching confidence score output by the network reflects the degree of similarity between the current visual image and each landmark; a higher confidence score indicates that the current environment is closer to the visual features of that landmark.

[0076] Ultimately, the system selects the landmark with the highest matching confidence as the closest reference landmark. The robot then uses this landmark as an anchor point for visual localization to help determine its position on the inspection path. When there is a deviation between environmental features and historical landmarks, the system can also adjust the robot's turning angle based on the deviation direction information, achieving dynamic correction. In this way, the robot can achieve stable and reliable visual localization through deep learning feature matching even in situations of blurred or partially occluded vision, significantly improving inspection accuracy and path continuity in low-visibility environments.

[0077] The aforementioned deep learning matching network undertakes the task of accurately aligning fuzzy features with remembered features. The network as a whole employs a two-branch feature matching structure based on a fusion of convolutional neural networks (CNNs) and an attention mechanism to ensure high recognition capability and robustness even under low visibility and low contrast conditions.

[0078] First, the network input consists of two sets of features: one set is the multi-scale feature vector obtained from the current blurred visual image through a feature extraction network, such as the Feature Pyramid Network (FPN); the other set is the standard feature vector of landmarks in the visual memory. These two sets of features are then fed into a shared-weight convolutional backbone network. This backbone network comprises convolutional layers, batch normalization layers, non-linear activation layers, and downsampling layers. A typical structure can be found in ResNet or MobileNet, used to extract stable spatial hierarchical features and eliminate noise interference.

[0079] In the intermediate layer, the system introduces a feature alignment module. This module dynamically adjusts the response weights of different feature layers through channel attention and spatial attention mechanisms, enabling the network to strengthen its focus on key regions, such as landmark outlines and brightness gradient boundaries, while automatically reducing weights in blurry or low-contrast regions. This ensures that even when the input image is unclear, the network can still focus on the visual information that contributes most to recognition.

[0080] Subsequently, the network enters the feature matching and similarity estimation stage. This stage employs a feature difference and feature interaction mechanism: the two sets of features are aligned layer by layer at multiple scales, their difference features are calculated, and similarity features are extracted through a set of shallow convolutional and fully connected layers. The final output layer uses a Sigmoid or Softmax activation function to map the similarity result to a matching confidence score between 0 and 1. The landmark with the highest matching confidence score is selected as the reference landmark.

[0081] After determining the reference landmark, it is necessary to determine the correction direction. In this embodiment, the process of calculating the correction direction is based on the positional changes of the reference landmark in the image coordinate system obtained by visual recognition. By comparing the relative positional differences of the same landmark in the current frame and the historical reference frame, the system can determine the direction of the robot's current deviation from the standard inspection path, thereby generating a correction command to gradually return the robot to the center area of ​​the standard fingerprint strip.

[0082] Specifically, when the robot performs inspection tasks in low-visibility environments, the camera acquires real-time images of the scene ahead and identifies reference landmarks through a visual matching network. The system extracts the coordinates of the landmark in the current frame image, for example, by establishing a two-dimensional coordinate system with the image center as the origin, where the horizontal axis corresponds to the robot's left and right directions and the vertical axis corresponds to the forward and backward directions.

[0083] Simultaneously, the system call history uses the image coordinates of the reference landmark at the point when the robot successfully matched the center of the standard fingerprint strip at the current matching inspection position as a reference point. This reference point represents the correct visual position of the landmark observed by the robot under the corresponding position of the ideal path in the history. Subsequently, the system calculates the displacement difference between the landmark coordinates and the reference point coordinates in the current frame in the horizontal and vertical directions. If the landmark shifts to the left relative to the reference point in the current image, it indicates that the robot is biased to the right relative to the standard path; if the landmark shifts upward or downward in the vertical direction, it indicates that there is an error in the robot's forward and backward position.

[0084] Based on this, the system integrates the displacement differences in the horizontal and vertical directions into a correction direction vector. The control module adjusts the steering angle and wheel speed difference of the robot chassis accordingly, causing it to gradually correct its path in the opposite direction. As the robot moves, the position of the landmark in the camera image gradually moves closer to the reference point. Visual recognition is discontinued once the currently inspected fingerprint is within the standard fingerprint band, reducing reliance on visual recognition in low-visibility conditions.

[0085] This method offers advantages such as intuitiveness and real-time performance, achieving correction control without the need for complex modeling or deep network estimation. By directly correcting orientation based on image coordinate differences, the system can maintain high path stability and visual alignment accuracy in environments with smoke, haze, or changing lighting.

[0086] In a further embodiment, such as Figure 4 As shown, a dual position confirmation mechanism is introduced to enhance reliability. First, the current inspection position is obtained through vibration fingerprint matching. Second, the corresponding path position of the reference landmark is obtained through visual matching. If the distance between the two is less than a preset range, such as 1 meter, they are confirmed to be consistent, and a normal status is output to continue the inspection. If the distance exceeds the limit, it may be due to a vibration sensor malfunction or visual mismatch, immediately triggering a rematch of the vibration fingerprint or a re-determination of the landmark.

[0087] The entire process of this invention can be embedded in industrial control software, and its modular design allows it to be adapted to different models of security robots. The vibration acquisition module integrates sensor drivers, the fingerprint strip construction module runs on an edge computing unit, the fingerprint matching module and the vision matching module share GPU acceleration, and the control module outputs PWM signals to the motor driver. The software supports parameterized configuration, such as offset thresholds or window lengths, facilitating deployment in various scenarios such as factories, warehouses, or outdoors, and enabling autonomous inspection process control.

[0088] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent inspection of security robots for multiple scenarios, characterized in that, Includes the following steps: In environments with good visibility, when performing standard inspection tasks, the robot collects body vibration data and historical visual images generated by the interaction between the robot and the ground, and establishes a standard fingerprint strip on the inspection path based on the body vibration data. The standard fingerprint strip consists of a central fingerprint sequence and allowable offset ranges on both sides. A visual memory bank is constructed based on the historical visual images. In low-visibility environments, the robot collects blurred visual images and body vibration data over a period of time in real time, generates a current inspection fingerprint based on the body vibration data, and matches the current inspection fingerprint with the standard fingerprint strip. When the current inspection fingerprint is detected to be outside the allowable offset range on both sides, the visual memory library with pre-established reference landmarks is invoked. Combined with the currently acquired blurry visual image, the reference landmark closest to the current position is determined by a deep learning-based visual matching network. The deviation direction of the robot is determined by the reference landmark in the current image and the image when the robot successfully matches the center position of the standard fingerprint strip in the history. Adjust the steering angle according to the correction direction, so that the robot gradually approaches along the center direction of the standard fingerprint strip until the currently inspected fingerprint is located within the standard fingerprint strip; The method is applied to industrial control software and deployed on various models of security robots to achieve process control of inspection. When matching the current inspection fingerprint with the standard fingerprint strip, the feature similarity of the current inspection fingerprint is calculated within the sliding window along the path of the standard fingerprint strip. The path position of the sliding window with the highest feature similarity score is selected as the current matching inspection position. At the same time, the reference landmark closest to the current position is determined through a deep learning visual matching network, and the distance between the current matching inspection position and the reference landmark is compared. When the distance between the current inspection location and the reference landmark is less than or equal to the preset range, the robot's positioning result is determined to be consistent; otherwise, the positioning result is determined to be abnormal, and the current inspection fingerprint is re-matched with the standard fingerprint strip or the reference landmark is re-determined.

2. The intelligent inspection method for security robots oriented towards multiple scenarios according to claim 1, characterized in that, The process of establishing a standard fingerprint strip along the inspection path based on fuselage vibration data includes: During multiple cycles of the robot's operation along the predetermined inspection path, data on the robot's body vibration generated by its interaction with the ground is collected. Multiple vibration signals at the same path location are preprocessed and feature extracted, and the features include vibration amplitude sequence, spectral distribution and time-domain waveform features; Clustering and feature fusion are performed on the features to extract a representative central vibration feature sequence as the central fingerprint; The allowable offset range is determined based on the range of vibration characteristics, forming a standard fingerprint band around the central fingerprint.

3. The intelligent inspection method for security robots oriented towards multiple scenarios according to claim 1 or 2, characterized in that, The standard fingerprint band area at the sliding window position with the highest similarity score is used as the comparison benchmark. If the currently inspected fingerprint is within the allowable offset range of the comparison benchmark, the robot's driving state is determined to be normal. Otherwise, it is determined that the current fingerprint feature has exceeded the standard fingerprint band range and correction is triggered.

4. The intelligent inspection method for security robots oriented towards multiple scenarios according to claim 3, characterized in that, The process of determining the closest reference landmark includes: using a convolutional neural network or feature pyramid network to extract multi-scale features from the currently acquired blurred visual image to obtain a current visual feature vector containing low-frequency contours and light intensity distribution; Retrieve visual feature vectors of each landmark pre-stored under good visibility conditions from the visual memory bank; A deep learning matching network is used to align the current visual feature vector with the visual feature vectors of each landmark in the memory and generate matching confidence. The landmark with the highest matching confidence level is selected as the closest reference landmark.

5. The intelligent inspection method for security robots oriented towards multiple scenarios according to claim 4, characterized in that, The process of calculating the correction direction includes: Obtain the image coordinates of the reference landmark in the current frame; retrieve the image coordinates of the reference landmark from the history when the robot successfully matched the center of the standard fingerprint strip at the current matching inspection position as the reference point; The correction direction is obtained based on the difference in horizontal and vertical displacement between the image coordinates of the reference landmark in the current frame and the reference point, so that when the robot moves toward the correction direction, the image coordinates of the reference landmark in the current frame gradually move toward the position of the reference point.

6. The intelligent inspection method for security robots oriented towards multiple scenarios according to claim 1, characterized in that, The criteria for distinguishing between environments with good visibility and environments with low visibility include: The ambient light sensor or vision module installed on the robot detects ambient brightness and image clarity in real time. When the ambient brightness is higher than a preset threshold and the image clarity index is higher than a preset threshold, it is determined to be an environment with good visibility; otherwise, it is determined to be an environment with low visibility.

7. The intelligent inspection method for security robots oriented towards multiple scenarios according to claim 6, characterized in that, The image sharpness index is a structural similarity index between the current image and a reference sharp image.

8. A system for implementing the intelligent inspection method for security robots oriented towards multiple scenarios as described in claim 1, characterized in that, include: The vibration acquisition module is used to collect data on the robot's body vibration and historical visual images generated by the robot's interaction with the ground in environments with good visibility. The fingerprint strip construction module is used to establish a standard fingerprint strip on the inspection path based on the body vibration data. The standard fingerprint strip consists of a central fingerprint sequence and allowable offset ranges on both sides, which are used to characterize the vibration characteristic range of the robot under normal inspection conditions. The fingerprint matching module is used to acquire blurred visual images and fuselage vibration data in real time in low visibility environments, generate the current inspection fingerprint, and match the current inspection fingerprint with the standard fingerprint strip. The visual matching module is used to call a pre-established visual memory library with reference landmarks when the current inspection fingerprint is detected to be outside the allowable offset range on both sides. Combined with the currently acquired blurry visual image, the module uses a deep learning-based visual matching network to determine the reference landmark closest to the current position. Based on the deviation direction of the reference landmark in the current image and the historical image when the robot successfully matched to the center position of the standard fingerprint strip, the module determines the correction direction required by the robot. The control module is used to adjust the steering angle according to the correction direction, so that the robot gradually approaches along the center direction of the standard fingerprint strip until the currently inspected fingerprint is once again located within the standard fingerprint strip.

9. The system according to claim 8, characterized in that, The system is embedded in industrial control software and is used to deploy on various models of security robots to achieve process control of inspection.

Images