Event monitoring method under tunnel track type inspection robot environment

By using monocular camera and IMU data in a tunnel rail-type inspection robot, combined with the encoder-decoder model to generate a depth map, the problem of event monitoring accuracy and three-dimensional information acquisition in the tunnel environment is solved, and the tunnel is realized Accurate monitoring of events and three-dimensional coordinate calculations improve tunnel safety.

CN119152452AActive Publication Date: 2024-12-17OB TELECOM ELECTRONICS

Patent Information

Application Number
CN202411642168.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-18
Publication Date
2024-12-17
Estimated Expiration
2044-11-18

AI Technical Summary

Technical Problem

The prior art is difficult to achieve real-time, comprehensive and accurate event monitoring and location determination in tunnel environments, especially the inability to provide three-dimensional spatial information, which poses safety risks.

Method used

A monocular camera is used to capture and pre-process the road surface image, extract features through CSPDarknet and jump connection network, combine IMU data to predict camera pose changes, use an encoder-decoder to generate depth maps and reproject, calculate the three-dimensional coordinates of the object, and realize the matching of the object. Accurate monitoring of events in the tunnel.

Benefits of technology

It improves the accuracy of event detection, overcomes the shortcomings of the monocular vision system in depth estimation, realizes accurate monitoring of various traffic events in the tunnel and calculates three-dimensional coordinates, and enhances tunnel safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119152452A_ABST
    Figure CN119152452A_ABST
Patent Text Reader

Abstract

The invention provides an event monitoring method in a tunnel track type inspection robot environment, and relates to the technical field of tunnel monitoring and detection.The method comprises the steps that S1, a monocular camera is installed on a track, and a road surface image is shot and preprocessed in real time; s2, extracting image features through a CSPDarknet network, and obtaining information such as a target bounding box and segmentation mask coordinates by using a detection head and a segmentation head; s3, performing nearest neighbor matching on the segmentation mask points of the front and back frames, and predicting camera attitude change in combination with IMU data; s4, estimating the depth map based on an encoder-decoder model, and generating multi-view depth information; and S5, detecting a tunnel event according to the camera and the target object information and triggering an alarm. According to the invention, the motion posture and depth estimation of the monocular camera are improved, and the event detection accuracy is improved. Object feature points are obtained through target detection, KNNMatch and IMU data, camera motion is predicted, meanwhile, a depth map is generated and re-projected to multiple view angles, and tunnel events are accurately monitored.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel monitoring and detection, and in particular to an event monitoring method in a tunnel track inspection robot environment. Background Art

[0002] With the popularization of tunnel construction in transportation and infrastructure, the safety and maintenance needs of the tunnel environment are increasing. The internal space of the tunnel is limited, the environment is complex and the conditions are harsh. It is easy to have problems such as water seepage, cracks, foreign body intrusion, and fire hazards, threatening the structural safety of the tunnel and the lives of people passing through. Traditional tunnel inspections usually rely on manual work, but the tunnel environment is often insufficiently lit, and there are harsh conditions such as high humidity, high temperature, and toxic gases. The working environment and efficiency of manual inspections are greatly limited, making it difficult to achieve timely and comprehensive monitoring, and there are safety hazards.

[0003] For example, a Chinese patent discloses "an intelligent on-board monitoring system for traffic inspection vehicles" (patent number: CN118247761B). Through the video system carried by the traffic inspection vehicle, the target detection algorithm is used to monitor the road and vehicle dynamics in real time. The system will grid the captured images and redraw the vehicle outline, and then analyze the vehicle distance, time and speed information in the image data. For the identified vehicles that are too close, the system will issue an early warning and specially mark it, which can be used to monitor traffic violations, accidents and road conditions. This method is based on two-dimensional technology and lacks the estimation of the actual physical coordinates and distance of the target in three-dimensional space, so it cannot provide the target's location information for subsequent disposal work.

[0004] For example, a Chinese patent discloses "UAV gimbal adjustment control system and method for traffic inspection" (patent number: CN118759833A). The technical solution for calculating the target position is to use a drone to perform remote sensing photography of traffic infrastructure at low altitude, and perform correction processing after obtaining the image. Subsequently, the corrected image is input into the Mask-RCNN model to identify the characteristic elements along the traffic infrastructure. This model can obtain the bounding box coordinates of the target object, and calculate the center point of the entire target image and the center position of the target object in the image. However, the position calculated by this method is the position of the 2D plane, and the position information of the target in the physical space cannot be determined.

[0005] For example, a Chinese patent discloses "a method for determining traffic congestion based on the perspective of a drone" (patent number: CN118038673A). Based on the yolov8 target detection algorithm and other methods, the vehicle in the environmental image is identified, the identified vehicle is tracked, and the position difference of the current vehicle in two key frame images is obtained, and then the moving speed of the vehicle is calculated. This method is applied when the drone is stationary and cannot adapt to dynamic environments.

[0006] For example, a Chinese patent discloses "a vehicle illegal parking detection method based on real-time images of drone inspection" (patent number: CN117593673A). The drone is used to cruise the highway, and the video stream it shoots is sent to the background for frame-by-frame processing. The deep learning model is used to detect the vehicle in the video frame, and the upper left corner coordinates and lower right corner coordinates of the vehicle detection frame are obtained, and the lane line segmentation area, highway segmentation area and vehicle segmentation area are segmented; the prediction results of the YOLOv5 model and the STDC model are combined, and the specific judgment rules for illegal parking of the vehicle are used to determine whether the vehicle is illegally parked. This method uses dual models, has high computational complexity, poor real-time performance, and fails to provide 3D spatial information.

[0007] For example, a Chinese patent discloses "an environmental inspection system based on multi-sensor fusion of unmanned aerial vehicles" (patent number: CN118687630A). It uses a positioning module, a laser radar acquisition module, an image acquisition module, an infrared thermal imaging module, and a smoke detection module. It can automatically patrol according to conventional path planning strategies or emergency path planning strategies. The image acquisition module, infrared thermal imaging module, and smoke detection module communicate to obtain whether an emergency event occurs. If an emergency event occurs, the positioning module and the laser radar acquisition module work synchronously to obtain the location of the event. This method has high equipment costs and is not convenient for large-scale application.

[0008] Therefore, developing an event monitoring method based on a track-mounted inspection robot that can accurately identify various safety hazards in a tunnel environment and trigger alarms in a timely manner is of great significance to improving tunnel safety and reducing inspection blind spots. Summary of the invention

[0009] In order to solve the technical problem that the tunnel environment inspection method in the prior art is difficult to achieve real-time, comprehensive and accurate event monitoring and position determination in a complex environment, the present invention provides an event monitoring method in a tunnel track inspection robot environment.

[0010] The technical solution provided by the present invention is as follows: The present invention provides an event monitoring method in a tunnel track inspection robot environment, comprising: S1. Install a monocular camera on the track and take real-time road images, and adjust the road images to After the size is determined, preprocessing is performed, wherein the preprocessing includes normalization processing; S2. Input the preprocessed image into CSPDarknet to obtain the preliminary feature representation of the image, then extract different scale features through skip connection and feature pyramid network, and then build three detection heads and one segmentation head according to the different scale features. The input image data is subjected to confidence value and non-maximum suppression to eliminate targets with low confidence and high overlap ratio, and finally output the bounding box, segmentation mask coordinates, category label and feature points of the effective predicted target; S3, extracting corner points and weighted center points from the segmentation mask points with high confidence and consistent labels in the two frames A and B, and finding the nearest point in the latter frame for each of the mask points k Points are located in the center of the image, and the Euclidean distance is used to calculate the distance ratio between the nearest neighbor and the next nearest neighbor, and the validity of the match is determined. If the match is valid, the IMU data is combined to use the camera's motion and rotation information to predict the camera's position change. S4, intercept the maximum boundary range of the detected target as the effective image, use the encoder-decoder method to estimate the depth value of the effective image, extract image features from the effective image through the ResNet50 encoder, obtain deep multi-level features, then perform deconvolution and upsampling operations on the extracted multi-level features to generate a depth map, calculate the disparity of the two frames A and B in the scene, and the disparity includes angle and position changes. Then, use the depth map generated by the network to reproject the input image to other perspectives, and output the depth value of each pixel of the target in combination with the camera posture and internal parameters; S5. Detect the target object on the road surface according to the information of the camera and the information of the target object obtained in the previous step, and automatically analyze and issue a corresponding event alarm.

[0011] The beneficial effects brought about by the technical solution provided by the present invention include at least: In the present invention, the accuracy of event detection is improved by improving the motion posture estimation and object depth estimation of the monocular vision camera. The feature points of the object are obtained through target detection and segmentation, and the camera motion posture is accurately predicted by combining KNNMatch and IMU, which overcomes the shortcomings of the monocular vision system in depth estimation. At the same time, the image encoding-decoding model is used to generate a depth map and reproject it to different perspectives, accurately calculate the three-dimensional coordinates of the object, and realize accurate monitoring of various traffic events in the tunnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic flow chart of an event monitoring method in a tunnel track inspection robot environment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0015] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0016] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0017] In the embodiments of the present invention, sometimes the subscripts are as follows It may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.

[0018] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0019] Reference Manual Attached Figure 1 , showing a flow chart of an event monitoring method in a tunnel track inspection robot environment provided by an embodiment of the present invention.

[0020] The embodiment of the present invention provides an event monitoring method in a tunnel track inspection robot environment, and the processing flow may include the following steps: S1. Install a monocular camera on the track and take real-time road images, and adjust the road images to After the size is determined, preprocessing is performed, wherein the preprocessing includes normalization processing; S2. Input the preprocessed image into CSPDarknet to obtain the preliminary feature representation of the image, then extract different scale features through skip connection and feature pyramid network, and then build three detection heads and one segmentation head according to the different scale features. The input image data is subjected to confidence value and non-maximum suppression to eliminate targets with low confidence and high overlap ratio, and finally output the bounding box, segmentation mask coordinates, category label and feature points of the effective predicted target; S3, extracting corner points and weighted center points from the segmentation mask points with high confidence and consistent labels in the two frames A and B, and finding the nearest point in the latter frame for each of the mask points k Points are located in the center of the image, and the Euclidean distance is used to calculate the distance ratio between the nearest neighbor and the next nearest neighbor, and the validity of the match is determined. If the match is valid, the IMU data is combined to use the camera's motion and rotation information to predict the camera's position change. S4, intercept the maximum boundary range of the detected target as the effective image, use the encoder-decoder method to estimate the depth value of the effective image, extract image features from the effective image through the ResNet50 encoder, obtain deep multi-level features, then perform deconvolution and upsampling operations on the extracted multi-level features to generate a depth map, calculate the disparity of the two frames A and B in the scene, and the disparity includes angle and position changes. Then, use the depth map generated by the network to reproject the input image to other perspectives, and output the depth value of each pixel of the target in combination with the camera posture and internal parameters; S5. Detect the target object on the road surface according to the information of the camera and the information of the target object obtained in the previous step, and automatically analyze and issue a corresponding event alarm.

[0021] In a possible implementation manner, the S3 specifically includes: For each mask point (x 1i ,y 1i ), find the k points (x) closest to each other in image B 2i ,y 2i ), using Euclidean distance as the distance between two feature points:

[0022] in, i and j Take 1 to 5, and Represent the mask points of images A and B respectively, and the validity of the match is judged by the ratio test, which is the distance ratio between the nearest neighbor and the second nearest neighbor:

[0023] If the ratio is less than the set threshold , then the point match is considered valid, and the matching result of each mask point is recorded, including the matched point and the corresponding distance.

[0024] In a possible implementation, predicting the camera pose change includes: Assume the initial camera pose is , including translation and rotation information, and obtaining time series data from the IMU, including current acceleration and angular velocity , assuming that the acceleration and angular velocity in each time period are constant, divide the time interval into small segments , integrate in each time period, and update the rotation, translation, and velocity separately through integration: Update the rotation, angular velocity Convert to rotation matrix R:

[0025] Update the translation and update the camera position by the acceleration a:

[0026] in, v is the speed of the camera at the last moment, Update speed:

[0027] Updated pose By combining the current pose and the change:

[0028] Repeat the process of dividing the time interval into small segments to obtain the updated pose until all IMU data are processed, and then fuse the pose result of image matching with the pose predicted by IMU.

[0029] In a possible implementation, when the pose result of image matching and the pose predicted by IMU are fused, Kalman filtering is used to reduce noise and improve the accuracy of pose estimation.

[0030] In a possible implementation manner, the S4 specifically includes: The relationship between parallax and depth is:

[0031] in, is the camera focal length, as the baseline.

[0032] In a possible implementation manner, the S5 specifically includes: Automatically divide the vehicle forward driving area, reverse driving area, forward emergency lane and reverse emergency lane according to dotted lines and solid lines; For the target object to be detected, if its label is a pedestrian, animal, non-motor vehicle, smoke, flame, spilled object or hazardous chemical label, the corresponding event will be directly reported; If the target objects are workers, reflective cones, and construction vehicles, and two or more categories are detected at the same time, an alarm for a construction event will be generated; For all target objects detected as motor vehicles, if the coordinates of the center point of the bounding box of the motor vehicle target are detected to move across the solid line, the motor vehicle will be alarmed to cross the solid line; if the center point of the bounding box of the motor vehicle target is detected to change in the opposite direction, the motor vehicle will be alarmed to drive in the opposite direction; based on the change in the center point of the bounding box of the detected motor vehicle, the speed of the motor vehicle is calculated, and the motor vehicle will be automatically alarmed for speeding, congestion, slow driving and parking according to the driving speed.

[0033] In a possible implementation, the event monitoring method in the tunnel track inspection robot environment is performed in combination with corresponding equipment or systems: The system includes an image acquisition module, a target detection and segmentation module, an inspection motion posture estimation module, an object depth information estimation module, and an event detection module.

[0034] Image acquisition module: A monocular camera installed on the track captures the current road surface in real time. Then, the image is adjusted to , and then normalize it.

[0035] Object detection and segmentation module: The preprocessed image is input into CSPDarknet to obtain the initial feature representation of the image, and then multi-level and multi-scale features are extracted through skip connections and feature pyramid networks. Three detection heads and one segmentation head are constructed based on the features of different scales. The image data is input, and the objects with low confidence and high overlap ratio are eliminated through confidence value and non-maximum suppression, and finally the bounding box, segmentation mask coordinates, category label, feature points, etc. of the effective predicted target are output.

[0036] Inspection motion posture estimation module: After the target detection and segmentation module obtains the target coordinates, category and other information of the image, the segmentation mask points of n target objects with high confidence and corresponding labels of the same category in the two frames of images A and B are taken after the above operation. The segmentation mask points of the upper left, upper right, lower left and lower right of the two frames of data are selected, that is, (x 11 ,y 11 )、(x 12 ,y 12 )、(x 13 ,y 13)、(y 14 ,y 14 ) and (x 21 ,y 21 )、(x 22 ,y 22 )、(x 23 ,y 23 )、(y 24 ,y 24 ), and the center point of the weighted average of all segmentation mask points of the two frames of data before and after, that is, (x 15 ,y 15 ) and (x 25 ,y 25 ).

[0037] For each mask point (x 1i ,y 1i ), find the k points (x) closest to each other in image B 2i ,y 2i ), the Euclidean distance is used as the distance between two feature points, that is:

[0038] i and j are 1 to 5, and Respectively represent the mask points of image A and image B. The validity of the match is judged by the ratio test (i.e., the distance ratio between the nearest neighbor and the second nearest neighbor), and the calculation formula is:

[0039] If the ratio is less than the set threshold , then the point matching is considered valid. Record the matching results of each mask point, including the matched point and the corresponding distance. Finally, combine the IMU to capture the rapid movement and rotation of the camera to further predict the change of the camera's posture. The specific process is as follows: 1. Initialize the pose: Assume that the initial camera position is (usually represented as a 4x4 homogeneous transformation matrix) containing translation and rotation information.

[0040] 2. Get IMU data: Getting time series data from the IMU: linear acceleration and angular velocity .

[0041] 3. Time discretization: Assume that the acceleration and angular velocity are constant in each time period. Divide the time interval into small segments , and integrate within each time period.

[0042] 4. Integral update pose: a. Update rotation: set the angular velocity Convert to rotation matrix R:

[0043] b. Update translation: via acceleration Update the camera's position:

[0044] in is the speed of the camera at the last moment.

[0045] c. Update speed: Update speed:

[0046] 5. Combined pose: Updated pose It can be obtained by combining the current pose and the change:

[0047] 6. Iterative process: Repeat steps 3 to 5 until all IMU data are processed.

[0048] 7. Fusion image matching results: The pose results of image matching are fused with the pose predicted by IMU, and Kalman filtering is usually used to reduce noise and improve the accuracy of pose estimation.

[0049] Object depth information estimation module: The maximum boundary range of the target detected by the target detection and segmentation module is intercepted as the effective image. The encoder-decoder method is used to estimate the depth value of the effective image. The effective image is passed through the ResNet50 encoder to extract image features to obtain deep multi-level features, and then the extracted multi-level features are deconvolved and upsampled to generate a depth map. Calculate the angle and position changes of the two frames of images in the scene - disparity, and the relationship between disparity and depth:

[0050] in f is the camera focal length, B is the baseline.

[0051] Using the depth map generated by the network, the input image is reprojected to other perspectives, and then combined with the camera posture, internal parameters, etc., the depth value of each pixel of the target is output.

[0052] Event detection module: According to the previous modules, we can know the information of the camera and the target object. This module can accurately detect multiple road target objects such as motor vehicles, dotted lines, solid lines, cars, pedestrians, animals, non-motor vehicles, smoke, flames, spills, construction personnel, reflective cones, reflective barrels, construction vehicles, water barriers, and guide signs. First, according to the detection results, the vehicle forward driving area, reverse driving area, forward emergency lane, and reverse emergency lane are automatically divided according to dotted lines, solid lines, etc. For the detected target object, if its label is a pedestrian, animal, non-motor vehicle, smoke, flame, spills, or hazardous chemicals sign label, the corresponding event is directly alarmed. If the target object is a staff member, reflective cone (ice cream cone), or construction vehicle, and two or more categories are detected at the same time, the construction event is alarmed. For all target objects detected as motor vehicles, if the coordinates of the center point of the bounding box of the motor vehicle target are detected to move across the solid line, the motor vehicle is alarmed to cross the solid line; if the center point of the bounding box of the motor vehicle target is detected to change in the opposite direction, the motor vehicle is alarmed to drive in the opposite direction; according to the change in the center point position of the bounding box of the detected motor vehicle, the speed of the motor vehicle can be calculated. If its speed exceeds v max , then the alarm is given for the motor vehicle speeding; if the speed is lower than v low , then the congestion alarm is issued; if the driving speed is between (v low ,v low2 ), an alarm is sounded that the motor vehicle is traveling slowly; if the speed is 0 (the center point position hardly changes), an alarm is sounded that the vehicle is stopping.

[0053] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: In the present invention, the accuracy of event detection is improved by improving the motion posture estimation and object depth estimation of the monocular vision camera. The feature points of the object are obtained through target detection and segmentation, and the camera motion posture is accurately predicted by combining KNNMatch and IMU, which overcomes the shortcomings of the monocular vision system in depth estimation. At the same time, the image encoding-decoding model is used to generate a depth map and reproject it to different perspectives, accurately calculate the three-dimensional coordinates of the object, and realize accurate monitoring of various traffic events in the tunnel.

[0054] The above contents are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0055] There are a few points to note: (1) The drawings of the embodiments of the present invention only involve structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0056] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0057] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0058] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An event monitoring method in a tunnel track inspection robot environment, characterized in that: include: S1. Install a monocular camera on the track and take real-time road images, and adjust the road images to After the size is determined, preprocessing is performed, wherein the preprocessing includes normalization processing; S2. Input the preprocessed image into CSPDarknet to obtain the preliminary feature representation of the image, then extract different scale features through skip connection and feature pyramid network, and then build three detection heads and one segmentation head according to the different scale features. The input image data is subjected to confidence value and non-maximum suppression to eliminate targets with low confidence and high overlap ratio, and finally output the bounding box, segmentation mask coordinates, category label and feature points of the effective predicted target; S3, extracting corner points and weighted center points from the segmentation mask points with high confidence and consistent labels in the two frames A and B, and finding the nearest point in the latter frame for each of the mask points k Points are located in the center of the image, and the Euclidean distance is used to calculate the distance ratio between the nearest neighbor and the next nearest neighbor, and the validity of the match is determined. If the match is valid, the IMU data is combined to use the camera's motion and rotation information to predict the camera's position change. S4, intercept the maximum boundary range of the detected target as the effective image, use the encoder-decoder method to estimate the depth value of the effective image, extract image features from the effective image through the ResNet50 encoder, obtain deep multi-level features, then perform deconvolution and upsampling operations on the extracted multi-level features to generate a depth map, calculate the disparity of the two frames A and B in the scene, and the disparity includes angle and position changes. Then, use the depth map generated by the network to reproject the input image to other perspectives, and output the depth value of each pixel of the target in combination with the camera posture and internal parameters; S5. Detect the target object on the road surface according to the information of the camera and the information of the target object obtained in the previous step, and automatically analyze and issue a corresponding event alarm.

2. The event monitoring method in a tunnel track inspection robot environment according to claim 1 is characterized in that: The S3 specifically includes: For each mask point (x 1i ,y 1i ), find the k points (x) closest to each other in image B 2i ,y 2i ), using Euclidean distance as the distance between two feature points: in, i and j Take 1 to 5, and Represent the mask points of images A and B respectively, and the validity of the match is judged by the ratio test, which is the distance ratio between the nearest neighbor and the second nearest neighbor: If the ratio is less than the set threshold , then the point match is considered valid, and the matching result of each mask point is recorded, including the matched point and the corresponding distance.

3. The event monitoring method in a tunnel track inspection robot environment according to claim 2 is characterized in that: The predicted camera pose change includes: Assume the initial camera position is , including translation and rotation information, and obtaining time series data from the IMU, including current acceleration and angular velocity , assuming that the acceleration and angular velocity in each time period are constant, divide the time interval into small segments , integrate in each time period, and update the rotation, translation, and velocity separately through integration: Update the rotation, angular velocity Convert to rotation matrix R: Update the translation and update the camera position by the acceleration a: in, v is the speed of the camera at the last moment, Update speed: Updated pose By combining the current pose and the change: Repeat the process of dividing the time interval into small segments to obtain the updated pose until all IMU data are processed, and then fuse the pose result of image matching with the pose predicted by IMU.

4. The event monitoring method in a tunnel track inspection robot environment according to claim 3 is characterized in that: include: When fusing the pose results of image matching with the pose predicted by IMU, Kalman filtering is used to reduce noise and improve the accuracy of pose estimation.

5. The event monitoring method in a tunnel track inspection robot environment according to claim 1 is characterized in that: The S4 specifically includes: The relationship between parallax and depth is: in, is the camera focal length, as the baseline.

6. The event monitoring method in a tunnel track inspection robot environment according to claim 1 is characterized in that: The S5 specifically includes: Automatically divide the vehicle forward driving area, reverse driving area, forward emergency lane and reverse emergency lane according to dotted lines and solid lines; For the target object to be detected, if its label is a pedestrian, animal, non-motor vehicle, smoke, flame, spilled object or hazardous chemical label, the corresponding event will be directly reported; If the target objects are workers, reflective cones, and construction vehicles, and two or more categories are detected at the same time, an alarm for a construction event will be generated; For all target objects detected as motor vehicles, if the coordinates of the center point of the bounding box of the motor vehicle target are detected to move across the solid line, the motor vehicle will be alarmed to cross the solid line; if the center point of the bounding box of the motor vehicle target is detected to change in the opposite direction, the motor vehicle will be alarmed to drive in the opposite direction; based on the change in the center point of the bounding box of the detected motor vehicle, the speed of the motor vehicle is calculated, and the motor vehicle will be automatically alarmed for speeding, congestion, slow driving and parking according to the driving speed.

Citation Information

Patent Citations

  • Vehicle illegal parking detection method based on unmanned aerial vehicle routing inspection real-time image

    CN117593673A

  • Traffic jam judgment method and system based on view angle of unmanned aerial vehicle

    CN118038673A

  • An on-board monitoring system for intelligent traffic inspection vehicles

    CN118247761B

  • Environment patrol system based on multi-sensor fusion of unmanned aerial vehicle

    CN118687630A

  • Unmanned aerial vehicle holder adjustment control system and method for traffic inspection

    CN118759833A

Cited By

  • Image acquisition control method and system based on inspection robot

    CN120206503A