Inspection vehicle cooperative positioning method and system based on dynamic blurred image

Abstract

The invention provides an inspection vehicle cooperative positioning method and system based on a dynamic fuzzy image, and relates to the technical field of tunnel inspection, and the method comprises the following steps: S1, carrying out slam modeling through a laser radar; s2, the tethered unmanned aerial vehicle performs target detection on the AprilTags codes arranged in the tunnel at equal intervals through a carried RGB camera, and if a first AprilTags code candidate area is obtained through detection, the step S3 is executed; s3, performing deblurring processing on the first AprilTags code candidate region, performing image definition judgment on the processed first AprilTags code candidate region, and if the image is judged to be unclear, turning to step S4; if the image is judged to be clear, turning to step S5; s4, a control instruction is sent to the free unmanned aerial vehicle according to the first AprilTags code candidate area, a second AprilTags code candidate area is obtained, and the step S5 is executed; and S5, positioning according to the processed first AprilTags code candidate area or the processed second AprilTags code candidate area to obtain the position information of the inspection vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel inspection, and in particular to a collaborative positioning method and system for an inspection vehicle based on dynamic fuzzy images. Background Art

[0002] Traditional tunnel inspection methods rely primarily on manual labor, which is limited by low efficiency, high cost, and high risk. Therefore, intelligent, mechanized tunnel inspection methods have gradually become a research hotspot and an important direction for practical application. Drones and unmanned vehicles have their own advantages in tunnel inspection. Air-ground collaborative systems can leverage their respective strengths while compensating for their respective shortcomings, offering promising application prospects.

[0003] To achieve efficient air-ground collaboration, the first issue that needs to be addressed is the environmental perception, positioning, and navigation of drone and unmanned vehicle systems. Among existing tunnel positioning technologies, radio frequency identification (RFID) has a short range and poor anti-interference capabilities. Wi-Fi positioning technology, used indoors, only achieves an accuracy of approximately 2 meters, failing to achieve precise positioning. Bluetooth systems suffer from poor stability and are easily affected in complex environments. UWB requires at least three base stations, and signal propagation is blocked by obstacles such as walls, ceilings, doors, and people, resulting in reflection, refraction, and diffraction. Transmitted signals arrive at the UWB receiver at varying times and paths, ultimately reducing UWB accuracy and requiring complex filtering and optimization algorithms. Traditional Slam algorithms suffer from positioning accuracy at high speeds and in the absence of closed-loop detection. The resulting cumulative errors cause the Slam system to degenerate into a simple odometer, with the generated maps and trajectories gradually deviating from reality. Summary of the Invention

[0004] The purpose of the present invention is to provide a collaborative positioning method and system for inspection vehicles based on dynamic fuzzy images, which is applied to tunnel inspection scenarios. Several AprilTags codes are arranged at equal intervals inside the tunnel. Based on AprilTags code image recognition, precise positioning is achieved through the collaboration between the inspection vehicle, tethered drone and free drone, thereby improving the positioning accuracy of the inspection vehicle.

[0005] In order to solve the above technical problems, the present invention adopts the following solutions:

[0006] A collaborative positioning method for an inspection vehicle based on dynamic blur images is disclosed. The inspection vehicle includes a vehicle platform, a laser radar mounted on the vehicle platform, a tethered drone, and a free drone, each equipped with an RGB camera. The inspection vehicle conducts inspections inside a tunnel, where a plurality of AprilTags are evenly spaced. The collaborative positioning method includes the following steps:

[0007] S1. When the inspection vehicle is inspecting inside the tunnel, SLAM modeling is performed using LiDAR.

[0008] S2. When the inspection vehicle pulls the tethered drone through the power cable for synchronous movement, the tethered drone uses the RGB camera on board to detect the AprilTags codes set at equal intervals inside the tunnel. If the first AprilTags code candidate area is detected, the process proceeds to step S3.

[0009] S3, deblurring the first AprilTags candidate area, and determining the image clarity of the processed first AprilTags candidate area. If the image is not clear, proceed to step S4; if the image is clear, proceed to step S5;

[0010] S4. Send a control command to the free drone according to the first AprilTags code candidate area, so that the free drone receives the control command and flies to the AprilTags code corresponding to the first AprilTags code candidate area, performs target detection on it, obtains the second AprilTags code candidate area, and then goes to step S5;

[0011] S5. Calculate the position information of the tethered UAV based on the processed first AprilTags code candidate area or the second AprilTags code candidate area, and then perform relative position conversion and positioning based on the position information of the tethered UAV to obtain the position information of the inspection vehicle.

[0012] Furthermore, the tethered drone is equipped with an inertial sensor and a drone laser odometer, and the tethered drone can obtain the current position information of the tethered drone in real time through the drone laser odometer.

[0013] Furthermore, the S1 includes the following steps:

[0014] S11. When the inspection vehicle is inspecting inside the tunnel, it obtains laser scanning data inside the tunnel through the laser radar and pre-processes the laser scanning data;

[0015] S12, extracting key feature points from the preprocessed laser scanning data, and describing each key feature point to generate a feature sub-point that can characterize its attributes and position;

[0016] S13, estimating the position information of the inspection vehicle through the inertial sensor to obtain its motion model;

[0017] S14, establishing an extended Kalman filter based on the laser scanning data and the motion model of the inspection vehicle to locate the inspection vehicle;

[0018] S15. Combine the laser scanning data and the inspection vehicle's posture information through feature point matching to construct a three-dimensional point cloud.

[0019] Furthermore, in S2, the target detection process is:

[0020] The YOLO target detection algorithm is used to quickly locate and mark the region of interest in the current image captured by the RGB camera to obtain the first AprilTags code candidate area.

[0021] Furthermore, the step S3 includes the following steps:

[0022] S31, capturing motion information of an RGB camera carried by the tethered UAV through an inertial sensor, and deriving a diffusion function based on the motion information to perform preliminary processing on the first AprilTags code candidate area;

[0023] S32, generating an adversarial network consisting of a generator and a discriminant network, receiving the first AprilTags code candidate region after preliminary processing through the generator, extracting its semantic features for processing to obtain the processed first AprilTags code candidate region;

[0024] S33. The discriminator judges the processed first AprilTags candidate area, and performs image clarity judgment on the AprilTags candidate area according to the judgment result. If the image is judged to be unclear, go to step S4; if the image is judged to be clear, go to step S5.

[0025] Furthermore, in S33, the discriminator judges the processed first AprilTags candidate area, and performs image definition judgment on the AprilTags candidate area according to the judgment result. The specific process is as follows:

[0026] The second-order derivative of the first AprilTags code candidate area after processing is calculated by the Laplace operator, and its image clarity evaluation score is reflected by the second-order derivative. When the image clarity evaluation score exceeds the threshold, the image is judged to be clear; when the image clarity evaluation score does not exceed the threshold, the image is judged to be unclear.

[0027] Furthermore, the control instruction includes the global coordinate information of the AprilTags code corresponding to the first AprilTags code candidate area, so that when the free drone receives the global coordinate information, it flies to the corresponding AprilTags code, performs static shooting, and obtains the second AprilTags code candidate area through target detection.

[0028] Furthermore, the S5 includes the following steps:

[0029] S51. Obtaining the position information of the current tethered UAV when detecting the first AprilTags code candidate area through the UAV laser odometry;

[0030] S52, calculating and obtaining the position information of the tethered UAV according to the processed first AprilTags code candidate area or the second AprilTags code candidate area;

[0031] S53, correcting the current position information of the tethered drone using the position information of the tethered drone to obtain corrected position information of the tethered drone;

[0032] S54, performing relative posture conversion positioning on the corrected position information of the tethered UAV to obtain the position information of the inspection vehicle.

[0033] A patrol vehicle collaborative positioning system based on dynamic fuzzy images, applying the patrol vehicle collaborative positioning method based on dynamic fuzzy images, comprises:

[0034] Slam modeling module: When the inspection vehicle conducts inspections inside the tunnel, Slam modeling is performed using lidar;

[0035] Tethered UAV target detection module: When the inspection vehicle pulls the tethered UAV through the power cable for synchronous movement, the tethered UAV uses the RGB camera on board to detect AprilTags codes set at equal intervals inside the tunnel. If the first AprilTags code candidate area is detected;

[0036] Image blur processing module: performs deblurring processing on the first AprilTags code candidate area and determines the image clarity of the processed first AprilTags code candidate area;

[0037] Control command module: sends a control command to the free drone according to the first AprilTags code candidate area, so that the free drone receives the control command and flies to the AprilTags code corresponding to the first AprilTags code candidate area, performs target detection on it, and obtains the second AprilTags code candidate area;

[0038] Positioning module: The position information of the tethered UAV is obtained based on the processed first AprilTags code candidate area or the second AprilTags code candidate area, and then the position information of the inspection vehicle is obtained by relative position conversion and positioning based on the position information of the tethered UAV.

[0039] Beneficial effects of the present invention:

[0040] The present invention provides a collaborative positioning method and system for inspection vehicles based on dynamic blur images. The collaborative positioning method for inspection vehicles is applied to tunnel inspection scenarios. Several AprilTags codes are arranged at equal intervals inside the tunnel, and the tethered drone and the free drone are both equipped with RGB cameras. The AprilTags codes are recognized by the RGB cameras to obtain positioning information of the AprilTags codes. The drone laser odometer is calibrated using the positioning information to reduce cumulative errors, thereby achieving collaborative positioning among the inspection vehicle, the tethered drone, and the free drone.

[0041] In addition, by increasing the coordination between tethered drones and free drones and processing the dynamically blurred AprilTags code images multiple times, clearer images can be obtained, and the positioning information of the AprilTags code can be parsed more accurately, thereby improving the positioning accuracy of the inspection vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the process of the inspection vehicle collaborative positioning method in Example 1 of the present invention;

[0043] Figure 2 Schematic diagram of the process of eliminating dynamic blur of an image in Example 1 of the present invention;

[0044] Figure 3 Schematic diagram of the deblurring principle of the generative adversarial network in Example 1 of the present invention;

[0045] Figure 4 Schematic diagram of the AprilTag algorithm in Example 1 of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] Unless otherwise specifically stated, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention.

[0048] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0049] Additionally, descriptions of well-known structures, functions, and configurations may be omitted for clarity and conciseness. Those skilled in the art will recognize that various changes and modifications can be made to the examples described herein without departing from the spirit and scope of the present disclosure.

[0050] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0051] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0052] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments:

[0053] Example 1

[0054] In this embodiment, to address the inaccuracies of air-ground collaboration in the prior art, a patrol vehicle collaborative positioning method based on dynamic blur images is proposed. This patrol vehicle collaborative positioning method is primarily applied in tunnel scenarios, where a number of AprilTags are evenly spaced within the tunnel. During tunnel inspections, the patrol vehicle captures the AprilTags via a camera and uses image recognition to obtain positioning information. However, because the patrol vehicle is in motion, the AprilTags captured by the camera will be dynamically blurred, resulting in the majority of images captured by the patrol vehicle during the inspection process being dynamically blurred. Image recognition of dynamically blurred images often results in inaccurate recognition, leading to relatively inaccurate positioning information obtained through AprilTags image recognition in the prior art.

[0055] Therefore, the present invention focuses on the collaborative positioning of inspection vehicles based on dynamic blurred images. Specifically, the inspection vehicle includes a trolley platform, a laser radar arranged on the trolley platform, a tethered drone and a free drone. The tethered drone and the free drone are both equipped with RGB cameras. The RGB camera can not only be used to confirm the relative posture relationship with the inspection vehicle, but also can be used to visually identify the AprilTags code; and the tethered drone is equipped with an inertial sensor and a drone laser odometer. The tethered drone can obtain the current position information of the tethered drone in real time through the drone laser odometer.

[0056] like Figure 1 As shown, the inspection vehicle collaborative positioning method includes the following steps:

[0057] S1. When the inspection vehicle is inspecting inside the tunnel, SLAM modeling is performed using LiDAR.

[0058] S2. When the inspection vehicle pulls the tethered drone through the power cable for synchronous movement, the tethered drone uses the RGB camera on board to detect the AprilTags codes set at equal intervals inside the tunnel. If the first AprilTags code candidate area is detected, the process proceeds to step S3. Conversely, if the first AprilTags code candidate area is not detected, the position information of the inspection vehicle can be directly calculated based on the relative pose.

[0059] S3, deblurring the first AprilTags candidate area, and determining the image clarity of the processed first AprilTags candidate area. If the image is not clear, proceed to step S4; if the image is clear, proceed to step S5;

[0060] S4. Send a control command to the free drone according to the first AprilTags code candidate area, so that the free drone receives the control command and flies to the AprilTags code corresponding to the first AprilTags code candidate area, performs target detection on it, obtains the second AprilTags code candidate area, and then goes to step S5;

[0061] S5. Calculate the position information of the tethered UAV based on the processed first AprilTags code candidate area or the second AprilTags code candidate area, and then perform relative position conversion and positioning based on the position information of the tethered UAV to obtain the position information of the inspection vehicle.

[0062] In one embodiment, the step S1 includes the following steps:

[0063] S11. When the inspection vehicle is inspecting inside the tunnel, it obtains laser scanning data inside the tunnel through the laser radar and preprocesses the laser scanning data; wherein the preprocessing includes denoising and completion, etc., which can be used to improve the quality of the data;

[0064] S12, extracting key feature points from the preprocessed laser scanning data, and describing each key feature point to generate a feature sub-point that can characterize its attributes and position;

[0065] S13, estimating the position information of the inspection vehicle through the inertial sensor to obtain its motion model;

[0066] S14, establishing an extended Kalman filter based on the laser scanning data and the motion model of the inspection vehicle to locate the inspection vehicle;

[0067] S15. Combine the laser scanning data and the inspection vehicle's posture information through feature point matching to construct a three-dimensional point cloud to achieve SLAM modeling.

[0068] In one embodiment, in S2, the target detection process is:

[0069] The YOLO target detection algorithm is used to quickly locate and annotate the region of interest in the current image captured by the RGB camera to obtain the first AprilTags code candidate area, enabling preliminary screening and positioning of the AprilTags code in complex scenarios. The YOLO target detection algorithm is also used to efficiently extract the candidate area of ​​the AprilTags code from the image. This method combines the advantages of the YOLO algorithm's real-time detection and can quickly generate candidate area frames in a multi-interference environment, providing a reliable foundation for subsequent accurate recognition and decoding of the AprilTags code.

[0070] In one embodiment, S3 includes the following steps:

[0071] S31, capturing motion information of an RGB camera carried by the tethered UAV through an inertial sensor, and deriving a diffusion function based on the motion information to perform preliminary processing on the first AprilTags code candidate area;

[0072] S32, generating an adversarial network consisting of a generator and a discriminant network, receiving the first AprilTags code candidate region after preliminary processing through the generator, extracting its semantic features for processing to obtain the processed first AprilTags code candidate region;

[0073] S33. The discriminator judges the processed first AprilTags candidate area, and performs image clarity judgment on the AprilTags candidate area according to the judgment result. If the image is judged to be unclear, go to step S4; if the image is judged to be clear, go to step S5.

[0074] like Figure 2 As shown in the figure, by removing the dynamic blur of the AprilTags image, a clearer AprilTags image can be obtained, and then the AprilTags are decoded to obtain more accurate positioning information, thereby realizing accurate collaborative positioning of the inspection vehicle.

[0075] Specifically, in S31, the total variation regularization deblurring method can be implemented with the assistance of inertial sensors. This method first uses inertial sensors to capture the motion information of the onboard camera, and then derives the propagation function (PSF) based on this information, thereby effectively avoiding the problems that traditional algorithms encounter when interfered by factors such as complex textures, low contrast or noise. The estimated PSF is then used in combination with the total variation regularization technology to restore the image; and the split Bregman iteration technology is introduced to decompose the complex optimization problem into a series of simple sub-problems, thereby speeding up the calculation speed and achieving high-precision image deblurring.

[0076] The specific process of this method is as follows:

[0077] Project the world point (X, Y, Z) to K different positions (xk, yk) through the homography matrix Hk, sample K different postures through the inertial measurement sensor, and obtain the motion trajectory;

[0078] The homography matrix can be expressed as:

[0079] Among them, K is the intrinsic parameter matrix of the camera, Rk, tk rotation matrix and translation vector, d is the distance from the object plane to the camera frame, and n is the normal vector of the object plane relative to the camera frame;

[0080] The image PSF function is expressed as:

[0081] Where K is the number of diffusion points sampled by inertial measurement, xk and yk are the coordinates of the diffusion points, and h(m,n) represents the diffusion intensity at position (m,n).

[0082] The image blurring process can be described by the model:

[0083] Where u0 is the column vector obtained by arranging the observed blurred image in column order, u is the column vector obtained by arranging the original clear image in column order, H is the blur operator, which is a circulant matrix constructed based on the PSF, and w is the column vector obtained by arranging the noise matrix in column order.

[0084] The total variation regularized restoration algorithm transforms the image restoration problem into an unconstrained optimization problem, namely Among them, the first term is the fidelity term, which ensures that the distance between the original clear image and the observed image after blurring is small enough; the second term is the regularization term; Diu represents the map of the image in the horizontal and vertical directions; u is the regularization term coefficient, which is used to balance the fidelity term and the regularization term;

[0085] In order to quickly solve this optimization problem and reduce the running time of the TV regularization algorithm, the split Bregman algorithm is used. This algorithm decomposes the complex optimization problem into multiple sub-problems by splitting the operator, and then applies the Bregman iteration method to solve these sub-problems, thereby improving the efficiency of the algorithm.

[0086] By introducing the auxiliary vector d, the above minimization problem can be expressed as follows after the operator splitting: Among them, di and bi are variables related to the Bergman iterative algorithm;

[0087] Solving the above formula can be transformed into multiple iterations to solve the following two sub-problems:

[0088] (1) Fix u and solve the minimum optimization problem about di, that is,

[0089] According to the two-dimensional contraction theorem, the closed-form solution of this equation is:

[0090]

[0091] (2) Fix di and solve the minimum optimization problem about u, that is,

[0092] The solution is: Where D is the difference operator matrix;

[0093] (3) Update bik+1=dik-Diuk;

[0094] (4) Determine ||u k+1 -u k ||2>σ, if it holds, return to step 1, otherwise exit the loop.

[0095] Specifically, in S32, the dynamic blurred image is further deblurred by generating an adversarial network, such as Figure 3As shown in the figure, the GAN further processes the image obtained in the previous step. The GAN consists of a generator G and a discriminator D. The generator receives a blurred image and extracts its semantic features to generate a clear image sample to deceive the discriminator. The discriminator's task is to distinguish whether the input image is a real image or a fake clear image generated by the generator. The loss obtained by the discriminator is the adversarial loss, which is used to directly train the discriminator to improve its discrimination ability. In addition, the discriminator's loss is also used to calculate the generator's loss, thereby indirectly training the generator. After multiple training iterations, the data generated by the generator will become increasingly similar to real images, and the discriminator's discriminative ability will also increase. In theory, the generator and discriminator will eventually reach a Nash equilibrium, at which point the images generated by the generator are indistinguishable from real clear images and cannot be easily distinguished by the discriminator.

[0096] Moreover, since the tethered UAV is connected to the trolley platform via a power supply cable, when the inspection vehicle moves, the tethered UAV is driven to move with the inspection vehicle. The setting of the tethered UAV can meet the power supply needs of the inspection vehicle for the UAV during the inspection process, thereby realizing inspection at all times. However, under the connection mode of the power supply cable, the tethered UAV can only fly not far from the inspection vehicle, and the clarity of the image captured by the RGB camera carried by the tethered UAV is limited, and during the movement, it is easy to cause dynamic blur. Therefore, the image captured by the RGB camera carried by the tethered UAV is more likely to have inaccurate image recognition. The present invention also carries a free UAV on the trolley platform. The free UAV is a slave and the tethered UAV is a host. The free UAV is usually a small UAV located in the UAV hangar, which can be started only when the image collected by the tethered UAV is relatively inaccurate.

[0097] Based on the above principles, the present invention re-judges the discrimination result of the first AprilTags code candidate area after processing by the discriminator in S33, and performs image clarity judgment on the AprilTags code candidate area according to the judgment result, with the aim of ensuring the clarity of the AprilTags code image used for positioning and improving the positioning accuracy of the inspection vehicle. The present invention focuses on increasing the collaboration between the tethered UAV and the free UAV based on the AprilTags code image after dynamic blur image processing, and realizes multiple processing of the dynamically blurred AprilTags code image, in order to avoid the image that can be captured by the RGB camera carried by the tethered UAV being relatively unclear after dynamic blur processing, resulting in the positioning information obtained after decoding the image still being inaccurate.

[0098] In one embodiment, in S33, the discriminator judges the processed first AprilTags candidate area, and performs image clarity judgment on the AprilTags candidate area according to the judgment result. The specific process is as follows:

[0099] The second-order derivative of the first AprilTags code candidate area after processing is calculated by the Laplace operator, and its image clarity evaluation score is reflected by the second-order derivative. When the image clarity evaluation score exceeds the threshold, the image is judged to be clear; when the image clarity evaluation score does not exceed the threshold, the image is judged to be unclear.

[0100] Specifically, since image clarity is usually related to the sharpness of edges and details, the gradient can reflect the degree of change in pixel values ​​in the image. Therefore, the present invention uses the Laplace operator to calculate the second-order derivative of the image to reflect the edge and detail information of the image. Among them, the edges and details of a clear image are usually sharper, and the response of the second-order derivative is stronger; while the edges and details of a blurred image are relatively smooth, and the response of the second-order derivative is weaker. The image clarity evaluation score is:

[0101]

[0102] in, Represents the Laplace response (second-order derivative) of image I at position (x, y)

[0103]

[0104] In one embodiment, the control instruction includes the global coordinate information of the AprilTags code corresponding to the first AprilTags code candidate region. Upon receiving this global coordinate information, the free drone flies to the corresponding AprilTags code, performs static photography, and obtains the second AprilTags code candidate region through target detection. Since, after SLAM modeling, the inspection vehicle, tethered drone, and free drone share a coordinate system, and the spacing between AprilTags codes is significantly greater than the positioning error, once the global coordinate information is obtained, the free drone can fly out of the drone hangar and directly accurately locate the location of the AprilTags code using the global coordinate information. Furthermore, static observation of the AprilTags code can be achieved, resolving the issue of inaccurate positioning information obtained after decoding due to failure of the generative adversarial network or unclear first AprilTags code candidate region after deblurring.

[0105] In one embodiment, the step S5 includes the following steps:

[0106] S51. Obtaining the position information of the current tethered UAV when detecting the first AprilTags code candidate area through the UAV laser odometry;

[0107] S52, calculating and obtaining the position information of the tethered UAV according to the processed first AprilTags code candidate area or the second AprilTags code candidate area;

[0108] S53, correcting the current position information of the tethered drone using the position information of the tethered drone to obtain corrected position information of the tethered drone;

[0109] S54, performing relative posture conversion positioning on the corrected position information of the tethered UAV to obtain the position information of the inspection vehicle.

[0110] Specifically, in S52, the AprilTags code is used to calculate the posture information of the tethered UAV, such as Figure 4 As shown in the figure, the AprilTag algorithm consists of twelve processing steps, among which Steps 1-5: filter and denoise the image, calculate the gradient of the pixels to cluster and extract the edges, fit the edge lines, and add vectors from dark areas to bright areas to the edge lines; Steps 6-9: connect the edge lines to obtain quad loops, identify the quad loops and decode them, and recognize the QR code ID and rotation angle; Steps 10-12: obtain camera parameters, construct the PnP equation of the pose data, and solve it to obtain the pose information of the tethered drone.

[0111] Specifically, in S53, the tethered drone's position information is used to correct the current tethered drone's position information. Since the Kalman filter is an optimal estimation method, it establishes a dynamic model and an observation model for the system. The dynamic model describes the evolution of the system state, while the observation model describes the relationship between sensor measurements and the system state. Therefore, in the present invention, the QR code information is treated as the observation value, and the Kalman filter is used to fuse the precise position information provided by the QR code with the laser odometry data to correct the accumulated error of the drone's laser odometry.

[0112] Specifically, in S54, the position information of the inspection vehicle is obtained through relative posture conversion. The relative posture sensor can obtain the relative posture of the vehicle relative to the UAV. Through posture conversion, the absolute position of the unmanned vehicle in the UAV coordinate system can be obtained, thereby achieving positioning.

[0113] In summary, the present invention provides a collaborative positioning method and system for inspection vehicles based on dynamic blur images. The collaborative positioning method for inspection vehicles is applied to tunnel inspection scenarios. Several AprilTags codes are arranged at equal intervals inside the tunnel, and the tethered UAV and the free UAV are equipped with RGB cameras. The AprilTags codes are recognized by the RGB cameras to obtain the positioning information of the AprilTags codes. The UAV laser odometer is calibrated using the positioning information to reduce the cumulative error, thereby realizing collaborative positioning among the inspection vehicle, the tethered UAV and the free UAV.

[0114] In addition, by increasing the coordination between tethered drones and free drones and processing the dynamically blurred AprilTags code images multiple times, clearer images can be obtained, and the positioning information of the AprilTags code can be parsed more accurately, thereby improving the positioning accuracy of the inspection vehicle.

[0115] Example 2

[0116] A patrol vehicle collaborative positioning system based on dynamic fuzzy images, applying the patrol vehicle collaborative positioning method based on dynamic fuzzy images, comprises:

[0117] Slam modeling module: When the inspection vehicle conducts inspections inside the tunnel, Slam modeling is performed using lidar;

[0118] Tethered UAV target detection module: When the inspection vehicle pulls the tethered UAV through the power cable for synchronous movement, the tethered UAV uses the RGB camera on board to detect AprilTags codes set at equal intervals inside the tunnel. If the first AprilTags code candidate area is detected;

[0119] Image blur processing module: performs deblurring processing on the first AprilTags code candidate area and determines the image clarity of the processed first AprilTags code candidate area;

[0120] Control command module: sends a control command to the free drone according to the first AprilTags code candidate area, so that the free drone receives the control command and flies to the AprilTags code corresponding to the first AprilTags code candidate area, performs target detection on it, and obtains the second AprilTags code candidate area;

[0121] Positioning module: The position information of the tethered UAV is obtained based on the processed first AprilTags code candidate area or the second AprilTags code candidate area, and then the position information of the inspection vehicle is obtained by relative position conversion and positioning based on the position information of the tethered UAV.

[0122] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Based on the technical essence of the present invention and within the spirit and principles of the present invention, any simple modification, equivalent replacement and improvement of the above embodiment shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A collaborative positioning method for inspection vehicles based on dynamic fuzzy images, characterized in that: The inspection vehicle includes a trolley platform, a laser radar installed on the trolley platform, a tethered drone and a free drone. The tethered drone and the free drone are both equipped with RGB cameras. The inspection vehicle conducts inspections inside the tunnel, where several AprilTags are evenly spaced. The inspection vehicle collaborative positioning method comprises the following steps: S1. When the inspection vehicle is inspecting inside the tunnel, SLAM modeling is performed using LiDAR. S2. When the inspection vehicle pulls the tethered drone through the power cable for synchronous movement, the tethered drone uses the RGB camera on board to detect the AprilTags codes set at equal intervals inside the tunnel. If the first AprilTags code candidate area is detected, the process proceeds to step S3. S3, deblurring the first AprilTags candidate area, and determining the image clarity of the processed first AprilTags candidate area. If the image is not clear, proceed to step S4; if the image is clear, proceed to step S5; S4. Send a control command to the free drone according to the first AprilTags code candidate area, so that the free drone receives the control command and flies to the AprilTags code corresponding to the first AprilTags code candidate area, performs target detection on it, obtains the second AprilTags code candidate area, and then goes to step S5; S5. Calculate the position information of the tethered UAV based on the processed first AprilTags code candidate area or the second AprilTags code candidate area, and then perform relative position conversion and positioning based on the position information of the tethered UAV to obtain the position information of the inspection vehicle.

2. The inspection vehicle collaborative positioning method based on dynamic fuzzy images according to claim 1 is characterized in that: The tethered drone is equipped with an inertial sensor and a drone laser odometer, and the tethered drone can obtain the current position information of the tethered drone in real time through the drone laser odometer.

3. The inspection vehicle collaborative positioning method based on dynamic fuzzy images according to claim 2 is characterized in that: The S1 includes the following steps: S11. When the inspection vehicle is inspecting inside the tunnel, it obtains laser scanning data inside the tunnel through the laser radar and pre-processes the laser scanning data; S12, extracting key feature points from the preprocessed laser scanning data, and describing each key feature point to generate a feature sub-point that can characterize its attributes and position; S13, estimating the position information of the inspection vehicle through the inertial sensor to obtain its motion model; S14, establishing an extended Kalman filter based on the laser scanning data and the motion model of the inspection vehicle to locate the inspection vehicle; S15. Combine the laser scanning data and the inspection vehicle's posture information through feature point matching to construct a three-dimensional point cloud.

4. The inspection vehicle collaborative positioning method based on dynamic fuzzy images according to claim 2 is characterized in that: In S2, the target detection process is: The YOLO target detection algorithm is used to quickly locate and mark the region of interest in the current image captured by the RGB camera to obtain the first AprilTags code candidate area.

5. The inspection vehicle collaborative positioning method based on dynamic fuzzy images according to claim 2 is characterized in that: The S3 includes the following steps: S31, capturing motion information of an RGB camera carried by the tethered UAV through an inertial sensor, and deriving a diffusion function based on the motion information to perform preliminary processing on the first AprilTags code candidate area; S32, generating an adversarial network consisting of a generator and a discriminant network, receiving the first AprilTags code candidate region after preliminary processing through the generator, extracting its semantic features for processing to obtain the processed first AprilTags code candidate region; S33. The discriminator judges the processed first AprilTags candidate area, and performs image clarity judgment on the AprilTags candidate area according to the judgment result. If the image is judged to be unclear, go to step S4; if the image is judged to be clear, go to step S5.

6. The inspection vehicle collaborative positioning method based on dynamic fuzzy images according to claim 5 is characterized in that: In S33, the discriminator judges the processed first AprilTags candidate area, and performs image definition judgment on the AprilTags candidate area according to the judgment result. The specific process is as follows: The second-order derivative of the first AprilTags code candidate area after processing is calculated by the Laplace operator, and its image clarity evaluation score is reflected by the second-order derivative. When the image clarity evaluation score exceeds the threshold, the image is judged to be clear; when the image clarity evaluation score does not exceed the threshold, the image is judged to be unclear.

7. The inspection vehicle collaborative positioning method based on dynamic fuzzy images according to claim 1 is characterized in that: The control instruction includes the global coordinate information of the AprilTags code corresponding to the first AprilTags code candidate area, so that when the free drone receives the global coordinate information, it flies to the corresponding AprilTags code, performs static shooting, and obtains the second AprilTags code candidate area through target detection.

8. The inspection vehicle collaborative positioning method based on dynamic fuzzy images according to claim 2 is characterized in that: The S5 includes the following steps: S51. Obtaining the position information of the current tethered UAV when detecting the first AprilTags code candidate area through the UAV laser odometry; S52, calculating and obtaining the position information of the tethered UAV according to the processed first AprilTags code candidate area or the second AprilTags code candidate area; S53, correcting the current position information of the tethered drone using the position information of the tethered drone to obtain corrected position information of the tethered drone; S54, performing relative posture conversion positioning on the corrected position information of the tethered UAV to obtain the position information of the inspection vehicle.

9. A patrol vehicle collaborative positioning system based on dynamic fuzzy images, characterized in that: The method for collaborative positioning of an inspection vehicle based on dynamic fuzzy images as described in any one of claims 1 to 8 comprises: Slam modeling module: When the inspection vehicle conducts inspections inside the tunnel, Slam modeling is performed using lidar; Tethered UAV target detection module: When the inspection vehicle pulls the tethered UAV through the power cable for synchronous movement, the tethered UAV uses the RGB camera on board to detect AprilTags codes set at equal intervals inside the tunnel. If the first AprilTags code candidate area is detected; Image blur processing module: performs deblurring processing on the first AprilTags code candidate area and determines the image clarity of the processed first AprilTags code candidate area; Control command module: sends a control command to the free drone according to the first AprilTags code candidate area, so that the free drone receives the control command and flies to the AprilTags code corresponding to the first AprilTags code candidate area, performs target detection on it, and obtains the second AprilTags code candidate area; Positioning module: The position information of the tethered UAV is obtained based on the processed first AprilTags code candidate area or the second AprilTags code candidate area, and then the position information of the inspection vehicle is obtained by relative position conversion and positioning based on the position information of the tethered UAV.

Images