Unmanned aerial vehicle adaptive inspection and data processing method and device for closed scene

Abstract

The application provides a UAV adaptive inspection and data processing method and device for a closed scene, relates to data processing technology, controls a ground cooperative device to perform inspection on a closed area, and determines an abnormal area; the ground cooperative device has a positioning screen for controlling an air cooperative device; a positioning cursor in the positioning screen is customized to move according to the area form of the abnormal area and the current position of the air cooperative device; the air cooperative device is controlled to move to the abnormal area based on the positioning cursor to perform collection, and an abnormal image of the abnormal area is sent to a management end; the actual situation under the closed scene can be utilized by visual guidance, the communication environment requirement is reduced while normal inspection is ensured, and therefore the reliability of inspection under the closed scene is improved.

Description

Technical Field

[0001] This invention relates to data processing technology, and more particularly to a method and apparatus for adaptive inspection and data processing of unmanned aerial vehicles (UAVs) in closed environments. Background Technology

[0002] Currently, with the rapid development of urban infrastructure construction, the maintenance needs of enclosed areas such as tunnels, culverts, and underground utility tunnels are increasing. The stability of their internal structures is directly related to traffic operation and urban safety. Therefore, drones are frequently used to conduct regular inspections of enclosed areas. Furthermore, drones are generally equipped with high-definition cameras. Cameras, infrared sensors, and other data acquisition devices capture images of the interior surfaces of enclosed spaces and transmit the data back. The collected abnormal images are then sent to management personnel for analysis, thereby promptly identifying potential hazards such as cracks and water seepage.

[0003] In existing technologies, wireless communication links or global satellite navigation are typically used to control the positioning and movement of drones in enclosed spaces. However, there are obvious drawbacks. Enclosed environments such as tunnels and culverts have natural signal shielding characteristics. Due to the obstruction of thick concrete and metal structures, wireless signals are prone to severe attenuation or even interruption. In order to maintain control, existing technologies often require the construction of temporary communication base stations. Even after construction, it is impossible to guarantee that the signal will remain stable throughout the entire enclosed section. Drones are prone to losing contact or colliding, which brings serious safety hazards.

[0004] Therefore, how to utilize visual guidance based on the actual conditions in closed scenarios, while ensuring normal inspections and reducing the requirements for the communication environment, thereby improving the reliability of inspections in closed scenarios, has become an urgent problem to be solved. Summary of the Invention

[0005] This invention provides an adaptive inspection and data processing method and apparatus for unmanned aerial vehicles (UAVs) in closed environments. It can use visual guidance based on the actual conditions in closed environments to reduce the requirements of the communication environment while ensuring normal inspection, thereby improving the reliability of inspection in closed environments.

[0006] A first aspect of the present invention provides an adaptive inspection and data processing method for unmanned aerial vehicles (UAVs) in closed environments, comprising: The ground-based collaborative equipment is used to inspect closed areas and identify abnormal areas. The ground-based collaborative equipment has a positioning screen that controls the aerial collaborative equipment. Based on the regional shape of the abnormal area and the current position of the aerial collaborative device, the positioning cursor in the positioning screen is moved in a customized manner; Based on the positioning cursor, the aerial collaborative device is controlled to move to the abnormal area to collect data, and the abnormal image of the abnormal area is sent to the management terminal.

[0007] Optionally, in one possible implementation of the first aspect, the customized movement of the positioning cursor on the positioning screen based on the regional shape of the abnormal area and the current position of the aerial collaborative device includes: The positioning cursor in the positioning screen corresponding to the current position of the air-cooperative device is obtained as the initial cursor, and the center point of the initial cursor is taken as the initial point. When the region of an anomaly is determined to be blocky, the corresponding anomaly region is treated as a blocky region. Based on the block-shaped region and the initial position, the positioning cursor in the positioning screen is moved in a customized manner; When the abnormal region is determined to be strip-shaped, the corresponding abnormal region is treated as a strip-shaped region. Based on the strip-shaped area and the initial position, the positioning cursor in the positioning screen is moved in a customized manner.

[0008] Optionally, in one possible implementation of the first aspect, the customized movement of the positioning cursor in the positioning screen based on the block region and the initial position includes: The blocky region is fitted to obtain a blocky fitting area, and the blocky fitting area is laid based on the preset acquisition area of ​​the aerial collaborative device to determine the first control point of the blocky laying area. The first control points are sorted according to the laying sequence of the block laying area to obtain the first sequence; Add the initial point to the front of the first sequence to obtain the second sequence. Connect the adjacent points in the second sequence in turn to obtain the movement path of the positioning cursor. The cursor is moved based on the movement path control.

[0009] Optionally, in one possible implementation of the first aspect, the step of fitting the blocky region to obtain a blocky fitting area, and laying the blocky fitting area based on a preset acquisition area of ​​the aerial collaborative device, and determining the first control point of the blocky laying area, includes: The blocky regions are processed by coordinate transformation, and a blocky fitting region is constructed based on the regional coordinate extreme values ​​of the blocky regions. The block fitting area is laid out based on the preset collection area of ​​the aerial collaborative equipment to obtain multiple block laying areas, and the center point of the block laying area is taken as the first point. At a preset collection distance from the first point, a first collection point is determined, and the first collection point is converted to the first control point of the positioning cursor in the positioning screen.

[0010] Optionally, in one possible implementation of the first aspect, the customized movement of the positioning cursor in the positioning screen based on the strip-shaped region and the initial position includes: Multiple strip-shaped fitting areas are obtained by fitting the strip-shaped region, and the strip-shaped fitting areas are laid out based on the preset acquisition area of ​​the aerial collaborative device to determine the second control point of the strip-shaped laying area; The second control points are sorted according to the laying sequence of the strip laying area to obtain the third sequence; The initial point is added before the third sequence to obtain the fourth sequence. Adjacent points in the fourth sequence are connected in sequence to obtain the movement path of the positioning cursor. The cursor is moved based on the movement path control.

[0011] Optionally, in one possible implementation of the first aspect, the process of fitting the strip-shaped region to obtain multiple strip-shaped fitting areas, and laying the strip-shaped fitting areas based on a preset acquisition area of ​​the aerial collaborative device, to determine the second control point position of the strip-shaped laying area, includes: Identify the main region and subordinate regions within the strip-shaped region, obtain the regional center line of the main region, and divide the regional center line according to the turning points of the regional center line to obtain multiple first line segments; Determine the translation direction perpendicular to the first line segment, and perform copy translation processing on the first line segment to both sides based on the translation direction until the number of intersections with the outline of the corresponding area of ​​the subordinate area is equal to 1, and it is not located in the main area, thus obtaining two translated line segments; Connect the intersection points on the same side of the two translation line segments to obtain multiple strip-shaped fitting regions corresponding to the strip-shaped regions; Based on the preset acquisition area of ​​the aerial collaborative device, the strip fitting area is laid out sequentially to obtain multiple strip laying areas, and the center point of the strip laying area is used as the second point. At a preset collection distance from the second point, a second collection point is determined, and the second collection point is transferred to the second control point of the positioning cursor in the positioning screen.

[0012] Optionally, in one possible implementation of the first aspect, it also includes: When it is determined that the abnormal region is located at the top of the closed region, the corresponding abnormal region is designated as the area to be processed. When the area to be treated is identified as having a leakage anomaly, the corresponding area to be treated is designated as the leakage area, and the movement path is offset based on the leakage trajectory of the leakage area.

[0013] Optionally, in one possible implementation of the first aspect, when the leakage anomaly is identified in the area to be treated, the corresponding area to be treated is designated as the leakage area, and the movement path is offset based on the leakage trajectory of the leakage area, including: The two points between the locations of the leakage trajectory in the leakage area in the moving path are obtained as the change points; Take the change point at the front of the second or fourth sequence as the change point, take the change point at the back as the connection point, and take the point after the connection point as the intermediate point. Connect the changed point to the intermediate point, and connect the intermediate point to the connecting point.

[0014] Optionally, in one possible implementation of the first aspect, it also includes: The tilt amplitude of the ground-based collaborative equipment at the buffer device is obtained, and the aerial collaborative equipment is controlled to move in the opposite direction based on the tilt amplitude.

[0015] The second aspect of the present invention is an adaptive inspection and data processing device for unmanned aerial vehicles (UAVs) in closed environments, comprising: The control module is used to control the ground collaborative equipment to inspect the closed area and identify abnormal areas. The ground collaborative equipment has a positioning screen to control the aerial collaborative equipment. The moving module is used to move the positioning cursor in the positioning screen in a customized manner according to the regional shape of the abnormal area and the current position of the aerial collaborative device; The sending module is used to control the aerial collaborative device to move to the abnormal area based on the positioning cursor to collect data, and send the abnormal image of the abnormal area to the management terminal.

[0016] A third aspect of the present invention provides an electronic device comprising: a memory, a processor, and a computer program, the computer program being stored in the memory, and the processor executing the computer program to perform the methods described in the first aspect of the present invention and various possible methods related to the first aspect.

[0017] A fourth aspect of the present invention provides a storage medium storing a computer program, which, when executed by a processor, is used to implement the first aspect of the present invention and various methods possibly involved in the first aspect.

[0018] The beneficial effects of this invention are as follows: 1. This invention controls ground-based collaborative equipment to inspect enclosed areas and identify abnormal areas. A positioning cursor is displayed on the positioning screen of the ground-based collaborative equipment, and the cursor is moved in a customized manner based on the shape of the abnormal area and the current position of the aerial collaborative equipment, thereby guiding the aerial collaborative equipment to move and collect data. By replacing wireless signal control with visual guidance, this method eliminates dependence on the external communication environment and avoids connection loss or collision problems caused by signal shielding in enclosed environments. While ensuring normal inspection, it significantly improves the reliability of inspections in enclosed environments.

[0019] 2. This invention implements differentiated path planning for different morphologies of abnormal regions. For block-shaped abnormal regions, a block-shaped fitting area is obtained through fitting processing, and a path is laid based on a preset acquisition area to determine the first control point and generate the first sequence, thereby constructing a relatively comprehensive planar scanning path. For strip-shaped abnormal regions, the scheme identifies the main area and subordinate areas, and fits the strip-shaped region by dividing and translating the centerline to determine the second control point and generate the third sequence, constructing a path along the crack direction. Different processing and acquisition are performed for different situations.

[0020] 3. This invention implements dynamic obstacle avoidance for abnormal leakage. When a leakage trajectory is detected in the abnormal area at the top, the solution will obtain the affected points in the movement path and generate a detour path by connecting the change points, intermediate points and connecting points to avoid the drone directly crossing the dripping area. Attached Figure Description

[0021] Figure 1 The flowchart shows the UAV adaptive inspection and data processing method for closed scenarios provided by the present invention. Figure 2 This is a schematic diagram of the center line of the region provided by the present invention; Figure 3 This is a schematic diagram of the strip-shaped fitting region provided by the present invention; Figure 4 This is a schematic diagram of the structure of the UAV adaptive inspection and data processing device for closed scenarios provided by the present invention. Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.

[0024] It should be understood that in the various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0025] It should be understood that in this invention, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0026] It should be understood that in this invention, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0027] It should be understood that in this invention, "B corresponding to A", "B corresponding to A", "A and B correspond", or "B and A correspond" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0028] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0029] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0030] This invention provides an adaptive inspection and data processing method for unmanned aerial vehicles (UAVs) in closed environments, such as... Figure 1 As shown, steps S1-S3 are included: S1, control the ground collaborative equipment to inspect the closed area and identify abnormal areas, the ground collaborative equipment having a positioning screen to control the air collaborative equipment.

[0031] It's important to note that in enclosed environments like tunnels, culverts, or underground utility tunnels, the thick concrete and metal structures can block external signals. If a drone relies solely on wireless signals or GPS navigation in such places, it's highly susceptible to crashing into walls or losing connection due to signal interruptions. Therefore, we use a ground-based vehicle—a ground-based collaborative device—to perform a preliminary scan using cameras or radar. This vehicle is equipped with a positioning screen to guide the drone's movement. The drone simply needs to ensure the cursor on the positioning screen is centered on its data acquisition device. Subsequently, as the cursor moves, the drone will follow, ensuring the cursor remains centered. This eliminates the need for external signals; only the vehicle's coordination is required.

[0032] Among them, ground collaborative equipment refers to automated inspection devices that move on the ground, which can be unmanned vehicles with a display screen on the top or back for displaying cursors, i.e., a positioning screen. The enclosed area can be a closed area covered by cement or metal, such as a tunnel. The abnormal area refers to areas that require further inspection, such as wall cracks or structural detachment detected by image recognition technology.

[0033] Understandably, the server controls the ground-based collaborative equipment to enter the enclosed area and uses its onboard wide-angle camera or laser scanner to perform a preliminary scan of the tunnel walls. When the scan data reveals areas with significantly different textures from the surrounding environment—for example, identifying obvious longitudinal cracks or flaking—this location is identified as an anomaly. At this point, the ground-based collaborative equipment adjusts its position to the vicinity of the anomaly area, and a positioning cursor displayed on its positioning screen guides the aerial collaborative equipment to intervene.

[0034] S2, based on the regional shape of the abnormal area and the current position of the aerial collaborative device, the positioning cursor in the positioning screen is moved in a customized manner.

[0035] It should be noted that different types of anomaly areas have different geometries, requiring different flight trajectories to ensure complete and clear data acquisition. For example, blocky detachment areas or seepage areas require area coverage imaging, while strip-shaped cracks require line tracking imaging along the crack's direction.

[0036] Among them, the positioning cursor refers to the visual marker point displayed on the positioning screen, which can be a circle, a cross, etc., without specific limitations. Its main function is to be locked by the aerial collaborative equipment and used to guide its flight. The area morphology refers to the geometric appearance of the abnormal area, which is mainly divided into block morphology and strip morphology. For example, block-shaped peeling or water seepage, and strip-shaped gaps. The current position of the aerial collaborative equipment refers to the current position of the UAV, which can be determined by using the car as a reference.

[0037] Understandably, the server obtains the current location of the aerial collaborative device to determine the starting point for guidance. Contour analysis is performed on identified abnormal areas to extract their geometric features. The device, combining the geometric features of the abnormal area with the starting point location, plans a specific cursor movement trajectory within the display area of ​​the positioning screen for subsequent targeted data collection by the aerial collaborative device.

[0038] In some embodiments, step S2 (customizing the movement of the positioning cursor on the positioning screen according to the regional shape of the abnormal area and the current position of the aerial collaborative device) includes S21-S25: S21, obtain the positioning cursor in the positioning screen corresponding to the current position of the air-cooperative device as the initial cursor, and take the center point of the initial cursor as the initial point.

[0039] It should be noted that the aerial collaborative device, i.e., the drone, will continuously follow the vehicle as it moves to ensure that the positioning cursor is always in the center of its data collection area.

[0040] Therefore, we will obtain the initial position on the positioning screen corresponding to the current position of the drone, that is, the initial point. For example, when no abnormal area is detected, the initial point is located in the middle of the positioning screen.

[0041] S22, when the region shape of the abnormal region is determined to be blocky, the corresponding abnormal region is treated as a blocky region.

[0042] It should be noted that some anomalies within the tunnel, such as large-area concrete hollowing and spalling or patchy water seepage, may appear visually as blocky formations. For these anomalies, a single linear scan cannot fully cover their surface details, easily leading to missed detections.

[0043] Among them, the region morphology refers to the geometric shape of the abnormal region. The region morphology can be automatically identified by existing technologies. For example, OpenCV can be used to identify whether the shape of the region is blocky peeling and water seepage, or subsequent linear cracks. This is existing technology and will not be elaborated here.

[0044] Therefore, when the morphology of an abnormal region is determined to be blocky, the corresponding abnormal region is treated as a blocky region.

[0045] S23, based on the block area and the initial position, the positioning cursor in the positioning screen is moved in a customized manner.

[0046] In some embodiments, step S23 (customizing the movement of the positioning cursor in the positioning screen according to the block area and the initial position) includes S231-S234: S231, the block area is fitted to obtain a block fitting area, and the block fitting area is laid based on the preset acquisition area of ​​the aerial collaborative device to determine the first control point of the block laying area.

[0047] It's important to note that real-world concrete spalling or hollow areas often have irregular edges. Flying a drone along these irregular edges is difficult to control and risks missing the central area. Furthermore, the single-shot range of a drone camera is limited; larger anomaly areas cannot be captured in a single image. Therefore, we used regular rectangles to frame the irregular anomaly areas—a process known as "fitting." Then, based on the drone camera's maximum shooting range, a pre-defined acquisition area was created, filling this regular shape like laying floor tiles. The center of each tile is a shooting point. This approach breaks down large, irregular targets into several smaller, regular targets, ensuring comprehensive data acquisition and easy control.

[0048] Among them, the block fitting area refers to a regular rectangular area that can completely enclose the block area; the preset acquisition area is the acquisition area pre-configured by the personnel for the aerial collaborative equipment, that is, the acquisition range at a fixed distance from the acquisition point, which can be a rectangular area; the block laying area refers to each sub-area obtained after the fitting area is filled with grids using the preset acquisition area; the first control point refers to the pixel coordinates of each block laying area mapped to the positioning screen, that is, the position where the positioning cursor should stay when guiding the shooting of the sub-area.

[0049] In some embodiments, step S231 (fitting the blocky region to obtain a blocky fitting area, and laying the blocky fitting area based on the preset acquisition area of ​​the aerial collaborative device, and determining the first control point of the blocky laying area) includes S2311-S2313: S2311: Coordinate processing is performed on the block region, and a block fitting region is constructed based on the regional coordinate extreme values ​​of the block region.

[0050] It should be noted that the images collected by the ground-based collaborative equipment can be processed to determine the corresponding collection points, and then the drone can be controlled by the positioning screen to collect detailed data on the cracks or blocky detachment areas.

[0051] Therefore, any point in the image can be used as the origin to perform coordinate processing on the block region. Then, based on the regional coordinate extreme values ​​of the block region, that is, the maximum and minimum values ​​in the horizontal and vertical directions, a block fitting region of the block region can be constructed.

[0052] S2312, the block fitting area is laid based on the preset collection area of ​​the aerial collaborative device to obtain multiple block laying areas, and the center point of the block laying area is taken as the first point.

[0053] It's important to note that after constructing the rectangular frame, it's necessary to calculate exactly where the drone should be photographing. Since the drone camera's field of view is limited, a single shot cannot cover the entirety of a large, unusual area. Therefore, the camera's single-shot range is used as a template, filling the previously constructed rectangular frame like laying floor tiles. Each filled square is an independent photographic unit, and the center of this square is the physical focal point the camera needs to be aimed at. This approach breaks down the large-area data collection task into multiple standardized sub-regions, ensuring that each sub-region has a clearly defined aiming center.

[0054] It is worth mentioning that during the installation process, there may be cases where the area is smaller than the preset collection area. In such cases, the installation will still be carried out within the preset collection area, and subsequent collection will still be carried out from the center point. That is, it is permissible to have normal collection areas, and it is also permissible to have overlapping areas between preset collection areas.

[0055] Understandably, multiple block-shaped paving areas are generated by paving along the constructed block-shaped fitting area. Then, the geometric center coordinates of each paving area are obtained and marked as the first point.

[0056] S2313, at a preset collection distance from the first point, determine the first collection point and convert the first collection point to the first control point of the positioning cursor in the positioning screen.

[0057] It's easy to understand that, since the data collection range is fixed, the drone determines the first data collection point at a preset distance from the first point, flies to that point to collect data, and can directly utilize existing coordinate system transformation technology combined with the relative installation parameters of the ground-based collaborative equipment and the positioning screen to construct a spatial projection matrix. The equipment substitutes the first data collection point in three-dimensional space into this projection matrix for calculation, maps it onto the two-dimensional plane where the positioning screen is located, and calculates the corresponding screen pixel coordinates, i.e., the first control point. Alternatively, perspective projection mapping technology can be used, which are existing technologies and will not be elaborated here, that is, deriving two-dimensional screen pixel points from three-dimensional spatial points, thereby obtaining the first control point on the positioning screen corresponding to each data collection point of the drone.

[0058] S232, the first control points are sorted according to the laying order of the block laying area to obtain the first sequence.

[0059] It's important to note that while all the locations for taking photos have been identified, if the drone flies haphazardly between these points, it will not only be inefficient but also prone to producing blurry images or becoming unstable due to large maneuvers. Therefore, a flight route needs to be planned.

[0060] The laying order refers to the pre-defined traversal logic, such as laying from top to bottom or from left to right.

[0061] Therefore, the first control points will be sorted according to the laying order of the block laying area to obtain the first sequence.

[0062] S233, add the initial point to the front of the first sequence to obtain the second sequence, and connect the adjacent points in the second sequence in turn to obtain the movement path of the positioning cursor.

[0063] It's important to note that after planning the sequence within the work area, it's also necessary to consider the initial point on the positioning screen corresponding to the drone's current hovering position. This initial point is used as the starting point of the path and added to the front of the task queue. Then, these points are connected sequentially. This is done to construct a complete trajectory that smoothly transitions from the current location to the work area and continuously traverses all data collection points, ensuring the continuity of the guidance process.

[0064] Therefore, the initial point is added to the front of the first sequence to obtain the second sequence. Adjacent points in the second sequence are then connected in sequence to obtain the movement path of the positioning cursor.

[0065] S234, Move the cursor based on the movement path.

[0066] S24, when the region shape of the abnormal region is determined to be strip-shaped, the corresponding abnormal region is treated as a strip-shaped region.

[0067] It should be noted that another common feature inside tunnels is cracks, which appear as thin, elongated lines. For this type of shape, using planar scanning would waste a lot of time photographing the surrounding, irrelevant walls, making it inefficient. Furthermore, it's easy to understand that strip-shaped patterns can also be directly identified using existing technologies such as OpenCV, which will not be elaborated upon here.

[0068] Therefore, when the abnormal region has a strip-shaped shape, the corresponding abnormal region is treated as a strip-shaped region.

[0069] S25, based on the strip-shaped area and the initial position, the positioning cursor in the positioning screen is moved in a customized manner.

[0070] In some embodiments, step S25 (customizing the movement of the positioning cursor in the positioning screen according to the strip-shaped area and the initial position) includes S251-S254: S251, the strip-shaped region is fitted to obtain multiple strip-shaped fitting areas, and the strip-shaped fitting areas are laid based on the preset acquisition area of ​​the aerial collaborative device to determine the second control point of the strip-shaped laying area.

[0071] It's important to note that cracks within tunnels typically exhibit elongated and curved shapes, unlike the regularity of blocky areas. Using a single rectangle to frame the crack would include a large amount of useless background, and a single photograph cannot cover the entire crack. Furthermore, the crack's direction often changes, requiring segmented processing. This step employs a segmented fitting strategy, dividing the elongated anomaly region into several smaller segments that can be enclosed by rectangles—the strip-shaped fitting areas. Then, the drone's camera's preset acquisition area is used to fill these smaller segments. This breaks down the complex curve acquisition task into a series of continuous straight-line segment acquisition tasks, ensuring the drone can capture images along the natural direction of the crack.

[0072] In some embodiments, step S251 (fitting the strip-shaped region to obtain multiple strip-shaped fitting areas, and laying the strip-shaped fitting areas based on the preset acquisition area of ​​the aerial collaborative device, and determining the second control point of the strip-shaped laying area) includes S2511-S2515: S2511, identify the main area and subordinate area in the strip region, obtain the regional center of the main area, and divide the regional center according to the turning point of the regional center to obtain multiple first line segments.

[0073] It's important to note that strip-shaped anomalies such as tunnel cracks are often not perfect straight lines, but rather irregular shapes with many fine burrs and twists. Therefore, we distinguish the core direction of the crack (the main region) from the edge burrs (the subordinate regions), extracting a centerline that represents the crack's skeleton-like structure. This can be done directly using existing image recognition techniques, such as identifying the crack's width value with the largest average as the main region and the remaining connected cracks as subordinate regions. Alternatively, we can use OpenCV for identification, leveraging existing technology. Then, we identify the bends along the centerline and use these points as boundaries to break the curved centerline into several straight segments. This simplifies the complex curved path, facilitating the subsequent construction of multiple strip-shaped fitting regions.

[0074] Among them, the main region refers to the path part with the best connectivity and highest pixel density in the strip region, representing the main direction of the crack; the subordinate region refers to the non-core region attached to the main region, such as the fine branch cracks; the center line of the region refers to the line that runs through the geometric center of the main region; the turning point can be a key node on the center line of the region where the curvature changes beyond a preset threshold, or it can be the turning point of a line segment; the first line segment refers to the line segment that is approximately a straight line obtained after the center line is divided by the turning point.

[0075] Understandably, the main and subordinate regions within the strip-shaped area are identified, and the midline of the main region is obtained. When the curvature of a point on the midline exceeds a set value, it is marked as a turning point. Using these turning points as cutting points, the entire curved midline is divided into multiple segments, each of which is considered a straight first line segment. See [link to relevant documentation]. Figure 2 The area representing the direction of the crack is the main area, and the areas corresponding to the other cracks connected to it are the subordinate areas. Obtain the midline of the main area, and then divide the midline according to the two turning points to obtain three first line segments.

[0076] S2512, determine the translation direction perpendicular to the first line segment, and perform copy translation processing on the first line segment to both sides based on the translation direction until the number of intersections with the outline of the corresponding area of ​​the subordinate area is equal to 1 and is not located in the main area, thus obtaining two translated line segments.

[0077] It should be noted that after determining the first line segment, the width of the crack also needs to be determined so that the generated shooting area can completely cover the crack. Since the width of the crack varies in different places, and the edges, i.e., the subordinate areas, are uneven, it is not possible to simply set a fixed width. Existing technologies often perform uniform and coarse processing, and cannot perform targeted shooting for different locations.

[0078] Therefore, the server determines the perpendicular normal direction of each first line segment as the translation direction. It then controls the simultaneous or separate copying and translation of this line segment to both sides of the main area. During translation, the intersection of the moving line segment with the outer contour of the strip-shaped region (including subordinate regions) is detected. Translation stops when the line segment just touches the outermost edge of the contour, meaning it has only one intersection point with the contour and is not located within the main area. The two lines at these extreme points on both sides then become the boundary lines of the crack segment, which are used as the translation line segments.

[0079] S2513, connect the intersection points on the same side of the two translation line segments to obtain multiple strip-shaped fitting areas corresponding to the strip-shaped regions.

[0080] It is understandable that connecting the intersection points on the same side of the two translated line segments yields multiple strip-shaped fitting regions corresponding to the strip-shaped regions, see [link to relevant documentation]. Figure 3 Connect the intersection points on the same side of two corresponding translation line segments to obtain the strip-shaped fitting area corresponding to the strip-shaped region. Here, only one of the first line segments is shown.

[0081] It is worth mentioning that when the two ends of the main trunk area are tilted, the two ends of the first line segment will be extended appropriately until they are aligned with the tilted position.

[0082] S2514, based on the preset acquisition area of ​​the aerial collaborative device, the strip fitting area is laid out sequentially to obtain multiple strip laying areas, and the center point of the strip laying area is taken as the second point.

[0083] Similarly, the strip-shaped fitting area is laid out using the preset collection area to obtain multiple strip-shaped laying areas, and the center point of the strip-shaped laying area is used as the second point.

[0084] S2515, at a preset acquisition distance from the second point, determine the second acquisition point and transfer the second acquisition point to the second control point of the positioning cursor in the positioning screen.

[0085] Similarly, consistent with the principle of step S2313, the second control point of the positioning cursor in the positioning screen is determined.

[0086] S252, the second control points are sorted according to the laying sequence of the strip laying area to obtain the third sequence.

[0087] It should be noted that the crack extends continuously, so the filming process must also be continuous. If the generated filming points are out of order, for example, filming the head of the crack first, then the tail, and then flying back to the middle, it will cause the drone's flight path to become chaotic.

[0088] S253, add the initial point to the front of the third sequence to obtain the fourth sequence, and connect the adjacent points in the fourth sequence in turn to obtain the movement path of the positioning cursor.

[0089] Similarly, the starting point is added before the third sequence to obtain the fourth sequence. Subsequently, adjacent points in the fourth sequence are connected to obtain the movement path of the positioning cursor.

[0090] S254, Move the cursor based on the movement path.

[0091] Based on the above embodiments, A1-A2 are also included: A1, when it is determined that the abnormal area is located at the top of the closed area, the corresponding abnormal area is taken as the area to be processed.

[0092] It should be noted that the tunnel roof is a rather unique location in the structure. Due to geological pressure and groundwater, the roof is most prone to developing through-cracks and causing water leakage. Therefore, anomalies at the roof cannot be treated in the same way as those at the sidewalls.

[0093] Therefore, when the abnormal area is located at the top of the closed area, the corresponding abnormal area is taken as the area to be processed. The top position can be the arc-shaped position at the top of the tunnel.

[0094] A2, when the area to be processed is identified as having leakage anomalies, the corresponding area to be processed is designated as the leakage area, and the movement path is offset based on the leakage trajectory of the leakage area.

[0095] The leakage trajectory is the path of water droplets falling in the leakage area.

[0096] In some embodiments, step A2 (when the area to be treated is identified to have a leakage anomaly, the corresponding area to be treated is taken as the leakage area, and the movement path is offset based on the leakage trajectory of the leakage area) includes A21-A23: A21, obtain two points between the leakage trajectory of the leakage area in the moving path as the change points.

[0097] It should be noted that when water leakage is detected at the top, the water flow will form a linear dripping trajectory along the gap or in the direction of gravity, i.e., the leakage trajectory. If the drone's original flight path happens to overlap or intersect this water path, flying directly over it will result in getting wet. To avoid this danger, the start and end points of the affected section must first be identified on the original flight path, but current technology does not handle this situation.

[0098] Therefore, the identified leakage area and its leakage trajectory are projected onto the coordinate system of the two-dimensional positioning screen and analyzed against the planned movement path, the second sequence, or the fourth sequence. The device identifies the range of the leakage trajectory on the path and selects one point before and one point after that range, marking these two points as change points.

[0099] A22, take the change point located at the front in the second or fourth sequence as the change point, take the change point located at the back as the connection point, and take the point located after the connection point as the intermediate point.

[0100] It should be noted that once the two affected points are identified, the path to avoid those two points can be taken.

[0101] Therefore, the server obtains the order of the points in the sequence, i.e., the order in which the drones flew, and names the two changed points identified in A21. The point that appears earlier in the sequence is marked as a changed point, and the point that appears later in the sequence is marked as a connecting point. Then, the device reads the next point after the connecting point in the original path sequence, the second sequence, or the fourth sequence, and marks it as an intermediate point.

[0102] A23, connect the changed point to the intermediate point, and connect the intermediate point to the connecting point.

[0103] Understandably, the server deletes the original connecting line segment between the changed point and the connecting point in the path planning data. Subsequently, the device establishes new connections: first, it generates a path segment from the changed point to the intermediate point, and then it generates a path segment from the intermediate point to the connecting point. Through this reconnection operation, the original path directly traversing the leak area is modified into a polyline path from the changed point to the intermediate point to the connecting point. The positioning cursor will guide the drone along this new path, thereby achieving safe avoidance of the leak area.

[0104] Based on the above embodiments, B1 is also included: B1, obtain the tilt amplitude of the ground cooperative equipment at the buffer device, and control the air cooperative equipment to move in the opposite direction based on the tilt amplitude.

[0105] It should be noted that ground-based collaborative equipment, such as unmanned vehicles, often encounter speed bumps or road protrusions when driving in tunnels or utility tunnels, which can cause the drone to shake.

[0106] Among them, buffer devices refer to obstacle facilities on the road surface that cause changes in the attitude of ground cooperative equipment, such as speed bumps.

[0107] Therefore, we will obtain the tilt amplitude of the ground cooperative equipment at the buffer device in advance. This can be done by inputting the height of the speed bump into the tilt sensor to read the tilt change, thereby obtaining the tilt amplitude of the ground cooperative equipment at the buffer device, or by directly calculating the tilt amplitude through existing technologies such as simulation. Subsequently, the aerial cooperative equipment will be controlled to move in the opposite direction based on the tilt amplitude, thereby avoiding shaking.

[0108] S3, based on the positioning cursor, control the aerial collaborative device to move to the abnormal area to collect data, and send the abnormal image of the abnormal area to the management terminal.

[0109] See Figure 4 This is a schematic diagram of the structure of an adaptive inspection and data processing device for unmanned aerial vehicles (UAVs) in closed environments provided in an embodiment of the present invention. The adaptive inspection and data processing device for unmanned aerial vehicles in closed environments includes: The control module is used to control the ground collaborative equipment to inspect the closed area and identify abnormal areas. The ground collaborative equipment has a positioning screen to control the aerial collaborative equipment. The moving module is used to move the positioning cursor in the positioning screen in a customized manner according to the regional shape of the abnormal area and the current position of the aerial collaborative device; The sending module is used to control the aerial collaborative device to move to the abnormal area based on the positioning cursor to collect data, and send the abnormal image of the abnormal area to the management terminal.

[0110] See Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. The electronic device 50 includes: a processor 51, a memory 52, and a computer program; wherein... The memory 52 is used to store the computer program, and the memory may also be flash memory. The computer program is, for example, an application program or functional module that implements the above method.

[0111] The processor 51 is configured to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0112] Alternatively, the memory 52 can be either standalone or integrated with the processor 51.

[0113] When the memory 52 is a device independent of the processor 51, the device may further include: Bus 53 is used to connect the memory 52 and the processor 51.

[0114] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the methods provided in the various embodiments described above.

[0115] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0116] The present invention also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the methods provided in the various embodiments described above.

[0117] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for adaptive inspection and data processing of unmanned aerial vehicles (UAVs) in closed environments, characterized in that, include: The ground-based collaborative equipment is used to inspect closed areas and identify abnormal areas. The ground-based collaborative equipment has a positioning screen that controls the aerial collaborative equipment. Based on the regional shape of the abnormal area and the current position of the aerial collaborative device, the positioning cursor in the positioning screen is moved in a customized manner; Based on the positioning cursor, the aerial collaborative device is controlled to move to the abnormal area to collect data, and the abnormal image of the abnormal area is sent to the management terminal.

2. The method according to claim 1, characterized in that, The step of customizing the movement of the positioning cursor on the positioning screen based on the regional shape of the abnormal area and the current position of the aerial collaborative device includes: The positioning cursor in the positioning screen corresponding to the current position of the air-cooperative device is obtained as the initial cursor, and the center point of the initial cursor is taken as the initial point. When the region of an anomaly is determined to be blocky, the corresponding anomaly region is treated as a blocky region. Based on the block-shaped region and the initial position, the positioning cursor in the positioning screen is moved in a customized manner; When the abnormal region is determined to be strip-shaped, the corresponding abnormal region is treated as a strip-shaped region. Based on the strip-shaped area and the initial position, the positioning cursor in the positioning screen is moved in a customized manner.

3. The method according to claim 2, characterized in that, The step of customizing the movement of the positioning cursor in the positioning screen based on the block region and the initial position includes: The blocky region is fitted to obtain a blocky fitting area, and the blocky fitting area is laid based on the preset acquisition area of ​​the aerial collaborative device to determine the first control point of the blocky laying area. The first control points are sorted according to the laying sequence of the block laying area to obtain the first sequence; Add the initial point to the front of the first sequence to obtain the second sequence. Connect the adjacent points in the second sequence in turn to obtain the movement path of the positioning cursor. The cursor is moved based on the movement path control.

4. The method according to claim 3, characterized in that, The process of fitting the blocky region to obtain a blocky fitting area, and then laying the blocky fitting area based on the preset acquisition area of ​​the aerial collaborative device, and determining the first control point of the blocky laying area, includes: The blocky regions are processed by coordinate transformation, and a blocky fitting region is constructed based on the regional coordinate extreme values ​​of the blocky regions. The block fitting area is laid out based on the preset collection area of ​​the aerial collaborative equipment to obtain multiple block laying areas, and the center point of the block laying area is taken as the first point. At a preset collection distance from the first point, a first collection point is determined, and the first collection point is converted to the first control point of the positioning cursor in the positioning screen.

5. The method according to claim 2, characterized in that, The step of customizing the movement of the positioning cursor in the positioning screen based on the strip-shaped region and the initial position includes: Multiple strip-shaped fitting areas are obtained by fitting the strip-shaped region, and the strip-shaped fitting areas are laid out based on the preset acquisition area of ​​the aerial collaborative device to determine the second control point of the strip-shaped laying area; The second control points are sorted according to the laying sequence of the strip laying area to obtain the third sequence; The initial point is added before the third sequence to obtain the fourth sequence. Adjacent points in the fourth sequence are connected in sequence to obtain the movement path of the positioning cursor. The cursor is moved based on the movement path control.

6. The method according to claim 5, characterized in that, The process of fitting the strip-shaped region yields multiple strip-shaped fitting areas, and these areas are then laid out based on a preset acquisition area of ​​the aerial collaborative device. The second control point of each strip-shaped area is determined, including: Identify the main region and subordinate regions within the strip-shaped region, obtain the regional center line of the main region, and divide the regional center line according to the turning points of the regional center line to obtain multiple first line segments; Determine the translation direction perpendicular to the first line segment, and perform copy translation processing on the first line segment to both sides based on the translation direction until the number of intersections with the outline of the corresponding area of ​​the subordinate area is equal to 1, and it is not located in the main area, thus obtaining two translated line segments; Connect the intersection points on the same side of the two translation line segments to obtain multiple strip-shaped fitting regions corresponding to the strip-shaped regions; Based on the preset acquisition area of ​​the aerial collaborative device, the strip fitting area is laid out sequentially to obtain multiple strip laying areas, and the center point of the strip laying area is used as the second point. At a preset collection distance from the second point, a second collection point is determined, and the second collection point is transferred to the second control point of the positioning cursor in the positioning screen.

7. The method according to claim 3 or 5, characterized in that, Also includes: When it is determined that the abnormal region is located at the top of the closed region, the corresponding abnormal region is designated as the area to be processed. When the area to be treated is identified as having a leakage anomaly, the corresponding area to be treated is designated as the leakage area, and the movement path is offset based on the leakage trajectory of the leakage area.

8. The method according to claim 7, characterized in that, When the area to be treated is identified as having a leakage anomaly, the corresponding area to be treated is designated as the leakage area, and the movement path is offset based on the leakage trajectory of the leakage area, including: The two points between the locations of the leakage trajectory in the leakage area in the moving path are obtained as the change points; Take the change point at the front of the second or fourth sequence as the change point, take the change point at the back as the connection point, and take the point after the connection point as the intermediate point. Connect the changed point to the intermediate point, and connect the intermediate point to the connecting point.

9. The method according to claim 1, characterized in that, Also includes: The tilt amplitude of the ground-based collaborative equipment at the buffer device is obtained, and the aerial collaborative equipment is controlled to move in the opposite direction based on the tilt amplitude.

10. An adaptive inspection and data processing device for unmanned aerial vehicles (UAVs) in closed environments, characterized in that: include: The control module is used to control the ground collaborative equipment to inspect the closed area and identify abnormal areas. The ground collaborative equipment has a positioning screen to control the aerial collaborative equipment. The moving module is used to move the positioning cursor in the positioning screen in a customized manner according to the regional shape of the abnormal area and the current position of the aerial collaborative device; The sending module is used to control the aerial collaborative device to move to the abnormal area based on the positioning cursor to collect data, and send the abnormal image of the abnormal area to the management terminal.

Images