Room processing method, system and device for full coverage search tasks

Abstract

The present invention relates to a room processing method, system and device for full coverage search tasks. By proposing a room processing method based on OpenCV edge detection and bidirectional connected edge table data structure, the map established by the drone is processed and the room segmentation is performed using the OpenCV open source computer vision library. Among them, Canny edge detection and Hough transform technology are used to segment the room, and the DCEL data structure is used to represent the room, which effectively completes the room segmentation and topology construction under the full coverage search task, and provides structured information for subsequent multi-robot (drone) task allocation and search. Experimental results show that this method performs well in processing various complex room structures.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and relates to a room processing method, system and device for a full coverage search task. Background Art

[0002] When performing a full coverage search task in a specified spatial area, there are two main methods for decomposing the spatial area environment, one is based on the classic cell decomposition method, and the other is based on the cell grid method. The cell decomposition method decomposes the unobstructed free space into simple, non-overlapping areas. One of the simplest cell decomposition methods is the trapezoidal decomposition method, which only handles planar polygonal obstacle spaces; the decomposition method based on Morse decomposition has the ability to handle non-polygonal spaces, but it cannot handle linear environments. The decomposition method based on the cell grid has the problem of choosing between grid size, roughness, and complexity. Therefore, in general, for the decomposition of rooms in a spatial environment with walls and doors, the current decomposition technology still has technical problems that make it difficult to effectively perform room segmentation and topology construction under full coverage search tasks. Summary of the invention

[0003] In response to the problems existing in the above-mentioned traditional methods, the present invention proposes a room processing method for full coverage search tasks, a room processing system for full coverage search tasks and a computer device, which can effectively perform room segmentation and topology construction under full coverage search tasks.

[0004] In order to achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0005] On the one hand, a room processing method for a full coverage search task is provided, comprising the steps of:

[0006] The original map is constructed by sweeping the mission space area with a drone and using the SLAM algorithm;

[0007] Apply the Canny algorithm in the OpenCV library to perform image enhancement and edge detection on the original map to obtain a clear map image;

[0008] Use Hough transform to extract line segments from map images, detect straight lines in map images and complete the digitization of map images;

[0009] According to the straight lines of the digitized map image, the DCEL data structure of the map image is established according to the set half-edge connection rules;

[0010] After extracting the vertices and edges in the map image using the DCEL data structure, all the half edges in the DCEL data structure are traversed to obtain all the faces;

[0011] Use vertices, edges and faces to construct the room topology corresponding to the map image.

[0012] On the other hand, a room processing system for a full coverage search task is also provided, including:

[0013] Mapping module, used to sweep the mission space area by drone and build the original map using SLAM algorithm;

[0014] Image enhancement module, used to apply the Canny algorithm in the OpenCV library to perform image enhancement and edge detection on the original map to obtain a clear map image;

[0015] A digitization module is used to extract line segments from a map image using Hough transform, detect straight lines in the map image and complete the digitization of the map image;

[0016] A structure building module is used to build a DCEL data structure of a map image according to the straight lines of the digitized map image and the set half-edge connection rules;

[0017] The plane segmentation module is used to extract vertices and edges in the map image using the DCEL data structure, and then traverse all half edges in the DCEL data structure to obtain all faces;

[0018] The topology building module is used to build a room topology map corresponding to the map image using vertices, edges and faces.

[0019] On the other hand, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned room processing method for full coverage search tasks when executing the computer program.

[0020] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the room processing method for the full coverage search task are implemented.

[0021] One of the above technical solutions has the following advantages and beneficial effects:

[0022] The room processing method, system and device for the full coverage search task proposed a room processing method based on OpenCV edge detection and bidirectional connected edge list (DCEL) data structure, and used the OpenCV open source computer vision library to perform image processing and room segmentation on the map established by the drone. Among them, Canny edge detection and Hough transform technology were used to segment the room, and the DCEL data structure was used to represent the room, effectively completing the room segmentation and topology construction under the full coverage search task, and providing structured information for subsequent multi-robot (drone) task allocation and search. Experimental results show that this method performs well in processing various complex room structures (such as concave polygons and polygons with holes). BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 It is a flowchart of a room processing method for a full coverage search task in one embodiment;

[0025] Figure 2 A schematic diagram of the effect of an original map constructed by a SLAM algorithm in one embodiment;

[0026] Figure 3 A schematic diagram of a Canny edge detection result in one embodiment;

[0027] Figure 4 A schematic diagram of the effect of Hough transform in detecting straight lines in one embodiment;

[0028] Figure 5 is an example schematic diagram of a doubly connected edge table in one embodiment;

[0029] Figure 6 A schematic diagram of a process of constructing a linked list of a DCEL data structure in one embodiment;

[0030] Figure 7 A schematic diagram of an application scenario of a room processing method for a full coverage search task in an embodiment;

[0031] Figure 8 Schematic diagram of room cutting, area and constructed topology in one embodiment; wherein (a) is a schematic diagram of room cutting and area, and (b) is a constructed topology;

[0032] Fig. 9 Schematic diagram of the module structure of a room processing system for full coverage search tasks in one embodiment. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0034] It should be noted that the reference to "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The presentation of this phrase at various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It will be appreciated by those skilled in the art that the embodiments described herein may be combined with other embodiments. The term "and / or" used in the specification of the present invention and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0035] The following will describe the implementation of the present invention in detail with reference to the accompanying drawings in the embodiment diagram of the present invention.

[0036] Planar subdivision is a fundamental problem in computational geometry. It refers to the process of dividing a plane into several non-intersecting regions using a set of line segments or curves. These regions are usually called faces or cells. Planar subdivision can be seen as a division or decomposition of the plane so that the points inside each region satisfy certain specific properties or relationships. Planar subdivision is usually represented by a planar straight-line graph, in which the vertices correspond to the endpoints and intersections of the line segments, the edges correspond to the line segments or curves themselves, and the faces correspond to the segmented regions.

[0037] There are many methods for plane segmentation, among which the scanline algorithm is the most common. The scanline algorithm uses a vertical scan line that sweeps across the plane from left to right, sorts and updates the line segments according to the intersection of the scan line and the line segments, and thus obtains the result of plane segmentation. The researchers applied the scanline algorithm to generate the optimal coverage path, which decomposes the coverage area into sub-areas, selects a sequence of these sub-areas, and then generates a path that covers each sub-area in turn. In the traditional scanline algorithm, all sub-areas have the same scanning direction. Based on the traditional scanline algorithm, the researchers proved that for convex polygon environments, an edge perpendicular to the boundary of the coverage area is the direction of the optimal single scan line decomposition.

[0038] On this basis, the researchers also proposed the concept of minimum height sum (MSA) decomposition, which can obtain better decomposition results by allowing different sub-regions to use different scanning directions. To generate MSA decomposition, the researchers used scan line decomposition in multiple directions to obtain an initial unit decomposition, and then used a dynamic programming algorithm to merge them. This method effectively combines the advantages of the scan line algorithm and dynamic programming. In addition, by analyzing the scan line decomposition process, the researchers revealed the intrinsic connection between the optimal decomposition and the polygon diameter function.

[0039] In recent years, another popular method is the method based on the DCEL (doubly connected edge list) structure. DCEL is a commonly used data structure that contains three basic elements: faces, half edges, and vertices. Some researchers have proposed a plane segmentation method based on DCEL. Its innovation is that it can handle overlapping polygons. It can also handle complex situations such as polygon overlays without edge intersections and polygons with holes. Other researchers have proposed a distributed DCEL method (distributed doubly connected edge list, DCEL), which extends DCEL to a distributed environment. Its DDCEL adopts a two-stage paradigm to generate vertices, half edges, and faces. This method has significant performance advantages over existing technologies that support large-scale spatial networks.

[0040] There is no in-depth research on room decomposition processing for spatial environments with walls and doors for full coverage search tasks, and this manual solves this problem through image processing calculations. This paper innovatively uses the DCEL data structure to represent and organize points, line segments and surfaces in bitmaps, segments the SLAM (Simultaneous Localization and Mapping) map, calculates the area of ​​each room, and successfully constructs a vertex weighted topological map to facilitate subsequent room task allocation.

[0041] The method based on the DCEL data structure can segment maps well, even overlapping maps, and DCEL can also be extended to distributed environments. However, there is no research in the prior art that uses the DCEL data structure for map segmentation in robot (drone) full coverage search tasks. In addition, the study found that there is currently no method to connect the segmented rooms through doors and model the vertices in the topology graph connected by edges. The construction of this topology graph is more critical for robots (drones) with full coverage search, because the topology graph is the basis for task allocation.

[0042] In order to better understand the technical solutions provided in this specification, the following first introduces the relevant technical terms involved in this specification:

[0043] SLAM is a technology widely used in robots and autonomous vehicles that allows robots to simultaneously build a map of the environment and determine their own position in the map in an unknown environment. This technology is achieved by combining input from sensors such as lidar, cameras, and inertial measurement units. The basic components of SLAM are:

[0044] (1) Sensor data: LiDAR calculates the distance by emitting a laser beam and measuring the time it takes for it to reflect back, thereby constructing a 3D point cloud map of the surrounding environment. The camera captures images of the environment and can estimate motion and build maps through visual algorithms (such as feature point matching, optical flow method, etc.). The inertial measurement unit (IMU) provides information about the robot's acceleration and angular velocity to assist in positioning and reduce errors. (2) Front-end processing: Real-time processing of sensor data, extraction of feature points or image features in the environment, and preliminary motion estimation. (3) Back-end optimization: Optimize the motion estimation and map information obtained by the front-end to reduce cumulative errors and improve the accuracy and robustness of the system. Common methods include graph optimization, filters (such as Kalman filters, particle filters, etc.). (4) Loop detection: Identify whether the robot has returned to a previously visited location, which helps to correct accumulated errors and achieve globally consistent map construction. (5) Map construction: Construct a map of the surrounding environment based on sensor data and processing results. The map can be a sparse feature point map, a semi-dense depth map, or a dense 3D reconstructed map.

[0045] Image processing: refers to the technology of using computers to analyze and process images to extract useful information, improve visual effects or meet other needs. Image processing generally includes image compression, enhancement and restoration, matching, description and recognition. Specifically, it can include the following aspects:

[0046] (1) Image transformation: Various image transformation methods (such as Fourier transform, Walsh transform and discrete cosine transform, etc.) are used to convert spatial domain processing into transform domain processing to reduce the amount of calculation and obtain more efficient processing. (2) Image coding and compression: The amount of data describing the image is reduced through coding and compression technology to save image transmission and processing time and reduce the occupied memory capacity. (3) Image enhancement and restoration: The purpose is to improve the quality of the image, such as removing noise and improving image clarity. Image enhancement does not consider the reasons for image degradation, while image restoration requires a certain understanding of the reasons for image degradation and establishes a "degradation model" based on the degradation process, and then uses a certain filtering method to restore or reconstruct the original image. (4) Image segmentation: It is to extract meaningful feature parts from the image, such as edges and regions in the image, which is the basis for further image recognition, analysis and understanding. (5) Image description: It is a necessary prerequisite for image recognition and understanding. For binary images, their geometric characteristics can be used to describe the characteristics of the object; for general images, image description includes two-dimensional shape description (boundary description and region description) and two-dimensional texture feature description. (6) Image classification (recognition): It belongs to the category of pattern recognition. Its main content is to perform image segmentation and feature extraction after certain preprocessing (enhancement, restoration and compression) of the image, so as to make judgments and classifications.

[0047] OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library that provides a wealth of functions and algorithms and is widely used in image processing, image analysis, and machine vision. Based on the Apache 2.0 license, it is a cross-platform computer vision library that can run on multiple operating systems such as Linux, Windows, macOS, Android, and iOS. The features of OpenCV include:

[0048] (1) Image processing: Provides basic image processing functions such as filtering, edge detection and image transformation, and can perform operations such as image enhancement, noise reduction, smoothing and repair. (2) Object detection and recognition: Supports algorithms such as face detection, target tracking and image segmentation, which can be used for real-time monitoring, image analysis and video surveillance. (3) Feature extraction and description: Provides feature extraction and description algorithms such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), and ORB (Oriented FAST and Rotated BRIEF), which are used for tasks such as image registration and panoramic stitching. (4) Machine learning: Contains machine learning algorithms such as normal Bayes, K nearest neighbor, support vector machine, decision tree, random forest and artificial neural network, which can be used for tasks such as image classification and clustering. (5) Deep learning: The DNN (Deep Neural Network) module supports loading pre-trained deep learning models for advanced tasks such as image recognition and target detection.

[0049] Hough (Hough Transform) Line: Hough Line Transform in OpenCV is a technique for detecting lines in an image. It converts the line detection problem into an accumulation problem in parameter space by representing the lines in the image in polar coordinate space. The following is a detailed explanation of Hough Line Transform in OpenCV:

[0050] (1) The basic principle of Hough transform: Hough space is a feature space used for image analysis, which is used to describe line segments or curves with the same shape in the image; for line detection, the dimension of the parameter space is 2, namely the slope and intercept of the line (usually expressed as distance and angle in polar coordinates). (2) Mapping mechanism: Map points in the image space to the parameter space. A line in the image space corresponds to a point in the parameter space, and a point in the image space corresponds to a curve in the parameter space. (3) Voting mechanism: Each edge point "votes" for each potential line in the parameter space to which it may belong. The position of the vote depends on the angle of the line and its distance from the origin; each cell in the accumulator array receives a vote and accumulates these votes. (4) Peak detection: Analyze the accumulator array to find cells with the highest voting counts. These peaks represent lines supported by the largest number of edge points and are considered to be the most likely lines in the image.

[0051] Double Connected Edge List (DCEL) is a data structure used to represent planar polygonal meshes (such as polygons, polygon partitions, etc.). It mainly constructs the structure of the graph by connecting edges, vertices, and faces, so that each element can access its adjacent elements in both directions. DCEL is widely used in the fields of computational geometry and graphics, especially when dealing with problems such as polygon segmentation, clipping, and Boolean operations. DCEL mainly consists of the following three parts:

[0052] (1) Edges: Each edge connects two vertices and belongs to two faces (or only to the external area, i.e., the boundary); each edge has two pointers in DCEL, one pointing to the vertex where it starts (origin) and the other pointing to the vertex where it ends (destination); at the same time, each edge also points to the face on its left (leftFace) and the face on its right (rightFace); in some cases, if the edge is part of the external boundary, there may be only one face or both face pointers point to a special "external" or "infinite" face. (2) Vertices: They are the corner points of a polygon, and each edge starts or ends at a vertex; in DCEL, each vertex maintains a list pointing to all edges starting from the vertex (usually a circular linked list or a doubly linked list). (3) Faces: They are the areas enclosed by edges and can be inside or outside a polygon; in DCEL, each face maintains a list pointing to all edges surrounding the face (usually also a circular linked list or a doubly linked list). For the external area, this list may include all boundary edges.

[0053] A topology map is an abstract graph that shows how devices in a network are connected, without showing their physical location or distance. It is derived from topology, a method that studies the properties of geometric figures that are independent of size and distance. In fields such as network management, communications, and computer science, topology maps play a key role because they can help people more clearly understand the structure and interaction of complex systems. Topology maps are mainly composed of the following elements:

[0054] (1) Node: represents the devices in the network, such as routers, switches, and servers. These nodes are the basic elements in the topology diagram and are used to represent the actual devices in the network. (2) Link: represents the connection relationship between nodes, usually represented by lines, cables, or optical fibers. Links are the key elements that connect nodes in the topology diagram, which show the physical or logical connection between devices. (3) Weight (optional): represents the cost or delay of the connection, such as latency or bandwidth. This element is very useful when evaluating network performance and can help people understand the pros and cons of different connection paths.

[0055] In one embodiment, Figure 1 As shown, a room processing method for a full coverage search task is provided, which may include the following processing steps S10 to S20:

[0056] S10, sweeping the mission space area by UAV and building the original map using SLAM algorithm;

[0057] S12, applying the Canny algorithm in the OpenCV library to perform image enhancement and edge detection on the original map to obtain a clear map image;

[0058] S14, extracting line segments from the map image using Hough transform, detecting straight lines in the map image and completing digitization of the map image;

[0059] S16, establishing a DCEL data structure of the map image according to the set half-edge connection rule based on the straight lines of the digitized map image;

[0060] S18, after extracting vertices and edges in the map image using the DCEL data structure, traverse all half edges in the DCEL data structure to obtain all faces;

[0061] S20, constructing a room topology map corresponding to the map image using vertices, edges and faces.

[0062] The room processing method for the full coverage search task proposed a room processing method based on OpenCV edge detection and bidirectional connected edge list (DCEL) data structure, and used the OpenCV open source computer vision library to perform image processing and room segmentation on the map established by the drone. Among them, Canny edge detection and Hough transform technology were used to segment the room, and the DCEL data structure was used to represent the room, effectively completing the room segmentation and topology construction under the full coverage search task, providing structured information for subsequent multi-robot (drone) task allocation and search. Experimental results show that this method performs well in processing various complex room structures (such as concave polygons and polygons with holes).

[0063] It should be noted that, assuming that the SLAM algorithm is constructed as follows Figure 2 A bitmap of the unknown spatial environment shown (called the original map), Figure 2 The lines in the image represent obstacles or walls. Since the original map constructed by the SLAM algorithm may have blurred map boundaries and noise, the original map needs to be processed first to enhance the clarity of the image. After obtaining a clearer map image, it is necessary to detect the line segments in the map image and use the line segments to cut the room apart to complete the room area processing.

[0064] First, image enhancement and edge detection are performed: the map constructed by the SLAM algorithm is an RGB three-channel image. The grayscale operation can convert it into a single channel to simplify the data processing process. In order to enhance the clarity of the wall (image outline), the classic Canny algorithm is used for edge detection. The Canny algorithm is a multi-scale edge detection algorithm that aims to obtain the accurate boundary of the image, which can achieve the effect of optimizing image clarity and making the image description more accurate after digitization.

[0065] Canny algorithm is widely used in the field of image processing, and is constantly improved and innovated. Using Canny edge detection, there are usually the following steps: denoise the image before detecting the edge of the image, usually a Gaussian smoothing filter can be used to reduce noise; calculate the gradient amplitude and direction, usually the gradient direction takes 4 angles, 0, 45, 90 and 135 degrees; perform non-maximum suppression to eliminate non-edge pixels, leaving only a small number of thin lines; select a hysteresis threshold, which requires retaining or excluding pixels. Two thresholds to select the edge. Using the Canny algorithm, you can get a clear map image such as Figure 3 shown.

[0066] Then perform line segment extraction: The task of cutting the room is completed by Hough transform, so in this embodiment, Hough transform is used to detect straight lines on the image generated by Canny edge detection to obtain line segment coordinates. Hough transform is mainly used to detect geometric shapes in images, such as straight lines and circles. The principle of Hough transform is to map the pixel points in the map image to the parameter space, and use accumulation to find the peak points in the parameter space to identify the target shape.

[0067] In terms of line detection, Hough transform maps each pixel in the map image to a curve in the parameter space, and the intersection of the straight lines in the parameter space is the straight line in the map image.

[0068] In one embodiment, regarding the above step S14, the process of detecting the straight line in the map image may further include the following processing steps:

[0069] The line segments in the map image are divided into two categories: horizontal line segments and vertical line segments, and a spacing threshold is set;

[0070] For horizontal line segments, if the difference between the ordinates of two horizontal line segments is less than the spacing threshold and there is overlap between the abscissas, the two horizontal line segments are determined to be overlapping line segments;

[0071] For vertical line segments, if the difference between the horizontal coordinates of two vertical line segments is less than the spacing threshold and the vertical coordinates overlap, the two vertical line segments are determined to be overlapping line segments;

[0072] The longest line segments among the overlapping line segments are retained respectively.

[0073] It should be noted that in this embodiment, by adjusting the parameters, the line segments obtained by Hough transform ignore the existence of the gate, and the effect can be as follows: Figure 4 As shown, it can be seen that although all line segments have been detected, there will be overlapping line segments. In order to solve the problem of overlapping, the line segments are divided into two categories: horizontal line segments and vertical line segments, and a suitable spacing threshold is set according to the size and shape of the map to distinguish horizontal and vertical line segments without missing them. For horizontal line segments, if the difference between the ordinates of two line segments is less than the spacing threshold and there is overlap between the abscissas, they are considered overlapping line segments; and for vertical line segments, if the difference between the abscissas of two line segments is less than the spacing threshold and the ordinates overlap, they are considered overlapping line segments. In this way, the overlapping line segments are screened out, and then the longest line segment among the overlapping line segments is retained, completing the line segment screening.

[0074] In one embodiment, regarding the above step S14, the process of completing the digitization of the map image may further include the following processing steps:

[0075] If a pair of horizontal line segments and vertical line segments are determined to be non-intersecting, and the difference between the ordinate of the horizontal line segment and the abscissa of the vertical line segment is less than the spacing threshold, the line segment with a relatively smaller length is extended so that the horizontal line segment and the vertical line segment intersect.

[0076] It can be understood that in order to prevent the originally intersecting line segments from being disconnected, the ordinates of the horizontal line segments and the abscissas of the vertical line segments are compared. If they are determined to be non-intersecting and the difference between the abscissas and ordinates is within the spacing threshold, they should have originally overlapped. At this time, the shorter line segment is extended to ensure that it intersects with the longer line segment. In this way, relatively accurate line segment coordinates are obtained, and the digitization of the map image is finally completed.

[0077] Next, we will carry out sub-region segmentation and area calculation based on the DCEL data structure. After the digitized map image, we need to cut out the boundaries of each room. Then, we can calculate the length of each room in the map image based on the conversion between the pixel length of the boundary and the actual length (such as the wall length). In this way, we can calculate the area of ​​each room, laying a good foundation for the subsequent balanced distribution of tasks and areas of multiple robots (drones) as much as possible. Therefore, the focus of this stage is how to use the DCEL data structure to represent vertices and edges of the floor plan and each line segment, as well as process and calculate.

[0078] After the walls in the map image are extracted, they are represented by multiple line segments. Which line segments constitute a room and which rooms a corner vertex is related to require the use of appropriate data structures and calculation steps.

[0079] like Figure 5As shown in the figure, DCEL (Doubly Connected Edge List) is a commonly used data structure suitable for representing planar connected graphs embedded in a plane. DCEL is a half-edge-based data structure that connects vertices, edges, and faces together. Given an input spatial network represented by a set of line segments, the DCEL constructor generates and stores each segmented vertex, half-edge, and face.

[0080] Vertices are nodes where two or more line segments intersect, corresponding to graph vertices in spatial networks, such as corners of rooms in a map area where multiple walls intersect. Halfedges are formed by splitting line segments along their length and have a direction component: a start vertex and a destination vertex. Two halfedges of opposite directions (double halfedges) represent each undirected line segment, where the start vertex of the first halfedge is the destination vertex of the second halfedge, and vice versa. Thus, each halfedge corresponds to a directed graph edge (wall) in space.

[0081] According to the directionality of half-edges, the connection between vertices and half-edges can be established: half-edges have a starting vertex and a target vertex, and each half-edge is assigned to its starting vertex, sorted clockwise and added to the half-edge list of the starting vertex. For example Figure 5 In the figure, the half-edges assigned to vertex A are e1, e2, and e3 after clockwise sorting. Since a face is formed by connecting vertices and half-edges with the same direction, to build the connection between half-edges and faces, it is necessary to set the half-edge connection rule. The half-edge connection rule can be as follows: the half-edges of vertex A are e1, e2, and e3 after clockwise sorting. First, let the next half-edge of half-edge e1.twin be e2, and then, by analogy, the next half-edge of half-edge e2.twin is e3, and the next half-edge of half-edge e3.twin is e1. Process the half-edge lists of all vertices according to this connection rule to complete the construction of the DCEL data structure.

[0082] Vertices and half-edges can be directly extracted from the input line segments; the two endpoints of the line segment represent its two vertices, and each line segment (undirected edge) is represented by two half-edges (directed edges). Unlike extracting vertices and half-edges, extracting faces is not simple. In order to generate all the faces of the plane segmentation, the DCEL constructor calls the following specific process, which extracts all closed polygons formed by a set of plane segments.

[0083] The specific process is as follows Figure 6As shown, suppose the vertex set V={v1,v2,…,vn} of the undirected planar graph G=(V,E), the edge set E={e1,e2,…,em}, n and m are both positive integers not less than 2, and the previous and next relations represent the front and back relations of the half-edges respectively. The DCEL data structure of the undirected planar graph G is a linked list composed of half-edges, and the input is the line segment information in the map. Since the Hough transform obtains the longest possible line segments, there are intersections in the middle of the line segments instead of undirected edges. In this embodiment, the intersecting line segments are found, and the longer line segments are divided into small line segments at the intersections to complete the re-representation of the room. At this time, there are no intersections in the middle of the line segments, which represent the undirected edges in the map image. Based on this information, the vertex list and half-edge list of the map image can be created.

[0084] Starting from one half side, passing through the connected half side and returning to the starting point, a surface can be formed. Figure 5 It can be found that the left side of the half-edge passed is the enclosed face. Therefore, after constructing the DCEL data structure, each half-edge is checked. If it is not associated with a face, it is associated with the left plane of the edge and loops to the next edge until it returns to the original edge. Finally, each face is matched with the first associated half-edge, so that all faces are represented and the connection between half-edges and faces is completed.

[0085] After the DCEL data structure is built, given a vertex, you can get the half-edge starting from the vertex, and sort them clockwise. Given a half-edge, you can correspond to the face on the left of the half-edge. Given a face, you can get one of the half-edges that make up the face. In this way, the connection between vertex-half-edge-face is completed.

[0086] Then, by traversing all the half-edges in the DCEL data structure, we can get all the faces and calculate their areas respectively. For a face, we traverse the half-edges that make up the face in the DCEL data structure, calculate the cross product of the adjacent half-edges and sum them, and finally divide by 2 to get the directed area. Finally, we output the area and vertices of each face (that is, room), such as Figure 8 shown. Figure 7 The room area value can correspond well to the area in the map image, so the method of map segmentation and area calculation through DCEL data structure is effective and accurate.

[0087] In practical applications, the drone can sweep the designated unknown space area (also called the mission space area) and use the SLAM algorithm to build the original map. The Canny algorithm in the OpenCV library is used for edge detection and the Hough transform is used to detect the straight lines in the map image. Then, according to the DCEL data structure processing process introduced in the previous article, the DCEL data structure of the map image is established through the straight line information, which can be conveniently segmented, solving the problem of representing and cutting the L-shaped room. Finally, the area and vertex information of each room are obtained to draw the topological map of the current mission environment of the drone, such as Figure 7 and 8 As shown, preparation for multi-robot (drone) task allocation is made. Figure 8 In the above equation, w represents the area of ​​the room in the physical world, and also corresponds to the weight of the corresponding vertex in the topological graph.

[0088] Specifically, after the aerial drone quickly scans a large area to generate the original map, the ground multi-robot system can use the above scheme to divide the map into small rooms, and then assign these rooms to each ground robot, so that the subsequent full coverage search process can be completed as quickly as possible without mutual interference between ground robots. The proposed scheme is a room processing method based on OpenCV edge detection and bidirectional connected edge list (DCEL) data structure. Experimental results verify the effectiveness of the scheme and its important basic role in task allocation for heterogeneous ground robot systems.

[0089] It should be understood that although Figure 1 The steps in the method are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps can be executed in other orders. Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequentially, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0090] In one embodiment, Fig. 9As shown, a room processing system 100 for full coverage search tasks is provided, which may include a mapping module 11, an image enhancement module 13, a data module 15, a structure building module 17, a plane segmentation module 19 and a topology building module 21. Among them, the mapping module 11 is used to sweep the task space area by a drone and use the SLAM algorithm to build an original map. The image enhancement module 13 is used to apply the Canny algorithm in the OpenCV library to perform image enhancement and edge detection on the original map to obtain a clear map image. The data module 15 is used to use the Hough transform to extract line segments from the map image, detect the straight lines in the map image and complete the dataization of the map image. The structure building module 17 is used to establish the DCEL data structure of the map image according to the set half-edge connection rules based on the straight lines of the digitized map image. The plane segmentation module 19 is used to extract the vertices and edges in the map image using the DCEL data structure, and then traverse all the half-edges in the DCEL data structure to obtain all the faces. The topology building module 21 is used to use the vertices, edges and faces to construct the room topology map corresponding to the map image.

[0091] The room processing system 100 for the full coverage search task proposes a room processing method based on OpenCV edge detection and a bidirectional connected edge list (DCEL) data structure, and uses the OpenCV open source computer vision library to perform image processing and room segmentation on the map established by the drone. Among them, Canny edge detection and Hough transform technology are used to segment the room, and the DCEL data structure is used to represent the room, which effectively completes the room segmentation and topology construction under the full coverage search task, and provides structured information for subsequent multi-robot (drone) task allocation and search. Experimental results show that this method performs well in processing various complex room structures (such as concave polygons and polygons with holes).

[0092] In one embodiment, the data module 15, during the process of detecting straight lines in the map image, is further used to: divide the line segments in the map image into two categories: horizontal line segments and vertical line segments, and set a spacing threshold; for horizontal line segments, if the difference between the longitudinal coordinates of two horizontal line segments is less than the spacing threshold and there is overlap between the horizontal coordinates, then the two horizontal line segments are determined to be overlapping line segments; for vertical line segments, if the difference between the horizontal coordinates of two vertical line segments is less than the spacing threshold and the longitudinal coordinates overlap, then the two vertical line segments are determined to be overlapping line segments; and retain the longest line segments in each overlapping line segment.

[0093] In one embodiment, during the process of completing the digitization of the map image, the digitization module 15 is also used to extend a line segment with a relatively small length so that the horizontal line segment and the vertical line segment intersect when it is determined that a pair of horizontal line segments and vertical line segments are non-intersecting and the difference between the longitudinal coordinate of the horizontal line segment and the abscissa of the vertical line segment is less than the spacing threshold.

[0094] It can be understood that for the specific limitations of the room processing system 100 for the full coverage search task, reference can be made to the corresponding limitations of the room processing method for the full coverage search task mentioned above, which will not be repeated here.

[0095] Each module in the room processing system 100 for full coverage search tasks can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a device with data processing function in the form of hardware, or can be stored in the memory of the aforementioned device in the form of software, so that the processor can call and execute the operations corresponding to each module. The aforementioned device can be, but is not limited to, various types of image computing devices available in the art.

[0096] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the following processing steps when executing the computer program: constructing an original map by sweeping a mission space area with an unmanned aerial vehicle and using a SLAM algorithm; performing image enhancement and edge detection on the original map using a Canny algorithm in an OpenCV library to obtain a clear map image; performing line segment extraction on the map image using a Hough transform, detecting straight lines in the map image and completing the digitization of the map image; establishing a DCEL data structure of the map image according to the set half-edge connection rules based on the straight lines in the digitized map image; extracting vertices and edges in the map image using the DCEL data structure, and traversing all half-edges in the DCEL data structure to obtain all faces; and constructing a room topology map corresponding to the map image using vertices, edges, and faces.

[0097] In one embodiment, when the processor executes the computer program, the processor may also implement the steps or sub-steps added in each embodiment of the above-mentioned room processing method for the full coverage search task.

[0098] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following processing steps are implemented: an original map is constructed by sweeping a mission space area by an unmanned aerial vehicle and using a SLAM algorithm; the Canny algorithm in the OpenCV library is applied to perform image enhancement and edge detection on the original map to obtain a clear map image; line segments are extracted from the map image using the Hough transform, straight lines in the map image are detected, and the map image is digitized; a DCEL data structure of the map image is established according to the set half-edge connection rules based on the straight lines in the digitized map image; after extracting vertices and edges in the map image using the DCEL data structure, all half-edges in the DCEL data structure are traversed to obtain all faces; and a room topology map corresponding to the map image is constructed using vertices, edges, and faces.

[0099] In one embodiment, when the computer program is executed by a processor, it can also implement the steps or sub-steps added in the above-mentioned room processing method embodiments for the full coverage search task.

[0100] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus dynamic random access memory (RambusDRAM, referred to as RDRAM) and interface dynamic random access memory (DRDRAM).

[0101] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of protection of the invention. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the attached claims.

Claims

1. A room processing method for a full coverage search task, characterized in that: Includes steps: The original map is constructed by sweeping the mission space area with a drone and using the SLAM algorithm; Apply the Canny algorithm in the OpenCV library to perform image enhancement and edge detection on the original map to obtain a clear map image; Using Hough transform to extract line segments from the map image, detect straight lines in the map image and complete digitization of the map image; According to the straight lines of the digitized map image, the DCEL data structure of the map image is established according to the set half-edge connection rule; the set half-edge connection rule is as follows: the half-edges of vertex A are e1, e2 and e3 after being sorted clockwise, first, the next half-edge of half-edge e1.twin is e2, and then by analogy, the next half-edge of half-edge e2.twin is e3, and the next half-edge of half-edge e3.twin is e1; After extracting vertices and edges in the map image using the DCEL data structure, traverse all half edges in the DCEL data structure to obtain all faces; The room topology map corresponding to the map image is constructed using the vertices, edges and faces.

2. The room processing method for full coverage search task according to claim 1 is characterized in that: The process of detecting the straight line in the map image further includes the steps of: Classifying the line segments in the map image into two categories: horizontal line segments and vertical line segments, and setting a spacing threshold; For a horizontal line segment, if the difference between the ordinates of two horizontal line segments is less than the spacing threshold and the abscissas overlap, the two horizontal line segments are determined to be overlapping line segments; For vertical line segments, if the difference between the horizontal coordinates of two vertical line segments is less than the spacing threshold and the vertical coordinates overlap, the two vertical line segments are determined to be overlapping line segments; The longest line segments among the overlapping line segments are retained respectively.

3. The room processing method for full coverage search task according to claim 2 is characterized in that: The process of completing the digitization of the map image also includes the following steps: If a pair of horizontal line segments and vertical line segments are determined to be non-intersecting, and the difference between the ordinate of the horizontal line segment and the abscissa of the vertical line segment is less than the spacing threshold, the line segment with a relatively smaller length is extended so that the horizontal line segment and the vertical line segment intersect.

4. A room processing system for full coverage search tasks, characterized in that: include: Mapping module, used to sweep the mission space area by drone and build the original map using SLAM algorithm; An image enhancement module is used to apply the Canny algorithm in the OpenCV library to perform image enhancement and edge detection on the original map to obtain a clear map image; A digitization module, used for extracting line segments from the map image using Hough transform, detecting straight lines in the map image and completing digitization of the map image; a structure building module, for building a DCEL data structure of the map image according to the straight lines of the digitized map image and the set half-edge connection rule; the set half-edge connection rule is as follows: the half-edges of vertex A are e1, e2 and e3 after being sorted clockwise, firstly, the next half-edge of half-edge e1.twin is e2, and then by analogy, the next half-edge of half-edge e2.twin is e3, and the next half-edge of half-edge e3.twin is e1; A plane segmentation module, used for extracting vertices and edges in the map image using the DCEL data structure, and then traversing all half edges in the DCEL data structure to obtain all faces; A topology construction module is used to construct a room topology map corresponding to the map image using the vertices, edges and faces.

5. The room processing system for full coverage search tasks according to claim 4 is characterized in that: In the process of detecting the straight line in the map image, the data module is further used to: Classifying the line segments in the map image into two categories: horizontal line segments and vertical line segments, and setting a spacing threshold; For a horizontal line segment, if the difference between the ordinates of two horizontal line segments is less than the spacing threshold and the abscissas overlap, the two horizontal line segments are determined to be overlapping line segments; For vertical line segments, if the difference between the horizontal coordinates of two vertical line segments is less than the spacing threshold and the vertical coordinates overlap, the two vertical line segments are determined to be overlapping line segments; The longest line segments among the overlapping line segments are retained respectively.

6. The room processing system for full coverage search tasks according to claim 5, characterized in that: In the process of completing the digitization of the map image, the digitization module is also used to extend the line segment with a relatively small length so that the horizontal line segment and the vertical line segment intersect when it is determined that a pair of horizontal line segments and vertical line segments are non-intersecting and the difference between the longitudinal coordinate of the horizontal line segment and the abscissa of the vertical line segment is less than the spacing threshold.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the room processing method for a full coverage search task as described in any one of claims 1 to 3 are implemented.

Images