Rescue Mission Population Map Stitching System and Method Based on Custom Descriptors

Abstract

The present invention discloses a rescue mission crowd map stitching system and method based on a custom descriptor in the field of image stitching technology, including: obtaining map data to be stitched; detecting the positions of pedestrians in the map data to be stitched; generating a custom descriptor based on the detected pedestrian positions and a descriptor for the image to be stitched; pairwise matching and detecting the number of descriptors for all images to be stitched; performing transformation estimation stitching and verification on the images in descending order according to the number of matching pairs. In the present invention, the custom descriptor is based on the detected crowd information, can make full use of the crowd information, and can more efficiently perceive the whole picture.

Description

Technical Field

[0001] The invention relates to a rescue mission crowd map splicing system and method based on a custom descriptor, and belongs to the technical field of image splicing. Background Art

[0002] Image stitching is a subtask of computer vision and is often used in satellite image fusion, panoramic image generation and other fields. Situational awareness is an ability to understand security risks dynamically and holistically based on the environment. It is a way to improve the ability to discover, identify, understand, analyze, respond and deal with security threats from a global perspective based on security big data. The ultimate goal is to make decisions and take actions, which is the implementation of security capabilities. Among them, the recognition of the scale of the crowd in an emergency can be regarded as an important part of situational awareness. Specifically, the task is defined as given multiple local crowd pictures to be rescued, we need to perceive the global scale, and we need to count all the rescue tasks and the location of each person to be rescued, so that we can reasonably plan the rescue route and implement the rescue. The given local crowd pictures may have a small overlap, a sparse crowd, and too few map features, which will hinder the overall stitching of the image. For the case of a small number of images, it may be possible to stitch them directly by hand by staff, but humans cannot complete the stitching of a large number of local pictures in a short time. Rescue tasks often have high requirements for speed. In addition, if there are errors in the statistics of the crowd during the stitching process, the location and number of the crowd are missed or counted incorrectly, it will also have serious consequences for the entire rescue work.

[0003] Image stitching technology has been developed for many years and has many mature algorithms applied to practical scenarios. The main method is to generate descriptors (SIFT, SURF, ORB, etc.) for each image and then match and stitch the descriptors. These methods are based on the calculation of each descriptor. Only when the quality of the descriptor is high can the matching be effective. After the match is obtained, RANSAC is used to estimate the pose and synthesize the image.

[0004] However, for our scale recognition task, the feature points of existing methods are automatically generated, which does not fully utilize the key crowd information and cannot efficiently estimate the global population to be rescued. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a rescue mission crowd map splicing system and method based on a custom descriptor. The custom descriptor is based on the detected crowd information, can make full use of the crowd information, and perceive the global situation more efficiently.

[0006] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a method for stitching rescue mission population maps based on custom descriptors, including:

[0008] Obtain the map data to be stitched;

[0009] Detect the positions of pedestrians in the map data to be stitched;

[0010] Generate custom descriptors based on the detected pedestrian positions and descriptors for the images to be stitched;

[0011] For all pairs of images to be stitched, pairwise match and detect the number of descriptors;

[0012] Estimate the transformation for stitching and verification of the images in descending order according to the number of matching pairs.

[0013] Further, generating custom descriptors based on the detected pedestrian positions and descriptors for the images to be stitched includes:

[0014] Generate custom point descriptors based on the detected pedestrian positions;

[0015] Perform Delaunay triangulation on the images according to the detected pedestrian positions and generate custom triangle descriptors;

[0016] Detect straight lines in the image according to the image and generate straight line binary descriptors.

[0017] Further, generating custom point descriptors based on the detected pedestrian positions includes:

[0018] Statistically analyze the information of the neighborhood of the feature points of the pedestrian positions with a fixed multiple of the feature point radius as the threshold;

[0019] Select the point closest to the feature point of the pedestrian position as the main direction, set every d degrees as an interval, and divide the angle into 360 / d intervals;

[0020] Statistically analyze the distances from the feature points of the pedestrian positions to the feature point of the pedestrian position in each of the remaining intervals, divide the distance of each interval by the radius of the point, and generate custom point descriptors with a dimension of 360 / d for each feature point. The general form of the custom point descriptor is:

[0021] des = {s1, s2,, …, s n}(n = 360 / d)

[0022] s i = (∑x aj (j ∈ z i )) / r a (i = 1 ~ n)

[0023] where des is the neighborhood descriptor, n is the number of intervals, and si is a point descriptor, x aj is the distance from the feature point within the interval to this point, r a is the key point radius, and zi is the angle interval corresponding to si.

[0024] Furthermore, the Delaunay triangulation includes:

[0025] Step 1: Find a rectangle that contains the given point set, connect any diagonal of the rectangle to form two triangles, and establish an initial Delaunay triangular mesh;

[0026] Step 2: Insert another point into the Delaunay triangular mesh. Starting from the triangle where this point is located, search for the adjacent triangles of this triangle and perform an empty circumcircle detection. Find all the triangles whose circumcircles contain this point and delete these triangles to form a polygon cavity containing this point as the Delaunay cavity. Connect this point to each vertex of the Delaunay cavity to form a new Delaunay triangular mesh;

[0027] Step 3: In response to when all points in the given point set have been inserted into the new Delaunay triangular mesh, delete all the triangles whose vertices contain the auxiliary window R, otherwise repeat Step 2;

[0028] Step 4: Take the incenter of each new Delaunay triangular mesh as the feature point, define the radius as the average radius of the three endpoints, define the descending order lengths of the three sides to form a feature of dimension 3, divide the length of each side by the feature point radius, and generate a custom triangle descriptor. The general form is as follows:

[0029] des2 = {s1 / r, s2 / r, s3 / r} (s1 ≤ s2 ≤ s3)

[0030] r = (r a + r b + r c ) / 3

[0031] where des2 is the custom triangle descriptor, r is the feature point radius, s1, s2, and s3 are the lengths of each side, r a 、r b and r c are the radii of the three endpoints.

[0032] Furthermore, detect the straight lines in the image according to the image and generate a straight line binary descriptor, including:

[0033] Generate straight lines in the given scale space according to the EDline algorithm. Each line has a direction, calculate the statistical features of the line segment from the support region of the line segment, divide the support region into multiple line bands, and each line band can generate a local descriptor. The descriptor form of the line segment is as follows:

[0034]

[0035] Among them, LBD is the descriptor of the line segment, is the local descriptor;

[0036] Accumulate the gradients of each local line band in all directions:

[0037]

[0038]

[0039] Among them, d L is the straight line direction, d ┴ is the direction orthogonal to the straight line, g’ is the pixel gradient of the local coordinate system of the image, and are the gradient accumulations of the k-th row of the strips in the orthogonal direction, the orthogonal reverse direction, the straight line direction, and the straight line reverse direction respectively. λ = f g (k)f l (k), f g (k) is the global weight coefficient, f l (k) is the local weight coefficient; generate the description matrix of each strip:

[0040]

[0041] Among them, BDM j is the description matrix of each strip, and n is the number of rows of the strip; calculate the mean vector and standard variance of the matrix, and convert the descriptor form of the binary form of the line segment to:

[0042]

[0043] Among them, is the mean vector, is the standard variance.

[0044] Furthermore, for all pairs of pictures to be stitched, detect the number of matching descriptors, including: for each type of descriptor of all given map pictures, perform matching, use the Euclidean distance of the features as the matching criterion, set the ratio of the nearest neighbor to the second nearest neighbor to be less than a certain threshold as an effective match, and count the number of effective matches of each of the three descriptors.

[0045] Furthermore, estimate, stitch, and verify the transformation of the images in descending order according to the number of matching pairs, including:

[0046] Sum up the effective matches of the custom point descriptor and the LBD line descriptor, and sort them in descending order for all map images;

[0047] Perform pose estimation in order, use the RANSAC algorithm to calculate the transformation matrix, and verify the calculation results;

[0048] Use the union-find data structure to store the mutual relationship of the maps. When the verification condition is satisfied, together with two pictures, when all maps form a connected domain or the traversal of map matching pairs ends;

[0049] Select one picture with the largest number of matching pairs as the base picture, transform all effective matching pictures to the coordinate system of the base picture according to the transformation relationship stored in the union-find, and then perform splicing.

[0050] In a second aspect, the present invention provides a rescue mission crowd map splicing system based on a custom descriptor, including:

[0051] Data acquisition module: used to acquire the map data to be spliced;

[0052] Detection module: used to detect the pedestrian positions in the map data to be spliced;

[0053] Descriptor generation module: used to generate custom descriptors based on the detected pedestrian positions and descriptors for the pictures to be spliced;

[0054] Matching module: used to pairwise match and detect the number of descriptors for all pictures to be spliced;

[0055] Splicing verification module: used to perform transformation estimation, splicing and verification on the images in descending order according to the number of matching pairs.

[0056] In a third aspect, the present invention provides a rescue mission crowd map splicing device based on a custom descriptor, including a processor and a storage medium;

[0057] The storage medium is used to store instructions;

[0058] The processor is used to operate according to the instructions to execute the steps of the rescue mission crowd map splicing method based on a custom descriptor according to any one of the above.

[0059] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the rescue mission crowd map splicing method based on a custom descriptor according to any one of the above are implemented.

[0060] Compared with the prior art, the beneficial effects achieved by the present invention:

[0061] 1. The present invention is mainly applied to scale recognition in rescue tasks, fully considering various situations in the map stitching process and the problems encountered by the aforementioned existing algorithms. By generating custom descriptors, the problem of inconsistent descriptors when the overlap degree is small is significantly optimized. The custom descriptors are based on the detected crowd information, can make full use of the crowd information, and can perceive the global situation more efficiently. In addition, the verification part of our stitching method can effectively avoid the occurrence of invalid global results, ensure the validity of the stitched graph, and maximize the rescue effect.

[0062] 2. The present invention uses custom descriptors for feature matching and transformation estimation, has a lower requirement for the overlap degree of local images, and uses the crowd as feature points for the custom descriptors, which can effectively process map data with sparse textures. Multiple descriptors participate in the calculation together, which can effectively ensure the accuracy of map stitching and crowd counting.

[0063] 3. The present invention includes a verification process after stitching. By verifying the positions of the crowd after stitching, it can ensure that incorrect stitching will not be included in the final result and thus affect the global crowd scale recognition. When there are incorrect pictures in the input local pictures, this verification method can exclude the incorrect input, making the stitching process have strong robustness.

[0064] 4. Different from the traditional local feature descriptor method, the present invention needs to generate keys according to pixel values. However, the present invention generates descriptors based on the crowd positions and each descriptor has a lower feature dimension, which can significantly reduce the amount of calculation during the calculation process and ensure the efficient output of the stitching result by the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is the algorithm flowchart provided by Embodiment 1 of the present invention;

[0066] Figure 2 is the schematic diagram of the custom point descriptor provided by Embodiment 1 of the present invention;

[0067] Figure 3 is the schematic diagram of the custom triangle descriptor provided by Embodiment 1 of the present invention;

[0068] Figure 4 is the schematic diagram of the invalid stitching result provided by Embodiment 1 of the present invention;

[0069] Figure 5 is the example diagram of descriptor matching provided by Embodiment 1 of the present invention;

[0070] Figure 6 is the example diagram of comparing with the existing algorithm result provided by Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0071] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and cannot be used to limit the protection scope of the present invention.

[0072] Embodiment 1:

[0073] A method for stitching maps of rescue mission populations based on custom descriptors, including:

[0074] Step S1: Obtain the map data to be stitched;

[0075] Step S2: Detect the positions of pedestrians in the map data to be stitched;

[0076] Step S3: Generate custom descriptors based on the detected pedestrian positions and descriptors for the images to be stitched. The custom descriptors can effectively stitch images with less overlap and are robust to maps with sparse textures, having high accuracy, good application generalization, and environmental requirements;

[0077] Step S4: Pairwise match and detect the number of descriptors for all images to be stitched;

[0078] Step S5: Use the RANSAC algorithm to perform transformation estimation, stitching, and verification on the images in descending order of the number of matching pairs. The verification process can effectively correct the stitching results of images with failed matches, reduce the impact of difficult-to-stitch images on the final stitching result of the overall image. If the verification passes, the stitching result is added; otherwise, the stitching pair is discarded.

[0079] Step S2 includes:

[0080] After obtaining the local map set in step S1, identify the persons to be rescued in each map. Since different types of map forms may be different, the present invention does not specifically specify a certain person detection method. The dataset used takes black dots as the positions of the persons to be rescued.

[0081] Step S3 specifically includes:

[0082] Step S31: Generate custom point descriptors based on the detected pedestrian positions: For each identified pedestrian position feature point, count the information within the neighborhood of the pedestrian position feature point with a fixed multiple s of the feature point radius as the threshold. Select the point closest to this point as the main direction. Divide the angle into 360 / d intervals at every d degrees, and count the distances from the pedestrian position feature points in each interval to this point. To ensure scale invariance, divide the distance in each interval by the radius of this point. In this way, a point descriptor with a dimension of 360 / d is generated for each feature point.

[0083] The steps for generating line feature descriptors are as follows: First, the LSD algorithm is used to detect straight lines in the map. Then, a binary descriptor is generated for each straight line. The descriptor dimension of each straight line is 256. To facilitate matching, the straight line descriptor is converted into a descriptor with the midpoint of the straight line as the feature point. As Figure 2 shown: Set d to 90 degrees. The angle is divided into 4 intervals. Calculate the neighborhood descriptor of point a. Set the key point radius to r a , take s = 20, and count the neighborhood information of the circle with r a *20 as the radius. In this figure, point b is the closest to point a, so ab is selected as the main direction, and the clockwise direction is the positive direction. The sum of the distances from the points in each interval to point a is the eigenvalue of this interval. Thus, the neighborhood information of point a can be characterized as a 4D feature [ab + ac, ad, 0, 0]. To ensure scale invariance, the feature is divided by the radius of point a as a whole. Finally, the feature is [(ab + ac) / ra, ad / ra, 0, 0]. The general form of the custom point descriptor is given below:

[0084] des = {s1, s2,,…, s n}(n = 360 / d)

[0085] s i = (∑x aj (j ∈ z i )) / r a (i = 1~n)

[0086] where des is the neighborhood descriptor, n is the number of intervals, s i is the point descriptor, x aj is the distance from the feature point in the interval to this point, r a is the key point radius, zi is the angle interval corresponding to si. In this task, d = 5 and s = 20 are selected. Finally, each key point can generate a 64D feature descriptor.

[0087] Step S32: Perform Delaunay triangulation on the image according to the detected pedestrian positions and generate custom triangle descriptors: Perform Delaunay triangulation on the entire image, thus generating a graph structure containing nodes and edges. The specific steps of Delaunay triangulation are as follows:

[0088] Establish the initial triangular mesh: For the given point set V, find a rectangle R that contains this point set. We call R the auxiliary window. Connect any diagonal of R to form two triangles as the initial Delaunay triangular mesh.

[0089] Point-by-point insertion: Assume that there is already a Delaunay triangular mesh T. Now, insert a point P into it. It is necessary to find the triangle in which the point P is located. Starting from the triangle where P is located, search for the neighboring triangles of this triangle and perform an empty circumcircle detection. Find all the triangles whose circumcircles contain the point P and delete these triangles to form a polygonal cavity containing P, which we call the Delaunay cavity. Then connect P to each vertex of the Delaunay cavity to form a new Delaunay triangular mesh.

[0090] Delete the auxiliary window R: Repeat step 2). After all the points in the point set V have been inserted into the triangular mesh, delete all the triangles whose vertices contain the auxiliary window R.

[0091] Through the above steps, it is ensured that the triangular structures generated from the same graph are the same. The generated graph structure is as Figure 3 shown. The circled parts of the two graphs are the overlapping parts. Since they overlap at the edge, the point descriptors generated in step S31 cannot be matched due to different neighborhood information. However, after triangulation, it can be made to have a consistent triangular structure (as shown in the circled part), thus completing the matching and splicing. We take the incenter of each triangle as the feature point, and the radius is defined as the average radius of the three endpoints. Define the descending lengths of the three sides to form a feature of dimension 3. To ensure scale invariance, the length of each side is divided by the feature point radius. Its general form is as follows:

[0092] des2 = {s1 / r, s2 / r, s3 / r} (s1 ≤ s2 ≤ s3)

[0093] r = (r a + r b + r c ) / 3

[0094] In this way, the feature dimension of each triangular descriptor is 3.

[0095] Step S33: Detect the straight lines in the image according to the image and generate straight line binary descriptors: Here, the built-in line feature descriptors in opencv are used to generate the results. The specific process is as follows: First, use the EDline algorithm to generate straight lines in the given scale space. Each line has a direction. Calculate the statistical features of the line segment from the support domain of the line segment. Divide the support domain into m line bands, and each line band Bj can generate a local descriptor (BD). The descriptor form of this line segment is as follows:

[0096]

[0097] Among them, LBD is the descriptor of the line segment, is the local descriptor;

[0098] Accumulate the gradients of each local line band in all directions:

[0099]

[0100]

[0101] where d L is the straight line direction, d ┴ is the direction orthogonal to the straight line, g’ is the pixel gradient in the local coordinate system of the image, and are the gradient accumulations of the k-th row of the bands in the orthogonal direction, the opposite orthogonal direction, the straight line direction, and the opposite straight line direction respectively. λ = f g (k)f l (k), f g (k) is the global weight coefficient, f l (k) is the local weight coefficient. We can generate the description matrix (BDM) of each band from this:

[0102]

[0103] where n is the number of rows of the band. Calculate the mean vector M and the standard variance S of this matrix, and the LBD form can be converted to:

[0104]

[0105] where, is the mean vector, is the standard variance. Convert it to binary form. We set m = 32, each band is an 8Bit string, and concatenate 32 strings to get the final 256bit LBD line feature descriptor. In order to perform pose estimation together with the aforementioned custom descriptor, convert the LBD descriptor to the form of a point, and take the end point of the line segment as the key point, and this key point contains a descriptor with a dimension of 256.

[0106] Step S4: For each given map picture, match each descriptor, use the Euclidean distance of the features as the matching criterion, set the ratio of the nearest neighbor to the second nearest neighbor to be less than a certain threshold as an effective match, and count the number of effective matches of each of the three descriptors. The dimensions of the three descriptors of the present invention vary greatly. The relative threshold for the point descriptor and the binary line descriptor is set to 0.7. Since the dimension of the triangular descriptor is small, the threshold requirement is set to 0.4, and it only participates in the final transformation estimation and does not participate in the image matching sorting.

[0107] Step S5 specifically includes:

[0108] According to the previous step, the number of valid matches of different descriptors for each image can be obtained. Since the dimension of the triangle descriptor is relatively small and false matches are more likely to occur, we take the sum of the valid matches of the custom point descriptor and the LBD line descriptor and sort them in descending order for all map images. The more valid matches there are, the more overlapping parts there are and the easier it is to match. Then, pose estimation is performed in order, the transformation matrix is calculated using the RANSAC algorithm, and the calculation results are verified. Since the RANSAC algorithm can effectively remove the influence of outliers, even if there are false matches in the given matching pairs, it will not have a great impact on the generation of the transformation matrix. The union-find data structure is used to store the mutual relationships of the maps. If the verification conditions are met, the two images are combined. The whole process continues until all maps form a connected domain or the map matching pairs are traversed. The entire process can be represented by the following pseudocode:

[0109]

[0110] Among them, estimate is to estimate the transformation matrix of the image pair (i, j), and check(res) is to verify the validity of the stitching result. Only if it is valid will the transformation matrix be stored.

[0111] The method for verifying the validity of the stitching result is as follows: Given two images and their respective key points (human positions), the first image is transformed to the coordinate system of the second image according to the transformation matrix obtained by RANSAC, and calculate the Figure 1 nearest distance from each key point to the Figure 2 key point. If the error is less than the key point radius, it is considered a valid match. For two images, it is required that the intersection over union ratio of their key points is greater than 90% for a correct estimate. Only if it is a correct estimate will they be stitched, otherwise the pair of matches is invalid. As Figure 4 shown, if the stitching result is not verified, irregular stitching is likely to occur. This verification method can avoid the occurrence of irregular stitching. In addition, if there are incorrect images in the input, they will not be incorporated into the final result because they do not meet the validity requirements, ensuring the robustness of the algorithm of the present invention.

[0112] Select an image with the largest number of matching pairs as the base image. Since the transformation matrix is transitive, all valid matching images are transformed to the coordinate system of the base image according to the transformation relationship stored in the union-find, and then stitched. In this way, if n local maps are given, at most n - 1 stitches are required to generate the final result, effectively ensuring the efficiency of this algorithm. As Figure 6 shown, given the same input local map set, comparing the stitching results of the opencv encapsulated stitching class Stitcher, the present invention can efficiently and quickly give accurate stitching results, and its stitching effect is significantly better than the traditional method.

[0113] Example 2:

[0114] The rescue mission crowd map stitching system based on custom descriptors can implement the rescue mission crowd map stitching method based on custom descriptors described in Example 1, including:

[0115] Data acquisition module: used to acquire the map data to be stitched;

[0116] Detection module: used to detect the pedestrian positions in the map data to be stitched;

[0117] Descriptor generation module: used to generate custom descriptors based on the detected pedestrian positions and generate descriptors for the images to be stitched;

[0118] Matching module: used to pairwise match all the images to be stitched and detect the number of descriptors;

[0119] Stitching verification module: used to perform transformation estimation stitching and verification on the images in descending order of the number of matching pairs.

[0120] Example 3:

[0121] The embodiment of the present invention also provides a rescue mission crowd map stitching device based on custom descriptors, which can implement the rescue mission crowd map stitching method based on custom descriptors described in Example 1, including a processor and a storage medium;

[0122] The storage medium is used to store instructions;

[0123] The processor is used to operate according to the instructions to execute the steps of the following method:

[0124] Acquire the map data to be stitched;

[0125] Detect the pedestrian positions in the map data to be stitched;

[0126] Generate custom descriptors based on the detected pedestrian positions and generate descriptors for the images to be stitched;

[0127] Pairwise match all the images to be stitched and detect the number of descriptors;

[0128] Perform transformation estimation stitching and verification on the images in descending order of the number of matching pairs.

[0129] Example 4:

[0130] The embodiment of the present invention also provides a computer-readable storage medium, which can implement the rescue mission crowd map stitching method based on custom descriptors described in Example 1. A computer program is stored thereon, and when the program is executed by a processor, the steps of the following method are implemented:

[0131] Acquire the map data to be stitched;

[0132] Detect the pedestrian positions in the map data to be stitched;

[0133] Generate a custom descriptor based on the detected pedestrian positions and a descriptor for the image to be stitched;

[0134] For all pairs of images to be stitched, detect the number of matching descriptors;

[0135] Estimate the transformation for stitching and verification of the images in descending order according to the number of matching pairs.

[0136] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or multiple flows and / or blocks

[0138] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one or more of the flows Figure 1 or multiple flows and / or blocks

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide means for implementing the functions specified in Figure 1 one or more of the flows Figure 1Steps of functions specified in one or more boxes.

[0140] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for stitching maps of rescue mission populations based on custom descriptors, characterized in that, including: Obtain the map data to be stitched; Detect the pedestrian positions in the map data to be stitched; Generate a custom descriptor based on the detected pedestrian positions and a descriptor for the image to be stitched; For all pairs of images to be stitched, pairwise match and detect the number of descriptors; Estimate the transformation, stitch, and verify the images in descending order according to the number of matching pairs; Generate a custom descriptor based on the detected pedestrian positions and a descriptor for the image to be stitched, including: Generate a custom point descriptor based on the detected pedestrian positions; Perform Delaunay triangulation on the image according to the detected pedestrian positions and generate a custom triangle descriptor; Detect lines in the image and generate a line binary descriptor; Generate a custom point descriptor based on the detected pedestrian positions, including: Statistically analyze the information of the neighborhood of the feature points of the pedestrian positions with a fixed multiple of the feature point radius as the threshold; Select the point closest to the feature point of the pedestrian position as the main direction, set every d degrees as an interval, and divide the angle into 360 / d intervals; Statistically analyze the distances from the feature points of the pedestrian positions to the feature point of the pedestrian position in each of the remaining intervals, divide the distance of each interval by the radius of the point, and generate a custom point descriptor with a dimension of 360 / d for each feature point. The general form of the custom point descriptor is: des = {s1, s2, …, s n}(n = 360 / d) s i = (∑x aj (j ∈ z i )) / r a (i = 1 to n) Among them, des is the neighborhood descriptor, n is the number of intervals, s i is the point descriptor, x aj is the distance from the feature point in the interval to this point, r a is the key point radius, and zi is the angle interval corresponding to si.

2. The method for stitching rescue mission population maps based on custom descriptors according to claim 1, characterized in that, Delaunay triangulation includes: Step 1: Find a rectangle that contains the given point set, connect any diagonal of the rectangle to form two triangles, and establish an initial Delaunay triangular mesh; Step 2: Insert another point into the Delaunay triangular mesh. Starting from the triangle where the point is located, search for the neighboring triangles of this triangle and perform an empty circumcircle detection. Find all the triangles whose circumcircles contain this point and delete these triangles to form a polygon cavity containing this point as the Delaunay cavity. Connect this point to each vertex of the Delaunay cavity to form a new Delaunay triangular mesh; Step 3: In response to all points in the given point set having been inserted into the new Delaunay triangular mesh, delete all the triangles whose vertices contain the auxiliary window R. Otherwise, repeat Step 2; Step 4: Take the incenter of each new Delaunay triangular mesh as the feature point, define the radius as the average radius of the three endpoints, define the descending order lengths of the three sides as features with a dimension of 3, divide the length of each side by the radius of the feature point, and generate a custom triangle descriptor. The general form is as follows: des2 = {s1 / r, s2 / r, s3 / r} (s1 ≤ s2 ≤ s3) r＝(r a +r b +r c ) / 3 Among them, des2 is a custom triangle descriptor, r is the radius of the feature point, s1, s2, and s3 are the lengths of each side, and r a , r b , and r c are the radii of the three endpoints.

3. The method for stitching rescue mission population maps based on custom descriptors according to claim 1, characterized in that, Detect lines in the image and generate a line binary descriptor, including: Generate lines in the given scale space according to the EDline algorithm. Each line has a direction. Calculate the statistical features of the line segment from the support region of the line segment. Divide the support region into multiple line bands, and each line band can generate a local descriptor. The descriptor form of the line segment is as follows: Among them, LBD is the descriptor of the line segment, which is the local descriptor; Accumulate the gradients of each local line band in all directions: where d L is the straight line direction, d ┴ is the direction orthogonal to the straight line, g’ is the pixel gradient in the local coordinate system of the image, and are the gradient accumulations of the k-th row of the stripe in the orthogonal direction, the opposite orthogonal direction, the straight line direction, and the opposite straight line direction respectively. λ = f g (k)f l (k), f g (k) is the global weight coefficient, and f l (k) is the local weight coefficient; generate the description matrix for each stripe: Among them, BDM j is the description matrix of each strip, and n is the number of rows of the strip; calculate the mean vector and standard variance of the matrix, and the descriptor form of the line segment in binary form is converted as follows: Among them, is the mean vector, is the standard deviation.

4. The method for stitching rescue mission population maps based on custom descriptors according to claim 1, characterized in that For all pairs of images to be stitched, detect the number of matching descriptors, including: for each type of descriptor in all given map images, perform matching, use the Euclidean distance of features as the matching criterion, set the ratio of the nearest neighbor to the second nearest neighbor to be less than a certain threshold as a valid match, and count the number of valid matches for each of the three descriptors.

5. The method for stitching rescue mission population maps based on custom descriptors according to claim 1, characterized in that Estimate the transformation, stitch, and verify the images in descending order according to the number of matching pairs, including: Sort all map images in descending order according to the sum of the valid matches of the custom point descriptor and the LBD line descriptor; Perform pose estimation in order, use the RANSAC algorithm to calculate the transformation matrix, and verify the calculation results; Use the union-find data structure to store the mutual relationships of the maps. When the verification conditions are met, together with two images, when all maps form a connected domain or the traversal of map matching pairs ends; Select an image with the largest number of matching pairs as the base image, transform all valid matching images to the coordinate system of the base image according to the transformation relationship stored in the union-find, and then perform stitching.

6. The rescue mission crowd map stitching system based on a custom descriptor is characterized in that Including: Data acquisition module: used to acquire the map data to be stitched; Detection module: used to detect the pedestrian positions in the map data to be stitched; Descriptor generation module: used to generate custom descriptors based on the detected pedestrian positions and descriptors for the images to be stitched; Matching module: used to detect the number of descriptors for pairwise matching of all images to be stitched; Stitching verification module: used to estimate the transformation, stitch, and verify the images in descending order according to the number of matching pairs; Generate custom descriptors based on the detected pedestrian positions and descriptors for the images to be stitched, including: Generate custom point descriptors based on the detected pedestrian positions; Perform Delaunay triangulation on the image according to the detected pedestrian positions and generate custom triangle descriptors; Detect the straight lines in the image according to the image and generate straight line binary descriptors; Generate custom point descriptors based on the detected pedestrian positions, including: Statistically analyze the information of the neighborhood of the feature points of the pedestrian positions with a fixed multiple of the feature point radius as the threshold; Select the point closest to the feature point of the pedestrian position as the main direction, set every d degrees as an interval, and divide the angle into 360 / d intervals; Statistically analyze the distances from the feature points of the pedestrian positions in each of the remaining intervals to the feature point of the pedestrian position, divide the distance of each interval by the radius of the point, and generate custom point descriptors with a dimension of 360 / d for each feature point. The general form of the custom point descriptor is: des = {s1, s2, …, s n}(n = 360 / d) s i = (∑x aj (j ∈ z i )) / r a (i = 1 to n) where des is the neighborhood descriptor, n is the number of intervals, s i is the point descriptor, x aj is the distance from the feature point in the interval to this point, r a is the key point radius, and zi is the angle interval corresponding to si.

7. The rescue mission crowd map splicing device based on a custom descriptor, characterized in that, Including a processor and a storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the method for stitching rescue mission crowd maps based on custom descriptors according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method for stitching rescue mission crowd maps based on custom descriptors according to any one of claims 1 to 5.

Images