Closed loop detection method, electronic equipment and storage medium

Abstract

To provide a closed loop detection method, electronic equipment and a storage medium that improve accuracy and robustness of closed-loop detection.SOLUTION: A method comprises: acquiring a first after-processing image through semantic segmentation and polar coordinate conversion on a detected panoramic image; extracting the centers of gravity of respective illustration image titles of the first after-processing image as semantic nodes; forming several semantic node pairs based upon similarities between respective first sematic nodes of the first after-processing image and respective second sematic nodes of a second after-processing image; determining the semantic node pair having the highest similarity as a center semantic node pair; rotating and arraying the first after-processing image and second after-processing image on the basis of the center semantic node pair; and then determining a closed-loop detection result on the basis of the relative position relation between the first semantic nodes and second semantic nodes of the respective semantic node pairs.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the technical field of autonomous driving, and in particular to a closed-loop detection method, an electronic device, and a storage medium. [Background technology]

[0002] In visual simultaneous localization and mapping (SLAM), in order to build an accurate map and achieve accurate self-localization, it is necessary to correct the self-localization error by detecting the closed loop.

[0003] In the prior art, closed-loop detection methods first extract feature points and descriptors in an image, and then perform closed-loop detection by matching these feature points. The extraction and matching of feature points may cause errors due to factors such as changes in viewing angle, noise or interference, which will affect the accuracy and robustness of closed-loop detection. Summary of the Invention

[0004] The embodiments of the present invention aim to provide a closed-loop detection method, electronic device and storage medium to solve the problem in the prior art that errors occur when processing closed-loop detection under different viewing angles at the same position, affecting the accuracy and robustness of closed-loop detection. In order to solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0005] According to one aspect of the present invention, a method for detecting a panoramic image includes the steps of: performing semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image; extracting a centroid of each illustrative image tile in the first processed image as a semantic node, the illustrative image tiles being obtained by dividing the first processed image based on a semantic category of each pixel; forming several semantic node pairs by performing similarity-based pairing between each first semantic node included in the first processed image and each second semantic node included in the second processed image, the second processed image being obtained by performing semantic segmentation and polar coordinate transformation on a historical panoramic image; selecting, from each of the semantic node pairs, a semantic node pair in which a first semantic node and a second semantic node have the highest similarity as a central semantic node pair; rotating and aligning the first processed image and the second processed image based on the central semantic node pair, and determining a closed-loop detection result based on a relative positional relationship between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the second processed image; A closed loop detection method is provided, including:

[0006] The step of extracting the centroid of each illustration image tile in the first processed image as a semantic node includes: Dividing the first processed image into a number of illustrative image tiles based on the semantic category of each pixel; filtering each of the illustrative image tiles to select static illustrative image tiles whose semantic categories belong to static semantic categories; and extracting the centroid of each said static illustration image tile as a semantic node.

[0007] The method comprises: calculating the number of pixels contained in each of said static illustration image tiles; The method may further include extracting, as a semantic node, a center of gravity of each of the static illustration image tiles when the pixel count reaches a preset pixel count threshold.

[0008] The step of forming several semantic node pairs by performing similarity-based pairing of each first semantic node included in the first processed image and each second semantic node included in the second processed image, includes: Identifying a common semantic category set to which each of the first semantic nodes and each of the second semantic nodes correspond; and The method may further include a step of selecting a second semantic node from the second processed image that has the same semantic category as the first semantic node and is most similar to the first semantic node based on each of the first semantic nodes corresponding to each of the semantic categories in the same semantic category set, and configuring the first semantic node and the most similar second semantic node as a semantic node pair.

[0009] The step of selecting a second semantic node having the same semantic category and being most similar to the first semantic node from the second processed image includes: selecting from the second processed image a number of second semantic nodes having the same semantic category as the first semantic nodes; calculating a first distance from the first semantic node to a center of the first processed image and a second distance from each selected second semantic node to a center of the second processed image; The method may further include determining that a second semantic node corresponding to the second distance that is closest to the first distance is the most similar second semantic node.

[0010] Prior to the step of extracting the centroid of each illustration image tile in the first processed image as a semantic node, The method may further include calculating a similarity between a set of semantic categories corresponding to the first processed image and a set of semantic categories corresponding to the second processed image, If the similarity is smaller than a preset similarity threshold, it is determined that the detected panoramic image and the history panoramic image do not form a closed loop; If the similarity is greater than or equal to the similarity threshold, the method proceeds to extracting the center of gravity of each illustrative image tile in the first processed image as a semantic node.

[0011] The step of determining a closed-loop detection result based on a relative positional relationship between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the aligned second processed image includes: calculating an average value of Euclidean distances between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the aligned second processed image, If the average value of the Euclidean distance is greater than a preset distance threshold, the detected panoramic image and the history panoramic image do not form a closed loop; When the average value of the Euclidean distance is equal to or less than the distance threshold value, the detected panoramic image and the history panoramic image form a closed loop.

[0012] The step of determining a closed-loop detection result based on a relative positional relationship between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the aligned second processed image includes: The method may further include a step of calculating an average value of an included angle between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the aligned second processed image and a centroid of the first processed image / the second processed image, wherein a vertex of the included angle is the centroid; If the average value of the included angle is greater than a preset angle threshold, the detected panoramic image and the history panoramic image do not form a closed loop; When the average value of the included angle is equal to or less than the angle threshold value, the detected panoramic image and the history panoramic image form a closed loop.

[0013] According to another aspect of the present invention, there is provided an electronic device comprising a memory, a processor and a computer program stored and executed in the memory, the electronic device implementing the steps of the closed loop detection method described in any one of the above claims when the processor executes the program.

[0014] According to yet another aspect of the present invention, there is provided a computer readable storage medium having a computer program stored thereon, the computer program being capable of causing a processor to perform the steps of the closed loop detection method according to any one of the preceding claims when executed by the processor. Effect of the Invention

[0015] The beneficial effects of the embodiment of the present invention are as follows: Different from the prior art, the embodiment of the present invention provides a closed-loop detection method, which first performs semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image, extracts the centroid of each illustrated image tile in the first processed image as a semantic node, then performs similarity-based pairing on the first semantic node included in the first processed image and the second semantic node included in the second processed image to form several semantic node pairs, selects the semantic node pair in which the first semantic node and the second semantic node have the highest similarity from the several semantic node pairs as the central semantic node pair, and finally rotates and aligns the first processed image and the second processed image based on the central semantic node pair, and determines the closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node in each semantic node pair in the aligned first processed image and the second processed image. By adopting the method of the present invention, the panoramic image is converted into a polar coordinate representation to make the image rotation invariant; at the same time, features are extracted based on semantic segmentation; and the first processed image and the second processed image are rotated to the same viewing angle based on the extracted features, and then the similarity of the features is compared. This effectively solves the problem of errors occurring in closed-loop detection in the prior art when processing under different viewing angles at the same position, and improves the accuracy and robustness of closed-loop detection. [Brief description of the drawings]

[0016] One or more embodiments are illustratively illustrated by photographs in the accompanying drawings corresponding thereto, but this illustrative illustration is not intended to be limiting of the embodiments, elements in the accompanying drawings having the same reference numerals are similar elements, and unless otherwise specified, the figures in the accompanying drawings are not intended to be drawn to scale. [Figure 1] FIG. 2 is a schematic diagram of a flow chart of a closed-loop detection method according to an embodiment of the present invention; [Diagram 2]2 is a schematic diagram of a panoramic image and its polar coordinate display according to an embodiment of the present invention; [Diagram 3] 2 is a schematic diagram of a panoramic mask image and its polar coordinate representation according to an embodiment of the present invention; [Figure 4] FIG. 4 is a schematic diagram of a flow chart of another closed-loop detection method according to an embodiment of the present invention. [Diagram 5] FIG. 1 is a schematic diagram illustrating a configuration of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] In order to make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be described below clearly and completely with reference to the accompanying drawings of the embodiments of the present invention, but it is obvious that the described embodiments are only some of the embodiments of the present invention and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative labor belong to the scope of protection of the present invention.

[0018] In addition, the technical features according to various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0019] It should be noted that the steps shown in the flowcharts of the accompanying drawings may be performed, for example, on a set of computer systems capable of executing computer instructions, and that although a logical order is shown in the flowcharts, in some cases the steps may be performed in an order different from that shown or described.

[0020] An embodiment of the present invention provides a closed-loop detection method, and an execution subject of the closed-loop detection method may be a closed-loop detection device, for example, the closed-loop detection method may be executed by a terminal device, a server, or other electronic device, where the terminal device may be a mobile robot, an in-vehicle device, a user equipment (UE), a portable device, a user terminal, a terminal, a mobile phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, or a wearable device, etc.

[0021] Example 1 FIG. 1 is a flow chart of a closed-loop detection method according to an embodiment of the present invention. Referring to FIG. 1, the method includes the following steps:

[0022] Step S101: perform semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image. In one embodiment, a VSLAM (Visual SLAM, Visual Navigation and Mapping) system uses a visual sensor such as a camera or a webcam to obtain a detected panoramic image, and inputs the detected panoramic image into a deep learning model for semantic segmentation, and the deep learning model identifies and classifies the semantic category of each pixel in the detected panoramic image to obtain a detected mask image. The mask of each pixel in the detected mask image is the semantic category label of the corresponding pixel, and the semantic category label is an ID value corresponding to the semantic category. For example, the semantic category of a pixel corresponding to a vehicle pattern in the detected panoramic image is "vehicle", and the semantic category label is "10". In semantic segmentation, it is not necessary to separately segment different objects belonging to the same category.

[0023] Traditional feature point and feature vector methods often provide local feature descriptions, while the present invention uses panoramic images as input to extract global feature representations, which can better capture the context and overall information of the environment. By performing semantic segmentation on the panoramic images and using semantic segmentation features to measure the similarity of images, the algorithm is not only easier for humans to understand, but also helps to eliminate occlusions and distinguish between static and dynamic objects, and helps to understand the semantics of the environment, improving the accuracy and reliability of closed-loop detection.

[0024] Furthermore, the detected mask image is transformed into a polar coordinate system to obtain a first processed image. By transforming the panoramic image into a polar coordinate system, the image can be made rotationally invariant.

[0025] In another embodiment, the detected panoramic image may first be converted into polar coordinates to obtain a detected polar coordinate image, and then semantic segmentation may be performed on the detected polar coordinate image to identify and classify the semantic category of each pixel in the detected polar coordinate image to obtain a first processed image.

[0026] 2 and 3 are schematic diagrams of a panoramic image and its polar coordinate display according to an embodiment of the present invention, and a panoramic mask image and its polar coordinate display, respectively.

[0027] Step S102: Extract the center of gravity of each illustration image tile in the first processed image as a semantic node. In order to facilitate the extraction of information such as the shape and position of an object, the first processed image is divided into several illustrated image tiles based on the semantic category of each pixel. Each divided illustrated image tile is composed of several pixels that have the same semantic category and are adjacent in position. It should be understood that each semantic category can correspond to one or more illustrated image tiles in one image. As shown in FIG. 3, the illustrated image tile includes an illustrated image tile 1 that is a "building", an illustrated image tile 2 that is a "car", an illustrated image tile 3 that is a "road", and an illustrated image tile 4 that is a "road marking". Among them, the illustrated image tile 2 belongs to a dynamic semantic category, and the illustrated image tiles 1, 3, and 4 belong to a static semantic category.

[0028] Since illustrative image tiles of dynamic semantic categories are likely to interfere with closed-loop detection, in order to improve the accuracy of closed-loop detection, it is preferable to sort each illustrative image tile during closed-loop detection, select static illustrative image tiles whose semantic categories belong to static semantic categories, and discard illustrative image tiles whose semantic categories belong to dynamic semantic categories (e.g., vehicles, pedestrians, etc.).

[0029] Furthermore, in order to improve algorithm efficiency, illustrative image tiles with a relatively small number of pixels may be filtered out according to a set threshold, specifically, the number of pixels contained in each static illustrative image tile is calculated, and the static illustrative image tiles whose pixel count does not reach a preset pixel count threshold are discarded.

[0030] By filtering the illustrated image tiles based on dynamic / static semantic categories and pixel counts, some static illustrated image tiles whose pixel counts reach a pixel count threshold are obtained, and the centroids of these static illustrated image tiles are extracted as semantic nodes. Each semantic node represents one illustrated image tile in the detected panoramic image, and each semantic node corresponds to one semantic category.

[0031] Step S103: Perform similarity-based pairing between each first semantic node included in the first processed image and each second semantic node included in the second processed image to form several semantic node pairs. Here, the second processed image is obtained by performing semantic segmentation and polar coordinate conversion on the historical panoramic image. Then, the second processed image is divided into several illustrated image tiles by the same processing method as the first processed image, and the illustrated image tiles are sorted and filtered, and the centroids of each remaining static illustrated image tile are extracted as semantic nodes. The historical panoramic image is a panoramic image acquired before the VSLAM system acquires the detected panoramic image. When performing closed-loop detection, information such as the second processed image, semantic category, and semantic node obtained by processing the historical panoramic image can be stored. For convenience of explanation, each semantic node included in the first processed image is called a "first semantic node", and each semantic node included in the second processed image is called a "second semantic node".

[0032] The distance of the semantic node from the center of the polar coordinate image can represent the position of the illustrated image tile corresponding to the semantic node. The present invention compares the similarity of two images by extracting the position of each semantic node in the image as a feature. Before comparing the similarity of the two images, each first semantic node included in the first processed image and each second semantic node included in the second processed image are paired based on similarity to form several semantic node pairs.

[0033] Generally, there is no similarity between semantic nodes belonging to different semantic categories. In some embodiments, step S103 includes first identifying a semantic category to which each first semantic node and each second semantic node commonly correspond to configure a same semantic category set, and then selecting a second semantic node having the same and most similar semantic category to the first semantic node from the second processed image based on each first semantic node corresponding to each semantic category in the same semantic category set, and configuring a semantic node pair with the first semantic node and the most similar second semantic node. For example, the semantic categories corresponding to each first semantic node include "vegetation", "garbage can", and "building", and the semantic categories corresponding to each second semantic node include "vegetation" and "building". Here, "vegetation" corresponds to the first semantic nodes A, B, C, and D in the first processed image, and corresponds to the second semantic nodes E, F, and G in the second processed image. For any of the first semantic nodes A, B, C, and D, the most similar one of the second semantic nodes E, F, and G is selected to form a semantic node pair.

[0034] In one embodiment of the present invention, the step of selecting the second semantic node having the same semantic category and the most similar to the first semantic node from the second processed image includes the steps of: selecting some second semantic nodes having the same semantic category as the first semantic node from the second processed image; calculating a first distance from the first semantic node to the center of the first processed image and a second distance from each selected second semantic node to the center of the second processed image; and determining the second semantic node corresponding to the second distance closest to the first distance as the most similar second semantic node. For example, in the above example, the distances of the first semantic nodes A, B, C, and D from the center of the first processed image are 2 cm, 3 cm, 4 cm, and 5 cm, respectively, and the distances of the second semantic nodes E, F, and G from the center of the second processed image are 4 cm, 2 cm, and 7 cm, respectively. The most similar second semantic node to the first semantic node A is F, that is, the semantic node pair {A, F} is formed. Based on the above method, the final constructed semantic node pairs include {A, F}, {B, E}, {B, F}, {C, E} and {D, E}.

[0035] Step S104: From each of the semantic node pairs, select a semantic node pair in which a first semantic node and a second semantic node have the highest similarity as a central semantic node pair. In some embodiments, from each semantic node pair, a semantic node pair having a minimum difference between the distance from the first semantic node to the center of the first processed image and the distance from the second semantic node to the center of the second processed image is selected as the central semantic node pair. In the above example, the distance differences of the semantic node pairs {A, F} and {C, E} are both 0, and one of them can be selected as the central semantic node pair.

[0036] Step S105: Rotate and align the first processed image and the second processed image based on the central semantic node pair, and determine a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node contained in each of the semantic node pairs in the aligned first processed image and the second processed image. Rotating and aligning the first processed image and the second processed image based on the central semantic node pair means superimposing the first processed image and the second processed image based on the central semantic node pair. Since the central semantic node pair is the most similar feature of the first processed image and the second processed image, by rotating and aligning, i.e., rotating the first processed image and the second processed image to the same viewing angle, the problem that errors occur when processing at different viewing angles at the same position in closed-loop detection in the prior art is solved.

[0037] In some embodiments, the step of determining a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node included in each semantic node pair in the aligned first processed image and the second processed image includes a step of calculating an average value of Euclidean distances between the first semantic node and the second semantic node included in each semantic node pair in the aligned first processed image and the second processed image, and if the average value of the Euclidean distances is greater than a predetermined distance threshold, it is determined that the detected panoramic image and the historical panoramic image do not form a closed loop, and if the average value of the Euclidean distances is less than or equal to the distance threshold, the detected panoramic image and the historical panoramic image form a closed loop.

[0038] In another embodiment, the step of determining a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node included in each semantic node pair in the aligned first processed image and the second processed image includes a step of calculating an average value of an included angle between the first semantic node and the second semantic node included in each semantic node pair in the aligned first processed image and the second processed image and the centroid of the first processed image / second processed image, where the apex of the included angle is the centroid, and if the average value of the included angle is greater than a predetermined angle threshold, it is determined that the detected panoramic image and the historical panoramic image do not form a closed loop, and if the average value of the included angle is less than or equal to the angle threshold, the detected panoramic image and the historical panoramic image form a closed loop.

[0039] In order to reduce errors and improve the accuracy of closed-loop detection, before calculating the average value of the Euclidean distance or the average value of the included angle, the method further includes selecting each semantic node pair to ensure that each first semantic node corresponds to only one semantic node pair for the average value calculation. For example, in the above example, the first semantic node B corresponds to two semantic node pairs {B, E} and {B, F}, and one of them needs to be selected for the average value calculation and the other needs to be discarded. Specifically, when a first semantic node corresponds to a plurality of semantic node pairs, the Euclidean distance between the first semantic node and the second semantic node is calculated for each of the plurality of semantic node pairs, or the included angle between the first semantic node, the second semantic node, and the centroid of the first processed image / second processed image is calculated, and the semantic node pair with the smallest Euclidean distance or the smallest included angle is selected and used to calculate the average Euclidean distance or the average included angle.

[0040] FIG. 4 is a schematic flow chart of another closed-loop detection method according to an embodiment of the present invention. Referring to FIG. 4, the method includes the following steps.

[0041] Step S401: perform semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image.

[0042] Step S402: Calculate a similarity between the semantic category set corresponding to the first processed image and the semantic category set corresponding to the second processed image. The second processed image is obtained by performing semantic segmentation and polar coordinate transformation on the historical panoramic image. In one embodiment, Jaccard similarity is used as the index. Specifically, the intersection and union of the semantic category set corresponding to the first processed image and the semantic category set corresponding to the second processed image are calculated, and the number of semantic categories in the intersection is divided by the number of semantic categories in the union to obtain the similarity. For example, the semantic category set corresponding to the first processed image is {"road", "architecture", "vegetation"}, the semantic category set corresponding to the second processed image is {"car", "trash can", "vegetation"}, the number of semantic categories in the union is 5, and the number of semantic categories in the intersection is 1, so the similarity is 1 / 5. The semantic category set referred to here is the original semantic category set (including dynamic semantic categories) obtained by performing semantic segmentation on the detected panoramic image / history panoramic image.

[0043] Step S403: It is determined whether the similarity is smaller than a preset similarity threshold value. If YES, the process proceeds to step S404, and if NO, the process proceeds to step S405.

[0044] Step S404: It is determined that the detected panoramic image and the history panoramic image do not form a closed loop.

[0045] Step S405: Divide the first processed image into several illustrated image tiles according to the semantic category of each pixel, and extract the centroid of each illustrated image tile as a semantic node.

[0046] Step S406: Perform similarity-based pairing between each first semantic node included in the first processed image and each second semantic node included in the second processed image to form several semantic node pairs.

[0047] Step S407: From each of the semantic node pairs, select a semantic node pair in which the first semantic node and the second semantic node have the highest similarity as a central semantic node pair.

[0048] Step S408: Rotate and align the first processed image and the second processed image based on the central semantic node pair, and determine a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node contained in each of the semantic node pairs in the aligned first processed image and the second processed image.

[0049] The closed-loop detection method according to the embodiment shown in Figure 4 adds a semantic category similarity comparison to the closed-loop detection method of Figure 1, and directly determines that the detected panoramic image and the historical panoramic image do not constitute a closed loop if the overall semantic categories of the first processed image and the second processed image are significantly different, thereby improving the efficiency of closed-loop detection.

[0050] The closed-loop detection method according to an embodiment of the present invention first performs semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image, extracts the center of gravity of each illustration image tile in the first processed image as a semantic node, then performs similarity-based pairing on the first semantic node contained in the first processed image and the second semantic node contained in the second processed image to form several semantic node pairs, and from the several semantic node pairs, selects the semantic node pair between the first semantic node and the second semantic node that have the highest similarity as the central semantic node pair, and finally rotates and aligns the first processed image and the second processed image based on this central semantic node pair, and determines a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node in each semantic node pair in the aligned first processed image and second processed image. By adopting the method of the present invention, the panoramic image is converted into a polar coordinate representation to make the image rotation invariant; at the same time, features are extracted based on semantic segmentation; and the first processed image and the second processed image are rotated to the same viewing angle based on the extracted features, and then the similarity of the features is compared. This effectively solves the problem of errors occurring in closed-loop detection in the prior art when processing under different viewing angles at the same position, and improves the accuracy and robustness of closed-loop detection.

[0051] Example 2 According to an embodiment of the present invention, an electronic device is provided, and FIG. 5 is a schematic diagram of the configuration of the electronic device according to the embodiment of the present invention. As shown in FIG. 5, the electronic device may include a processor 501, a communication interface 502, a memory 503, and a communication bus 504, and the processor 501, the communication interface 502, and the memory 503 may be capable of communicating with each other via the communication bus 504. The processor 501 can call logic instructions in the memory 503 to execute the closed-loop detection method in the first embodiment, and the method includes the steps of: performing semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image; extracting the centroid of each illustrative image tile in the first processed image as a semantic node, the illustrative image tile being obtained by dividing the first processed image based on the semantic category of each pixel; and performing similarity-based pairing of each first semantic node included in the first processed image and each second semantic node included in the second processed image to form several semantic node pairs. The method includes the steps of: obtaining the second processed image by performing semantic segmentation and polar coordinate transformation on the historical panoramic image; selecting a semantic node pair in which a first semantic node and a second semantic node have the highest similarity from each of the semantic node pairs as a central semantic node pair; rotating and aligning the first processed image and the second processed image based on the central semantic node pair, and determining a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node included in each of the semantic node pairs in the aligned first processed image and the second processed image.

[0052] In addition, the logic instructions in the memory 503 may be realized in the form of a software functional unit and stored in some computer-readable storage media when sold or used as an independent product. Based on this understanding, the essence of the technical solution of the present invention or a part of the contribution to the prior art or a part of this technical solution is realized in the form of a software product, and the computer software product includes some instructions for making a computer computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of any of the steps of the method described in the first embodiment of the present invention, and is stored in a storage medium. The aforementioned storage medium includes media capable of storing various program codes, such as a USB flash drive, a portable HDD, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0053] The product can implement any of the closed-loop detection methods described in Example 1, and has corresponding functional modules and beneficial effects of the method. The technical details are not described in detail in this embodiment, but reference can be made to the closed-loop detection method according to Example 1 of the present invention.

[0054] Example 3 According to an embodiment of the present invention, a computer-readable storage medium is provided, the type of which is as described in Example 2, and a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the processor performs the steps of the closed-loop detection method described in Example 1.

[0055] From the above description of the embodiments, those skilled in the art can easily understand that each embodiment can be realized in a manner of combining software with a general-purpose hardware platform, and can also be realized by hardware. Based on this understanding, the essence of the above technical solution or a part of the contribution to the related art can be realized in the form of a software product, and the computer software product includes some instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to execute the method described in each embodiment or a part of the embodiment, and can be stored in a computer-readable storage medium such as a ROM / RAM, a magnetic disk, an optical disk, etc.

[0056] Finally, the above embodiments are only used to describe the technical solution of the present invention, and are not limited thereto, and in the spirit of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented according to any order, and there are many other variations in the different aspects of the present invention as described above, but for the sake of brevity, they are not described in detail. Although the present invention has been described in detail with reference to the above embodiments, it should be understood that those skilled in the art may amend the technical solutions described in the above embodiments or make equivalent replacements for some of the technical features, and these amendments or replacements do not cause the essence of the corresponding technical solution to deviate from the scope of the technical solution of each embodiment of the present application.

Claims

1. performing semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image; extracting a centroid of each illustrative image tile in the first processed image as a semantic node, the illustrative image tiles being obtained by dividing the first processed image based on a semantic category of each pixel; forming several semantic node pairs by performing similarity-based pairing between each first semantic node included in the first processed image and each second semantic node included in a second processed image, the second processed image being obtained by performing semantic segmentation and polar coordinate transformation on a historical panoramic image; selecting, from each of the semantic node pairs, a semantic node pair in which a first semantic node and a second semantic node have the highest similarity as a center semantic node pair; and rotating and aligning the first processed image and the second processed image based on the central semantic node pair, and determining a closed-loop detection result based on a relative positional relationship between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the second processed image. A closed loop detection method comprising:

2. The extraction of the centroid of each illustration image tile in the first processed image as a semantic node includes: Dividing the first processed image into a number of illustrative image tiles based on a semantic category of each pixel; filtering each of said illustrative image tiles to select static illustrative image tiles whose semantic categories belong to static semantic categories; and extracting a centroid of each said static illustrative image tile as a semantic node.

3. calculating a number of pixels contained in each of said static illustration image tiles; extracting a centroid of each of the static illustration image tiles as a semantic node when the pixel count of the static illustration image tiles reaches a preset pixel count threshold.

3. The method of claim 2 .

4. forming several semantic node pairs by performing similarity-based pairing of each first semantic node included in the first processed image and each second semantic node included in the second processed image; Identifying a common semantic category set to which each of the first semantic nodes and each of the second semantic nodes correspond; and 2. The method of claim 1, further comprising: selecting a second semantic node from the second processed image that has the same and most similar semantic category as the first semantic node based on each of the first semantic nodes corresponding to each of the semantic categories in the same semantic category set; and forming a semantic node pair between the first semantic node and the most similar second semantic node.

5. Selecting a second semantic node from the second processed image that has the same semantic category as and is most similar to the first semantic node includes: selecting from the second processed image a number of second semantic nodes having the same semantic category as the first semantic nodes; calculating a first distance from the first semantic node to a center of the first processed image and a second distance from each selected second semantic node to a center of the second processed image; and determining a second semantic node corresponding to the second distance that is closest to the first distance as the most similar second semantic node.

6. Prior to extracting the centroids of each illustrative image tile in the first processed image as semantic nodes, calculating a similarity between a set of semantic categories corresponding to the first processed image and a set of semantic categories corresponding to the second processed image; If the similarity is smaller than a preset similarity threshold, it is determined that the detected panoramic image and the history panoramic image do not form a closed loop; 2. The method of claim 1, further comprising: if the similarity is greater than or equal to the similarity threshold, proceeding to extract a centroid of each illustrative image tile in the first processed image as a semantic node.

7. Determining a closed-loop detection result based on a relative positional relationship between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the aligned second processed image includes: calculating an average value of Euclidean distances between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the aligned second processed image; determining that the detected panoramic image and the history panoramic image do not form a closed loop when the average value of the Euclidean distance is greater than a preset distance threshold; 2. The method of claim 1, further comprising: determining that the detected panoramic image and the historical panoramic image form a closed loop if the average Euclidean distance is less than or equal to the distance threshold.

8. Determining a closed-loop detection result based on a relative positional relationship between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the aligned second processed image includes: Calculating an average value of an included angle between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the second processed image and a centroid of the first processed image / the second processed image, wherein the vertex of the included angle is the centroid; determining that the detected panoramic image and the history panoramic image do not form a closed loop when the average value of the included angle is greater than a preset angle threshold; 2. The method of claim 1, further comprising: determining that the detected panoramic image and the historical panoramic image form a closed loop if the average value of the included angle is less than or equal to the angle threshold.

9. 9. An electronic device comprising a memory, a processor, and a computer program stored and executed in the memory, the electronic device implementing the steps of the closed loop detection method according to any one of claims 1 to 8 when the processor executes the computer program.

10. 9. A computer-readable storage medium having a computer program stored thereon, the computer program being configured to cause a processor to execute steps of a closed-loop detection method according to any one of claims 1 to 8 when the processor is executed.

Images