Closed-loop detection method, electronic device and storage medium

Abstract

To provide a closed loop detection method, electronic equipment and a storage medium that improve accuracy and robustness of closed-loop detection.SOLUTION: A method comprises: acquiring a first after-processing image through semantic segmentation and polar coordinate conversion on a detected panoramic image; extracting the centers of gravity of respective illustration image titles of the first after-processing image as semantic nodes; forming several semantic node pairs based upon similarities between respective first sematic nodes of the first after-processing image and respective second sematic nodes of a second after-processing image; determining the semantic node pair having the highest similarity as a center semantic node pair; rotating and arraying the first after-processing image and second after-processing image on the basis of the center semantic node pair; and then determining a closed-loop detection result on the basis of the relative position relation between the first semantic nodes and second semantic nodes of the respective semantic node pairs.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the technical field of autonomous driving, and in particular to a closed-loop detection method, an electronic device, and a storage medium. [Background technology]

[0002] In visual simultaneous localization and mapping (SLAM), in order to build an accurate map and achieve accurate self-localization, it is necessary to correct the self-localization error by detecting the closed loop.

[0003] In the prior art, closed-loop detection methods first extract feature points and descriptors from an image, and then perform closed-loop detection by matching these feature points. Errors may occur in the feature point extraction and matching due to factors such as changes in the viewing angle, noise, or interference, which affect the accuracy and robustness of closed-loop detection. Summary of the Invention

[0004] The embodiments of the present invention aim to provide a closed-loop detection method, electronic device, and storage medium to solve the problem in the prior art that errors occur when processing closed-loop detection under different viewing angles at the same position, affecting the accuracy and robustness of closed-loop detection. To solve the above technical problems, the embodiments of the present invention provide the following technical solutions.

[0005] According to one aspect of the present invention, a method for detecting a panoramic image includes the steps of: performing semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image; extracting a centroid of each illustrative image tile in the first processed image as a semantic node, the illustrative image tile being obtained by dividing the first processed image based on the semantic category of each pixel; forming several semantic node pairs by performing similarity-based pairing between each first semantic node included in the first processed image and each second semantic node included in the second processed image, wherein the second processed image is obtained by performing semantic segmentation and polar coordinate transformation on a historical panoramic image; selecting, from each of the semantic node pairs, a semantic node pair in which a first semantic node and a second semantic node have the highest similarity as a central semantic node pair; rotating and aligning the first processed image and the second processed image based on the central semantic node pair, and determining a closed-loop detection result based on a relative positional relationship between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the second processed image; A closed-loop detection method is provided, including:

[0006] The step of extracting the centroid of each illustration image tile in the first processed image as a semantic node includes: Dividing the first processed image into several illustrative image tiles based on the semantic category of each pixel; filtering each of the illustrative image tiles to select static illustrative image tiles whose semantic categories belong to static semantic categories; and extracting the centroid of each of the static illustration image tiles as a semantic node.

[0007] The method comprises: calculating the number of pixels contained in each of said static illustration image tiles; The method may further include extracting, as a semantic node, the centroid of each of the static illustration image tiles whose pixel count reaches a preset pixel count threshold.

[0008] The step of forming several semantic node pairs by performing similarity-based pairing of each first semantic node included in the first processed image with each second semantic node included in the second processed image includes: Identifying semantic categories to which each of the first semantic nodes and each of the second semantic nodes commonly correspond, and forming the same set of semantic categories; The method may further include a step of selecting, based on each of the first semantic nodes corresponding to each of the semantic categories in the same semantic category set, a second semantic node that has the same semantic category as the first semantic node and is most similar to the first semantic node from the second processed image, and forming a semantic node pair between the first semantic node and the most similar second semantic node.

[0009] The step of selecting a second semantic node that has the same semantic category as and is most similar to the first semantic node from the second processed image includes: selecting from the second processed image a number of second semantic nodes that have the same semantic category as the first semantic nodes; calculating a first distance from the first semantic node to the center of the first processed image and a second distance from each selected second semantic node to the center of the second processed image; The method may further include determining the second semantic node corresponding to the second distance that is closest to the first distance as the most similar second semantic node.

[0010] Before the step of extracting the centroid of each illustration image tile in the first processed image as a semantic node, The method may further include calculating a similarity between a set of semantic categories corresponding to the first processed image and a set of semantic categories corresponding to the second processed image, If the similarity is smaller than a predetermined similarity threshold, it is determined that the detected panoramic image and the history panoramic image do not form a closed loop; If the similarity is equal to or greater than the similarity threshold, the method proceeds to extracting the centroid of each illustration image tile in the first processed image as a semantic node.

[0011] The step of determining a closed-loop detection result based on a relative positional relationship between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the aligned second processed image includes: The method may further include a step of calculating an average value of Euclidean distances between a first semantic node and a second semantic node included in each semantic node pair in the aligned first processed image and the aligned second processed image, If the average value of the Euclidean distance is greater than a preset distance threshold, the detected panoramic image and the history panoramic image do not form a closed loop; If the average value of the Euclidean distance is equal to or less than the distance threshold, the detected panoramic image and the history panoramic image form a closed loop.

[0012] The step of determining a closed-loop detection result based on a relative positional relationship between a first semantic node and a second semantic node included in each of the semantic node pairs in the aligned first processed image and the aligned second processed image includes: The method may further include a step of calculating an average value of an included angle between a first semantic node and a second semantic node included in each semantic node pair in the aligned first processed image and the aligned second processed image and a centroid of the first processed image / the second processed image, wherein the vertex of the included angle is the centroid; If the average value of the included angle is greater than a preset angle threshold, the detected panoramic image and the history panoramic image do not form a closed loop; If the average value of the included angle is equal to or less than the angle threshold value, the detected panoramic image and the history panoramic image form a closed loop.

[0013] According to another aspect of the present invention, there is provided an electronic device comprising a memory, a processor, and a computer program stored in and executed by the memory, the electronic device implementing the steps of the closed-loop detection method described in any one of the above claims when the processor executes the program.

[0014] According to yet another aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, the computer program being capable of causing a processor to perform the steps of the closed-loop detection method described in any one of the preceding claims when executed by the processor. [Effects of the Invention]

[0015] The advantageous effects of the embodiments of the present invention are as follows: Unlike the prior art, the embodiments of the present invention provide a closed-loop detection method that first performs semantic segmentation and polar coordinate transformation on a detected panoramic image to obtain a first processed image, extracts the centroid of each illustration image tile in the first processed image as a semantic node, then performs similarity-based pairing on a first semantic node included in the first processed image and a second semantic node included in a second processed image to form several semantic node pairs, selects from the several semantic node pairs a semantic node pair between which the first semantic node and the second semantic node have the highest similarity as a central semantic node pair, and finally rotates and aligns the first processed image and the second processed image based on the central semantic node pair, and determines a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node in each semantic node pair in the aligned first processed image and second processed image. By adopting the method of the present invention, the panoramic image is converted into a polar coordinate representation, thereby making the image rotation invariant; at the same time, features are extracted based on semantic segmentation; and the first processed image and the second processed image are rotated to the same viewing angle based on the extracted features, and then the similarity of the features is compared. This effectively solves the problem of errors occurring in closed-loop detection in the prior art when processing under different viewing angles at the same position, and improves the accuracy and robustness of closed-loop detection. [Brief explanation of the drawings]

[0016] One or more embodiments are illustratively illustrated by photographs in the accompanying drawings corresponding thereto, but this illustrative illustration is not intended to be limiting of the embodiments, elements in the accompanying drawings having the same reference numerals are similar elements, and unless otherwise specified, the figures in the accompanying drawings are not intended to be limited to scale. [Figure 1] 2 is a schematic diagram of a flow chart of a closed-loop detection method according to an embodiment of the present invention; [Figure 2]1 is a schematic diagram of a panoramic image and its polar coordinate representation according to an embodiment of the present invention; [Figure 3] 1 is a schematic diagram of a panoramic mask image and its polar coordinate representation according to an embodiment of the present invention; [Figure 4] FIG. 4 is a schematic diagram of a flow chart of another closed-loop detection method according to an embodiment of the present invention. [Figure 5] 1 is a schematic diagram illustrating a configuration of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] In order to clarify the objectives, technical solutions and advantages of the embodiments of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention, but it is clear that the described embodiments are only some of the embodiments of the present invention and do not represent all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without any creative work fall within the scope of protection of the present invention.

[0018] Furthermore, the technical features according to the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0019] It should be noted that the steps shown in the flowcharts of the accompanying drawings may be performed, for example, on a set of computer systems capable of executing computer instructions, and that although a logical order is shown in the flowcharts, in some cases the steps may be performed in an order different from that shown or described.

[0020] An embodiment of the present invention provides a closed-loop detection method, and an execution entity of the closed-loop detection method may be a closed-loop detection device, for example, the closed-loop detection method may be executed by a terminal device, a server, or other electronic device, where the terminal device may be a mobile robot, an in-vehicle device, a user equipment (UE), a portable device, a user terminal, a terminal, a mobile phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, or a wearable device, etc.

[0021] Example 1 FIG. 1 is a flowchart of a closed-loop detection method according to an embodiment of the present invention. Referring to FIG. 1, the method includes the following steps:

[0022] Step S101: perform semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image. In one embodiment, a VSLAM (Visual SLAM, Visual Navigation and Mapping) system uses a visual sensor, such as a camera or a webcam, to acquire a target panoramic image. The target panoramic image is then input to a deep learning model for semantic segmentation. The deep learning model then identifies and classifies the semantic category of each pixel in the target panoramic image to obtain a target mask image. The mask of each pixel in the target mask image is the semantic category label of the corresponding pixel, and the semantic category label is an ID value corresponding to the semantic category. For example, the semantic category of a pixel corresponding to a vehicle pattern in the target panoramic image is "vehicle," and the semantic category label is "10." Semantic segmentation does not require separate segmentation of different objects belonging to the same category.

[0023] While traditional methods using feature points and feature vectors often provide local feature descriptions, the present invention uses panoramic images as input to extract global feature representations, which can better capture the context and overall information of the environment. By performing semantic segmentation on panoramic images and using semantic segmentation features to measure image similarity, the algorithm not only becomes more human-understandable, but also helps eliminate occlusions and distinguish between static and dynamic objects, and supports semantic understanding of the environment, improving the accuracy and reliability of closed-loop detection.

[0024] Furthermore, the detected mask image is subjected to polar coordinate transformation to obtain a first processed image. By transforming the panoramic image into a polar coordinate representation, the image can be made rotationally invariant.

[0025] In another embodiment, the detected panoramic image may first be polar coordinate converted to obtain a detected polar coordinate image, and then semantic segmentation may be performed on the detected polar coordinate image to identify and classify the semantic category of each pixel in the detected polar coordinate image to obtain a first processed image.

[0026] 2 and 3 are schematic diagrams of a panoramic image and its polar coordinate display, and a panoramic mask image and its polar coordinate display, respectively, according to an embodiment of the present invention.

[0027] Step S102: Extract the center of gravity of each illustration image tile in the first processed image as a semantic node. To facilitate extraction of information such as the shape and position of an object, the first processed image is divided into several illustrated image tiles based on the semantic category of each pixel. Each divided illustrated image tile is composed of several pixels that have the same semantic category and are adjacently located. It should be understood that each semantic category can correspond to one or more illustrated image tiles within a single image. As shown in FIG. 3 , the illustrated image tiles include illustrated image tile 1, which is a "building," illustrated image tile 2, which is a "car," illustrated image tile 3, which is a "road," and illustrated image tile 4, which is a "road marking." Among these, illustrated image tile 2 belongs to the dynamic semantic category, and illustrated image tiles 1, 3, and 4 belong to the static semantic category.

[0028] Since illustration image tiles of dynamic semantic categories are likely to interfere with closed-loop detection, in order to improve the accuracy of closed-loop detection, it is preferable to sort each illustration image tile during closed-loop detection, select static illustration image tiles whose semantic categories belong to static semantic categories, and discard illustration image tiles whose semantic categories belong to dynamic semantic categories (e.g., vehicles, pedestrians, etc.).

[0029] Furthermore, to improve algorithm efficiency, illustrative image tiles with a relatively small number of pixels may be filtered out according to a set threshold value, specifically, by calculating the number of pixels contained in each static illustrative image tile and discarding static illustrative image tiles whose pixel count does not reach a preset pixel count threshold value.

[0030] By filtering the illustration image tiles based on dynamic / static semantic categories and pixel counts, several static illustration image tiles whose pixel counts reach a pixel count threshold are obtained, and the centroids of these static illustration image tiles are extracted as semantic nodes. Each semantic node represents one illustration image tile in the detected panoramic image, and each semantic node corresponds to one semantic category.

[0031] Step S103: Pairing each first semantic node included in the first processed image with each second semantic node included in the second processed image based on similarity to form several semantic node pairs. Here, the second processed image is obtained by performing semantic segmentation and polar coordinate transformation on the historical panoramic image. Then, using the same processing method as for the first processed image, the second processed image is divided into several illustration image tiles, and after sorting and filtering the illustration image tiles, the centroids of each remaining static illustration image tile are extracted as semantic nodes. The historical panoramic image is a panoramic image acquired before the VSLAM system acquires the panoramic image to be detected. When performing closed-loop detection, information such as the second processed image, semantic categories, and semantic nodes obtained by processing the historical panoramic image can be stored. For convenience of explanation, each semantic node included in the first processed image is referred to as a "first semantic node," and each semantic node included in the second processed image is referred to as a "second semantic node."

[0032] The distance of a semantic node from the center of the polar coordinate image can represent the position of the corresponding semantic node within the illustrated image tile. The present invention compares the similarity of two images by extracting the position of each semantic node within the image as a feature. Before comparing the similarity of the two images, several semantic node pairs are formed by pairing each first semantic node included in the first processed image with each second semantic node included in the second processed image based on similarity.

[0033] Generally, there is no similarity between semantic nodes belonging to different semantic categories. In some embodiments, step S103 includes first identifying a semantic category to which each first semantic node and each second semantic node commonly correspond to form a set of the same semantic categories; then, based on each first semantic node corresponding to each semantic category in the set of the same semantic categories, selecting a second semantic node from the second processed image that has the same and most similar semantic category to the first semantic node, and forming a semantic node pair between the first semantic node and the most similar second semantic node. For example, semantic categories corresponding to each first semantic node include "vegetation," "trash can," and "building," and semantic categories corresponding to each second semantic node include "vegetation" and "building." Here, "vegetation" corresponds to the first semantic nodes A, B, C, and D in the first processed image, and to the second semantic nodes E, F, and G in the second processed image. For any of the first semantic nodes A, B, C, and D, the most similar one of the second semantic nodes E, F, and G is selected to form a semantic node pair.

[0034] In one embodiment of the present invention, the step of selecting from the second processed image a second semantic node that has the same semantic category as the first semantic node and is most similar to the first semantic node includes the steps of: selecting from the second processed image several second semantic nodes that have the same semantic category as the first semantic node; calculating a first distance from the first semantic node to the center of the first processed image and a second distance from each selected second semantic node to the center of the second processed image; and determining the second semantic node corresponding to the second distance that is closest to the first distance as the most similar second semantic node. For example, in the above example, the distances of the first semantic nodes A, B, C, and D from the center of the first processed image are 2 cm, 3 cm, 4 cm, and 5 cm, respectively, and the distances of the second semantic nodes E, F, and G from the center of the second processed image are 4 cm, 2 cm, and 7 cm, respectively. The most similar second semantic node to the first semantic node A is F, that is, the semantic node pair {A, F} is formed. Based on the above method, the finally constructed semantic node pairs include {A, F}, {B, E}, {B, F}, {C, E} and {D, E}.

[0035] Step S104: From each of the semantic node pairs, select a semantic node pair in which the first semantic node and the second semantic node have the highest similarity as a central semantic node pair. In some embodiments, from each semantic node pair, a semantic node pair having a minimum difference between the distance from the first semantic node to the center of the first processed image and the distance from the second semantic node to the center of the second processed image is selected as the central semantic node pair. In the above example, the distance difference between the semantic node pairs {A, F} and {C, E} is 0, and either one of them can be selected as the central semantic node pair.

[0036] Step S105: The first processed image and the second processed image are rotated and aligned based on the central semantic node pair, and a closed-loop detection result is determined based on the relative positional relationship between the first semantic node and the second semantic node included in each of the semantic node pairs in the aligned first processed image and the second processed image. Rotating and aligning the first processed image and the second processed image based on the central semantic node pair means superimposing the first processed image and the second processed image based on the central semantic node pair. Since the central semantic node pair is the most similar feature of the first processed image and the second processed image, rotating and aligning them, i.e., rotating the first processed image and the second processed image to the same viewing angle, solves the problem in the prior art of errors occurring when processing at different viewing angles at the same position in closed-loop detection.

[0037] In some embodiments, the step of determining a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node included in each semantic node pair in the aligned first processed image and the second processed image includes a step of calculating the average value of the Euclidean distance between the first semantic node and the second semantic node included in each semantic node pair in the aligned first processed image and the second processed image, and if the average value of the Euclidean distance is greater than a predetermined distance threshold, it is determined that the detected panoramic image and the historical panoramic image do not form a closed loop, and if the average value of the Euclidean distance is less than or equal to the distance threshold, the detected panoramic image and the historical panoramic image form a closed loop.

[0038] In another embodiment, the step of determining a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node included in each semantic node pair in the aligned first processed image and the second processed image includes a step of calculating the average value of the included angle between the first semantic node and the second semantic node included in each semantic node pair in the aligned first processed image and the second processed image and the centroid of the first processed image / second processed image, where the vertex of the included angle is the centroid, and if the average value of the included angle is greater than a predetermined angle threshold, it is determined that the detected panoramic image and the historical panoramic image do not form a closed loop, and if the average value of the included angle is less than or equal to the angle threshold, the detected panoramic image and the historical panoramic image form a closed loop.

[0039] In order to reduce errors and improve the accuracy of closed-loop detection, before calculating the average value of the Euclidean distance or the average value of the included angle, the method further includes selecting each semantic node pair to be used in the calculation of the average value, so as to ensure that each first semantic node corresponds to only one semantic node pair. For example, in the above example, the first semantic node B corresponds to two semantic node pairs {B, E} and {B, F}, and one of them needs to be selected for the calculation of the average value, and the other needs to be discarded. Specifically, when a first semantic node corresponds to a plurality of semantic node pairs, the Euclidean distance between the first semantic node and the second semantic node is calculated for each of the plurality of semantic node pairs, or the included angle between the first semantic node, the second semantic node, and the centroid of the first processed image / second processed image is calculated, and the semantic node pair with the smallest Euclidean distance or the smallest included angle is selected and used to calculate the average value of the Euclidean distance or the average value of the included angle.

[0040] FIG. 4 is a schematic flow chart of another closed-loop detection method according to an embodiment of the present invention. Referring to FIG. 4, the method includes the following steps:

[0041] Step S401: Perform semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image.

[0042] Step S402: Calculate the similarity between the set of semantic categories corresponding to the first processed image and the set of semantic categories corresponding to the second processed image. The second processed image is obtained by performing semantic segmentation and polar coordinate transformation on the historical panoramic image. In one embodiment, Jaccard similarity is used as the metric. Specifically, the intersection and union of the semantic category set corresponding to the first processed image and the semantic category set corresponding to the second processed image are calculated, and the similarity is obtained by dividing the number of semantic categories in the intersection by the number of semantic categories in the union. For example, if the semantic category set corresponding to the first processed image is {"road," "architecture," "vegetation"} and the semantic category set corresponding to the second processed image is {"car," "trash can," "vegetation"}, the number of semantic categories in the union is 5, and the number of semantic categories in the intersection is 1, resulting in a similarity of 1 / 5. The semantic category set referred to here is the original semantic category set (including dynamic semantic categories) obtained by performing semantic segmentation on the detected panoramic image / historical panoramic image.

[0043] Step S403: It is determined whether the similarity is smaller than a preset similarity threshold value. If YES, the process proceeds to step S404, and if NO, the process proceeds to step S405.

[0044] Step S404: It is determined that the detected panoramic image and the history panoramic image do not form a closed loop.

[0045] Step S405: Divide the first processed image into several illustration image tiles based on the semantic category of each pixel, and extract the centroid of each illustration image tile as a semantic node.

[0046] Step S406: Perform similarity-based pairing between each first semantic node included in the first processed image and each second semantic node included in the second processed image to form several semantic node pairs.

[0047] Step S407: From each of the semantic node pairs, select the semantic node pair in which the first semantic node and the second semantic node have the highest similarity as the central semantic node pair.

[0048] Step S408: The first processed image and the second processed image are rotated and aligned based on the central semantic node pair, and a closed-loop detection result is determined based on the relative positional relationship between the first semantic node and the second semantic node included in each of the semantic node pairs in the aligned first processed image and the second processed image.

[0049] The closed-loop detection method according to the embodiment shown in Figure 4 adds a semantic category similarity comparison to the closed-loop detection method according to Figure 1, and if the overall semantic categories of the first processed image and the second processed image are significantly different, it is directly determined that the detected panoramic image and the history panoramic image do not form a closed loop, thereby improving the efficiency of closed-loop detection.

[0050] A closed-loop detection method according to an embodiment of the present invention first performs semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image, extracts the center of gravity of each illustration image tile in the first processed image as a semantic node, then performs similarity-based pairing on the first semantic node contained in the first processed image and the second semantic node contained in the second processed image to form several semantic node pairs, and from the several semantic node pairs, selects the semantic node pair between the first semantic node and the second semantic node with the highest similarity as the central semantic node pair, and finally rotates and aligns the first processed image and the second processed image based on this central semantic node pair, and determines a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node in each semantic node pair in the aligned first processed image and second processed image. By adopting the method of the present invention, the panoramic image is converted into a polar coordinate representation, thereby making the image rotation invariant; at the same time, features are extracted based on semantic segmentation; and the first processed image and the second processed image are rotated to the same viewing angle based on the extracted features, and then the similarity of the features is compared. This effectively solves the problem of errors occurring in closed-loop detection in the prior art when processing under different viewing angles at the same position, and improves the accuracy and robustness of closed-loop detection.

[0051] Example 2 According to an embodiment of the present invention, an electronic device is provided. Fig. 5 is a schematic diagram of the configuration of the electronic device according to the embodiment of the present invention. As shown in Fig. 5, the electronic device may include a processor 501, a communication interface 502, a memory 503, and a communication bus 504, and the processor 501, the communication interface 502, and the memory 503 may be able to communicate with each other via the communication bus 504. The processor 501 can call logic instructions in the memory 503 to execute the closed-loop detection method in the first embodiment, which includes the steps of: performing semantic segmentation and polar coordinate transformation on a detected panoramic image to obtain a first processed image; extracting the centroid of each illustrative image tile in the first processed image as a semantic node, where the illustrative image tile is obtained by dividing the first processed image based on the semantic category of each pixel; and performing similarity-based pairing between each first semantic node included in the first processed image and each second semantic node included in a second processed image to form several semantic node pairs. The method includes the steps of: obtaining the second processed image by performing semantic segmentation and polar coordinate transformation on the historical panoramic image; selecting a semantic node pair in which a first semantic node and a second semantic node have the highest similarity from each of the semantic node pairs and setting it as a central semantic node pair; rotating and aligning the first processed image and the second processed image based on the central semantic node pair, and determining a closed-loop detection result based on the relative positional relationship between the first semantic node and the second semantic node included in each of the semantic node pairs in the aligned first processed image and the second processed image.

[0052] Furthermore, the logic instructions in the memory 503 may be implemented in the form of a software functional unit and stored in several computer-readable storage media when sold or used as an independent product. Based on this understanding, the essence of the technical solution of the present invention or a part that contributes to the prior art or a part of the technical solution is realized in the form of a software product, and the computer software product includes several instructions that cause a computer device (which may be a personal computer, a server, a network device, etc.) to execute all or part of any of the steps of any of the methods described in Example 1 of the present invention, and is stored in a storage medium. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a portable HDD, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0053] The product can implement any of the closed-loop detection methods described in Example 1, and has corresponding functional modules and beneficial effects of the method. The technical details are not described in detail in this embodiment, but reference can be made to the closed-loop detection method according to Example 1 of the present invention.

[0054] Example 3 According to an embodiment of the present invention, there is provided a computer-readable storage medium, the type of which is as described in Example 2, and having a computer program stored thereon, the computer program being configured to cause a processor to execute the steps of the closed-loop detection method described in Example 1 when the computer program is executed by the processor.

[0055] From the above description of the embodiments, those skilled in the art will understand that each embodiment can be realized by combining software with a general-purpose hardware platform, and can also be realized by hardware. Based on this understanding, the essence of the above technical solutions or a part contributing to the related art can be realized in the form of a software product, and the computer software product includes several instructions that cause a computer device (which may be a personal computer, a server, a network device, etc.) to execute the method described in each embodiment or a certain part of the embodiment, and can be stored in a computer-readable storage medium such as a ROM / RAM, a magnetic disk, an optical disk, etc.

[0056] Finally, the above embodiments are only used to explain the technical solutions of the present invention, and are not intended to limit the same. Within the spirit of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations in the different aspects of the present invention as described above, which are not described in detail for the sake of brevity. Although the present invention has been described in detail with reference to the above embodiments, it should be understood that those skilled in the art may amend the technical solutions described in the above embodiments or make equivalent substitutions for some of the technical features, and these amendments or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. performing semantic segmentation and polar coordinate transformation on the detected panoramic image to obtain a first processed image; extracting a centroid of each illustrative image tile in the first processed image as a semantic node, wherein the illustrative image tile is obtained by dividing the first processed image based on a semantic category of each pixel; forming several semantic node pairs by performing similarity-based pairing between each first semantic node included in the first processed image and each second semantic node included in the second processed image, wherein the second processed image is obtained by performing semantic segmentation and polar coordinate transformation on a historical panoramic image; From each of the semantic node pairs, select a semantic node pair in which a first semantic node and a second semantic node have the highest similarity as a central semantic node pair; Rotating and aligning the first processed image and the second processed image based on the central semantic node pair; Calculating the average value of the included angle between the first semantic node and the second semantic node included in each semantic node pair in the aligned first processed image and the second processed image and the centroid of the first processed image / the second processed image, wherein the vertex of the included angle is the centroid; If the average value of the included angle is greater than a predetermined angle threshold, determining that the detected panoramic image and the history panoramic image do not form a closed loop; and determining that the detected panoramic image and the history panoramic image constitute a closed loop when the average value of the included angle is equal to or less than the angle threshold. A closed-loop detection method comprising:

2. The extraction of the centroid of each illustration image tile in the first processed image as a semantic node includes: Dividing the first processed image into several illustrative image tiles based on the semantic category of each pixel; filtering each of the illustrative image tiles to select static illustrative image tiles whose semantic categories belong to static semantic categories; and extracting the centroid of each said static illustration image tile as a semantic node.

3. calculating the number of pixels contained in each of said static illustration image tiles; extracting, as a semantic node, a centroid of each of the static illustration image tiles whose pixel count reaches a predetermined pixel count threshold.

3. The method of claim 2.

4. forming several semantic node pairs by performing similarity-based pairing of each first semantic node included in the first processed image with each second semantic node included in the second processed image; Identifying semantic categories to which each of the first semantic nodes and each of the second semantic nodes commonly correspond to form the same set of semantic categories; 2. The method of claim 1, further comprising: selecting, based on each of the first semantic nodes corresponding to each of the semantic categories in the same semantic category set, a second semantic node from the second processed image that has the same semantic category as the first semantic node and is most similar to the first semantic node; and forming a semantic node pair between the first semantic node and the most similar second semantic node.

5. Selecting a second semantic node from the second processed image that has the same semantic category as and is most similar to the first semantic node includes: selecting from the second processed image a number of second semantic nodes that have the same semantic category as the first semantic nodes; calculating a first distance from the first semantic node to the center of the first processed image and a second distance from each selected second semantic node to the center of the second processed image; and determining the second semantic node corresponding to the second distance that is closest to the first distance as the most similar second semantic node.

6. Before extracting the centroid of each illustration image tile in the first processed image as a semantic node, calculating a similarity between a set of semantic categories corresponding to the first processed image and a set of semantic categories corresponding to the second processed image; If the similarity is smaller than a predetermined similarity threshold, it is determined that the detected panoramic image and the history panoramic image do not form a closed loop; 2. The method of claim 1, further comprising: if the similarity is greater than or equal to the similarity threshold, proceeding to extract a centroid of each illustrative image tile in the first processed image as a semantic node.

7. 7. An electronic device comprising a memory, a processor, and a computer program stored in and executed by the memory, wherein, when the processor executes the computer program, the electronic device implements the steps of the closed-loop detection method according to any one of claims 1 to 6.

8. 7. A computer-readable storage medium having a computer program stored thereon, the computer program causing a processor to execute the steps of the closed-loop detection method according to any one of claims 1 to 6, when the computer program is executed by the processor.

Images