Image-based tunnel inspection self-positioning method and device

Abstract

The invention provides an image-based tunnel inspection self-positioning method and device. According to the method, the inspection robot is controlled to carry out constant-distance multi-dimensional shooting along the tunnel wall, the cosine similarity between a first image group and a plurality of preset track image groups is calculated, the most similar image group is determined by aggregating similarity scores, and absolute position information of the inspection robot is obtained by querying a target mapping relation. And efficient self-positioning in tunnel inspection is realized. The problem of inaccurate positioning of the tunnel inspection robot in a complex high-similarity environment in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of tunnel inspection, and in particular to an image-based tunnel inspection self-positioning method, an image-based tunnel inspection self-positioning device, a computer-readable storage medium and an inspection robot. Background Art

[0002] Power cables and tunnels are the main carriers of power transmission and power supply security in the capital's power grid. Due to historical reasons, some tunnels have problems such as aging and disrepair. In addition, due to the harsh underground environment, personnel inspections are very difficult and risky, and traditional inspection methods require a lot of manpower and material resources. Due to the narrow underground tunnels, complex terrain, and high similarity, a single robot is limited by its applicable scenarios, cruising capabilities, passability and other factors, and it is difficult to independently complete the inspection of the entire underground power tunnel. Therefore, the collaborative inspection of power tunnels based on inspection robot clusters is studied to break through key technologies such as data communication, task planning, and collaborative control of multi-robot inspections.

[0003] Each inspection robot is a key factor in forming a cluster. Therefore, during the inspection process, the robot cluster not only needs to rely on mutual positioning to complete tasks such as road planning, but also needs a set of accurate self-positioning methods, relying entirely on its own algorithms and sensors to identify its own position, which requires the inspection robot to have strong "individual" capabilities. Therefore, it is very necessary to develop an algorithm that completely relies on the robot's own capabilities to complete continuous and accurate self-positioning of high-similarity cable tunnels.

[0004] At present, the most commonly used technology to solve the positioning problem is SLAM technology. In complex and highly similar environments, SLAM technology is prone to positioning drift, especially after long-term operation, the error may accumulate and lead to inaccurate positioning. The computational complexity of the SLAM algorithm is high, especially when processing large amounts of sensor data and real-time map construction, which may increase the computational burden of the robot system. If the environment changes frequently or there are dynamic elements (such as moving objects), SLAM technology may find it difficult to accurately update the map and positioning information, resulting in positioning errors.

[0005] In addition, SLAM technology cannot solve the self-positioning problem in a high-similarity environment. In a cable tunnel, the similarity between any point A and any point B is very high. The inspection robot can only solve the problem of relative positioning by relying solely on SLAM technology, but cannot solve the problem of absolute positioning. That is, the task cannot be solved by relying solely on positioning sensors. Therefore, the tunnel inspection robot in the existing technology cannot accurately position itself in a complex and high-similarity environment. Summary of the invention

[0006] The main purpose of the present application is to provide an image-based tunnel inspection self-positioning method, an image-based tunnel inspection self-positioning device, a computer-readable storage medium and an inspection robot, so as to at least solve the problem of inaccurate positioning of tunnel inspection robots in complex and high-similarity environments in the prior art.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, there is provided an image-based tunnel inspection self-positioning method, comprising: controlling an inspection robot to perform constant-distance multi-dimensional shooting in real time along the wall on either side of the tunnel to obtain a first image group, wherein the constant-distance multi-dimensional shooting is to keep a constant distance from the tunnel wall and shoot in multiple directions at the same time; performing cosine similarity calculation based on the first image group and the images in multiple second image groups to obtain multiple similarity score groups, performing aggregation based on each of the similarity score groups to obtain a corresponding first similarity score, and determining the second image group corresponding to the maximum value of the first similarity score as a target image group, wherein the second image group is obtained by the inspection robot moving at a uniform speed along a preset trajectory in advance and performing the constant-distance multi-dimensional shooting, and the multiple first image groups correspond to different acquisition times; querying a target mapping relationship according to the target image group to obtain first position information, and completing the self-positioning of the inspection robot, wherein the target mapping relationship is a mapping relationship between the second image group and the absolute position of the inspection robot in the tunnel, and the absolute position is represented by the distance from the tunnel entrance.

[0008] Optionally, cosine similarity is calculated based on the first image group and the images in multiple second image groups to obtain multiple similarity score groups, including: converting the images corresponding to each dimension in the first image group into matrix representations, and performing convolution and dimensionality reduction to obtain multiple first target matrices; converting the images corresponding to each dimension in the second image group into matrix representations, and performing convolution and dimensionality reduction to obtain multiple second target matrices; calculating cosine similarity based on the first target matrix corresponding to the first image group and the second target matrices corresponding to each of the second image groups to obtain multiple similarity score groups.

[0009] Optionally, the images corresponding to each dimension in the first image group are converted into matrix representations, and convolution and dimensionality reduction are performed to obtain multiple first target matrices, including: matrix conversion of the image based on RGB channels to obtain three third target matrices; convolution operations are performed on each of the third target matrices to extract features of the image to obtain a corresponding fourth target matrix; the three fourth target matrices are added to obtain a fifth target matrix; and the fifth target matrix is ​​horizontally stretched and converted into a one-dimensional matrix to obtain the first target matrix.

[0010] Optionally, aggregation is performed based on each of the similarity score groups to obtain a corresponding first similarity score, including: identifying images in the first image group to determine whether there is a preset feature device in the image; in the case where there is an image with the preset feature device in the first image group, increasing the initial weight corresponding to the image with the preset feature device, and reducing the initial weight corresponding to other images in the first image group that do not have the preset feature device, and performing weighted calculation based on the similarity score group corresponding to the first image group according to the updated weights to obtain the first similarity score, the initial weights corresponding to the images are equal and the sum is 1; in the case where there is no image with the preset feature device in the first image group, performing weighted calculation based on the initial weights based on the similarity score group corresponding to the first image group to obtain the first similarity score.

[0011] Optionally, before controlling the inspection robot to perform real-time constant-distance multi-dimensional shooting along the wall on either side of the tunnel, the method further includes: determining the preset trajectory based on the same side wall and the same distance during the constant-distance multi-dimensional shooting of the first image group, wherein the starting point of the preset trajectory is the tunnel entrance, and the end point of the preset trajectory is the tunnel exit; controlling the inspection robot to move at a uniform speed according to the same dimension as the first image group, and performing the constant-distance multi-dimensional shooting to obtain multiple tunnel real-scene videos, wherein the tunnel real-scene videos correspond one-to-one to the dimensions; performing image acquisition on each of the tunnel real-scene videos at preset time intervals to obtain a corresponding third image group; dividing the images with the same acquisition time in the third image group into the same group, and reorganizing all the third image groups to obtain multiple second image groups.

[0012] Optionally, before querying the target mapping relationship according to the target image group to obtain the first position information, the method also includes: performing three-dimensional modeling of the tunnel to obtain a tunnel model; based on the total length of the tunnel and the total acquisition time of the second image group, determining the absolute position of each of the second image groups in the tunnel model to obtain the target mapping relationship.

[0013] Optionally, based on the total length of the tunnel and the number of the second image groups, the absolute position of each second image group in the tunnel model is determined, including: calculating the ratio between the total length S and the total acquisition time t to obtain the travel speed v of the inspection robot; sorting the second image groups based on the time sequence relationship of acquisition, and determining the absolute position x=(n-1)*m*v corresponding to each second image group based on the travel speed v and the preset time interval m, where n is the serial number corresponding to the second image group.

[0014] According to another aspect of the present application, there is provided an image-based tunnel inspection self-positioning device, the device comprising: a first acquisition unit, used to control the inspection robot to perform constant-distance multi-dimensional shooting along the wall on either side of the tunnel in real time to obtain a first image group, wherein the constant-distance multi-dimensional shooting is to keep a constant distance from the tunnel wall and shoot in multiple directions at the same time; a first calculation unit, used to perform cosine similarity calculation based on the first image group and the images in multiple second image groups respectively to obtain multiple similarity score groups, perform aggregation based on each of the similarity score groups respectively to obtain a corresponding first similarity score, and determine the second image group corresponding to the maximum value of the first similarity score as a target image group, wherein the second image group is obtained by the inspection robot moving at a uniform speed along a preset trajectory in advance and performing the constant-distance multi-dimensional shooting, and the multiple first image groups correspond to different acquisition times; a second acquisition unit, used to query a target mapping relationship according to the target image group, obtain first position information, and complete the self-positioning of the inspection robot, wherein the target mapping relationship is a mapping relationship between the second image group and the absolute position of the inspection robot in the tunnel, and the absolute position is represented by the distance from the tunnel entrance.

[0015] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described.

[0016] According to another aspect of the present application, a patrol robot is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the described methods.

[0017] By applying the technical solution of the present application, the inspection robot is first controlled to perform constant-distance multi-dimensional shooting in real time along the wall on either side of the tunnel to obtain a first image group, wherein the constant-distance multi-dimensional shooting is to keep a constant distance from the tunnel wall while shooting in multiple directions; then, cosine similarity calculation is performed based on the images in the first image group and multiple second image groups to obtain multiple similarity score groups, and aggregation is performed based on each of the similarity score groups to obtain a corresponding first similarity score, and the second image group corresponding to the maximum value of the first similarity score is determined as the target image group, wherein the second image group is obtained by the inspection robot moving at a uniform speed along a preset trajectory in advance and performing the constant-distance multi-dimensional shooting, and multiple first image groups correspond to different acquisition times; finally, the target mapping relationship is queried according to the target image group to obtain the first position information, and the self-positioning of the inspection robot is completed, wherein the target mapping relationship is the mapping relationship between the second image group and the absolute position of the inspection robot in the tunnel, and the absolute position is represented by the distance from the tunnel entrance. This application controls the inspection robot to perform constant-distance multi-dimensional shooting along the tunnel wall, and then calculates the cosine similarity between the first image group and multiple preset trajectory image groups, determines the most similar image group by aggregating similarity scores, and obtains the absolute position information of the inspection robot by querying the target mapping relationship, thereby realizing efficient self-positioning in tunnel inspection. This solves the problem of inaccurate positioning of tunnel inspection robots in complex and high-similarity environments in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A hardware structure block diagram of a mobile terminal for tunnel inspection self-positioning based on an image provided in an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram of a process flow of an image-based tunnel inspection self-positioning method provided according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of a process flow of a specific image-based tunnel inspection self-positioning method provided according to an embodiment of the present application is shown;

[0021] Figure 4 A structural block diagram of an image-based tunnel inspection self-positioning device provided according to an embodiment of the present application is shown.

[0022] The above drawings include the following reference numerals:

[0023] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] As introduced in the background technology, in the prior art, SLAM technology is prone to positioning drift in complex and highly similar environments. Especially after long-term operation, errors may accumulate and lead to inaccurate positioning. In order to solve the problem of inaccurate positioning of tunnel inspection robots in complex and highly similar environments, the embodiments of the present application provide an image-based tunnel inspection self-positioning method, an image-based tunnel inspection self-positioning device, a computer-readable storage medium, a processor and an electronic device.

[0028] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of an image-based tunnel inspection self-positioning method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, an image-based tunnel inspection self-positioning method running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0032] Figure 2 FIG. 1 is a flow chart of an image-based tunnel inspection self-positioning method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0033] Step S201, controlling the inspection robot to perform constant-distance multi-dimensional shooting along the wall on any side of the tunnel in real time to obtain a first image group, wherein the constant-distance multi-dimensional shooting is to keep a constant distance from the tunnel wall and shoot in multiple directions at the same time;

[0034] Specifically, ranging sensors A and B are installed at the front and rear ends of one side of the inspection robot. The ranging sensors are required to measure the distance to one side of the tunnel in real time. The distance between the ranging sensor and the one side of the tunnel is required to maintain a preset distance. This can ensure that the distance between the robot and the one side of the tunnel is constant during the entire shooting process, keep the video shooting angle unchanged, and keep the shooting angle consistent for the subsequent positioning and reasoning process, which helps to improve the accuracy of image similarity calculation.

[0035] It can be seen that the constant-distance multi-dimensional shooting of the robot inspection process is to improve the accuracy of the similarity calculation and also make the robot's travel route consistent with the construction route and shooting angle of the collection process. The shooting dimension is 4-dimensional, that is, shooting up, down, left and right to form an image group during the movement process.

[0036] Step S202, performing cosine similarity calculation based on the first image group and the images in the plurality of second image groups, obtaining a plurality of similarity score groups, performing aggregation based on the respective similarity score groups, obtaining a corresponding first similarity score, and determining the second image group corresponding to the maximum value of the first similarity score as the target image group, the second image group is obtained by the inspection robot traveling at a uniform speed along a preset trajectory and performing constant-distance multi-dimensional photography in advance, and the plurality of first image groups correspond to different acquisition times;

[0037] Specifically, the cosine similarity is calculated for the first image group and the images in the multiple second image groups. Cosine similarity is a commonly used indicator to measure the similarity between two vectors. The value is between -1 and 1. The closer the value is to 1, the higher the similarity. Aggregation is performed based on the values ​​of each similarity score group. Possible methods include taking the average, the maximum value, or other statistical methods to obtain a comprehensive first similarity score. From the multiple first similarity scores, find the maximum value and correspond it to the second image group it refers to, and determine that the group is the target image group. The images in this group are considered to best match the characteristics of the first image.

[0038] Step S203, query the target mapping relationship according to the target image group, obtain the first position information, and complete the self-positioning of the inspection robot. The target mapping relationship is the mapping relationship between the second image group and the absolute position of the inspection robot in the tunnel. The absolute position is represented by the distance from the tunnel entrance.

[0039] Specifically, the target mapping relationship refers to the correspondence between the second image group (the image group obtained by the robot traveling at a constant speed along a preset trajectory and taking multi-dimensional photos at a constant distance) and the absolute position of the inspection robot in the tunnel. This mapping relationship may be a pre-built or learned database that records the absolute position corresponding to each image group (usually expressed as the distance from the tunnel entrance). The construction of the mapping relationship can optimize the query speed and accuracy with the help of more efficient data structures (such as tree indexes) or machine learning models (such as regression models). Through the above steps, the inspection robot can achieve self-positioning through the image and position mapping relationship. This process makes full use of the pre-built mapping relationship and the currently collected image data to ensure the positioning accuracy of the robot in the tunnel.

[0040] By applying the technical solution of the present application, the inspection robot is first controlled to perform constant-distance multi-dimensional shooting in real time along the wall on either side of the tunnel to obtain a first image group. The constant-distance multi-dimensional shooting is to keep a constant distance from the tunnel wall while shooting in multiple directions; then, cosine similarity calculations are performed based on the images in the first image group and multiple second image groups to obtain multiple similarity score groups, and aggregation is performed based on each similarity score group to obtain a corresponding first similarity score, and the second image group corresponding to the maximum value of the first similarity score is determined as the target image group. The second image group is obtained by the inspection robot moving at a uniform speed along a preset trajectory and performing constant-distance multi-dimensional shooting in advance. The multiple first image groups correspond to different acquisition times; finally, the target mapping relationship is queried according to the target image group to obtain the first position information, and the self-positioning of the inspection robot is completed. The target mapping relationship is the mapping relationship between the second image group and the absolute position of the inspection robot in the tunnel, and the absolute position is expressed as the distance from the tunnel entrance. This application controls the inspection robot to take constant-distance multi-dimensional photos along the tunnel wall, and then calculates the cosine similarity between the first image group and multiple preset trajectory image groups, determines the most similar image group by aggregating similarity scores, and obtains the absolute position information of the inspection robot by querying the target mapping relationship. The inspection robot can rely on its own sensors and computer vision algorithms to accurately locate the absolute position in the tunnel, and realizes efficient self-positioning in tunnel inspection. This solves the problem of inaccurate positioning of tunnel inspection robots in complex and high-similarity environments in the prior art.

[0041] As a possible implementation, performing cosine similarity calculations on the first image group and the images in the plurality of second image groups to obtain a plurality of similarity score groups includes the following steps:

[0042] Step S301, converting the images corresponding to each dimension in the first image group into matrix representations, and performing convolution and dimensionality reduction to obtain a plurality of first target matrices;

[0043] Step S302, converting the images corresponding to each dimension in the second image group into matrix representations, and performing convolution and dimensionality reduction to obtain multiple second target matrices;

[0044] Specifically, the purpose of convolution and dimensionality reduction is to convert multi-channel and multi-dimensional matrices into one-dimensional data that can be calculated using cosine similarity, which can also be understood as extracting image features.

[0045] Step S303 , calculating cosine similarity based on the first target matrix corresponding to the first image group and the second target matrix corresponding to each second image group, to obtain a plurality of similarity score groups.

[0046] In summary, for each pair of paired matrices, the cosine similarity formula is used to calculate the similarity between them. Cosine similarity measures the similarity of two vectors in direction, and its value range is between -1 and 1. The closer the value is to 1, the more similar the two vectors are. The convolution and dimensionality reduction steps reduce the complexity of the data while extracting image features, making the calculation of cosine similarity more efficient and accurate.

[0047] As a possible implementation method, the images corresponding to each dimension in the first image group are converted into matrix representations, and convolution and dimensionality reduction are performed to obtain multiple first target matrices, including the following steps:

[0048] Step 401, performing matrix conversion on the image based on the RGB channels to obtain three third target matrices;

[0049] Step 402, performing convolution operations on each third target matrix respectively to extract features of the image to obtain a corresponding fourth target matrix;

[0050] Step 403, adding the three fourth target matrices to obtain a fifth target matrix;

[0051] Step 404: Transversely stretch the fifth target matrix to convert it into a one-dimensional matrix to obtain a first target matrix.

[0052] Specifically, convolution and dimensionality reduction are mainly divided into the following steps: First, convert the image into a 3-channel digital matrix, set the matrix dimension to m*n, and then extract image features from the 3 m*n matrices. Then, set a convolution kernel of appropriate size and convolve each m*n matrix. Through the convolution operation, the information of the sparse matrix is ​​compressed to form 3 new matrices x*y. After that, aggregate the channels, add the 3 x*y matrices, reduce the matrix dimension, and turn 3*x*y into an x*y matrix. This step is essentially to compress the information. Finally, convert the x*y matrix into a 1-dimensional matrix, stretch the matrix horizontally, and finally become a 1*p matrix, p=x*y. This 1-dimensional array of length p is the digital form of the image features for which we want to calculate cosine similarity.

[0053] As a possible implementation, performing aggregation based on each similarity score group to obtain a corresponding first similarity score includes the following steps:

[0054] Step 501, identifying images in the first image group to determine whether there is a preset feature device in the image;

[0055] Step 502: when there is an image with a preset feature device in the first image group, the initial weight corresponding to the image with the preset feature device is increased, and the initial weight corresponding to other images in the first image group without the preset feature device is reduced, and a weighted calculation is performed based on the similarity score group corresponding to the first image group according to the updated weight to obtain a first similarity score, where the initial weights corresponding to the images are equal and the sum is 1;

[0056] Step 503 , when there is no image of the preset feature device in the first image group, weighted calculation is performed based on the similarity score group corresponding to the first image group according to the initial weight to obtain a first similarity score.

[0057] Specifically, assuming that we can get z groups of scores through cosine similarity calculation, and each group has 4 scores (up, down, left, and right), we need to aggregate the four scores in each group.

[0058] The score aggregation algorithm includes an image recognition model. The main task of the image recognition model is to identify some feature devices around. The feature devices here refer to devices that can clearly distinguish the point from other points. When the model recognizes the device we specified in any dimension, we set this situation as case 2, otherwise it is case 1.

[0059] Case 1 is the majority of cases during the inspection process. In this case, the scores of the four dimensions of top, left, right, and bottom calculated based on cosine similarity are score1, score2, score3, and score4 respectively. We use the following formula to complete the score aggregation. Formula: Final_score = h1*score1+h2*score2+h3*score3+h4*score4, (h1=h2=h3=h4=0.25). Case 2, when the image recognizes a feature device, the corresponding weight of the dimension of the feature device is increased to increase the accuracy of position recognition. For example, when the feature device is recognized on the left, h2=0.4, h1=h3=h4=0.2. In this way, the scores of the four dimensions are aggregated into one score, that is, each image group corresponds to one score.

[0060] Through the above steps, the present application introduces cosine similarity calculation into image similarity calculation. This similarity calculation method is more suitable for the task scenario of image similarity calculation, and can greatly improve the accuracy of image comparison. It can effectively calculate and aggregate the cosine similarity between different data groups, and help the robot to perform more effective image evaluation and processing during operation and inspection.

[0061] As a possible implementation manner, before controlling the inspection robot to perform real-time constant-distance multi-dimensional shooting along the wall on either side of the tunnel, the method further includes the following steps:

[0062] Step 601, determining a preset trajectory based on the same side wall and the same distance during the constant distance multi-dimensional shooting process of the first image group, wherein the starting point of the preset trajectory is the tunnel entrance and the end point of the preset trajectory is the tunnel exit;

[0063] Specifically, the information in the first image group is first analyzed and utilized, and when shooting, the shooting position of the robot is ensured to be the same as that of the first image group, which will ensure consistent image features and reduce errors caused by distance changes.

[0064] Step 602, controlling the inspection robot to move at a constant speed in the same dimension as the first image group, and performing constant-distance multi-dimensional shooting to obtain a plurality of tunnel real-scene videos, where the tunnel real-scene videos correspond to the dimensions one by one;

[0065] Specifically, the entire shooting process should be kept at a constant speed. In the subsequent mapping process, the time and location of the video should correspond to the location of the tunnel model as accurately as possible. The video recording process needs to have good continuity and uniformity, and the time spent on each unit distance should be kept consistent. This will make the model and video mapping more accurate. The video is recorded in four directions. Multi-dimensional video recording can collect the features of the location from more angles. Combined with the score aggregation algorithm, it can better identify specific scene features from scenes with high similarity, and flexibly adjust the score weight of each dimension to increase the accuracy of recognition and positioning.

[0066] It can be seen that the entire data collection process and algorithm development process are low-cost. The initial data collection process only requires the completion of video recording, and the image set is completely acquired by taking frames from the video. The entire process is relatively easy to implement and has low cost.

[0067] Step 603, collecting images of each tunnel real-scene video at a preset time interval to obtain a corresponding third image group;

[0068] Step 604: divide the images in the third image group that are collected at the same time into the same group, and reorganize all the third image groups to obtain multiple second image groups.

[0069] By collecting tunnel real-life information in the above way, four video files are finally generated, namely, multi-dimensional real-life video of the tunnel. A multi-dimensional frame image set is established. The essence of a video file is an ordered and finite set of multi-needle images. The corresponding frames in the video can be collected at equal intervals to form an ordered image set. Each dimension collects frames according to this rule, and four image sets will be generated. The four images collected at the same frame time form an image group, which corresponds to time one by one.

[0070] As a possible implementation manner, before querying the target mapping relationship according to the target image group to obtain the first position information, the method further includes the following steps:

[0071] Step S701, three-dimensional modeling of the tunnel to obtain a tunnel model;

[0072] Specifically, high-similarity tunnels are modeled, and the purpose of modeling is to establish a one-to-one mapping between each set of images and tunnels.

[0073] Step S702: Based on the total length of the tunnel and the total acquisition time of the second image groups, the absolute position of each second image group in the tunnel model is determined to obtain a target mapping relationship.

[0074] Specifically, this application targets the task of self-positioning in a highly similar cable tunnel, so it is necessary to model the working scene of the inspection robot to provide a basic space for the positioning task. Since we mainly locate the absolute position of the inspection robot in the tunnel, the tunnel model only needs to display the feature of the tunnel length.

[0075] As a possible implementation manner, based on the total length of the tunnel and the number of the second image groups, determining the absolute position of each second image group in the tunnel model includes the following steps:

[0076] Step S801, calculating the ratio between the total length S and the total acquisition time t, and obtaining the traveling speed v of the inspection robot;

[0077] Step S802, sort the second image groups based on the time sequence of acquisition, and determine the absolute position x=(n-1)*m*v corresponding to each second image group based on the travel speed v and the preset time interval m, where n is the sequence number corresponding to the second image group.

[0078] Through the above steps, the absolute position of each second image group in the tunnel can be accurately calculated according to the robot's travel speed and time interval in the tunnel. The key to this process is to reasonably use speed, time and sequence information to ensure that the obtained image groups can correspond to the corresponding positions one by one. The inspection robot can complete position recognition at any position and at any time in a highly similar tunnel. The position recognition task becomes very coherent and flexible, and has continuous positioning capabilities.

[0079] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the image-based tunnel inspection self-positioning method of the present application will be described in detail below in combination with specific embodiments.

[0080] This technical solution is mainly divided into two parts: data collection part and positioning reasoning part.

[0081] The data collection part is mainly divided into three steps. First, the high-similarity tunnel is modeled. The purpose of modeling is to establish a one-to-one mapping between each set of images and the tunnel. The second step is data collection. The collection method is to use a quadruped robot to carry four video recording devices. These four devices are required to shoot in four dimensions: up, down, left, and right. During the shooting process, the robot is kept parallel to the wall on one side and at a constant distance. The uniform speed and constant distance parallel shooting method is to keep the shooting angle fixed, which helps to improve the accuracy of subsequent image similarity calculations. After completing the four-dimensional video recording and tunnel modeling of the entire tunnel, it is necessary to establish a mapping between the video and the tunnel model, that is, each frame of the video should correspond one-to-one to each point of the tunnel model. When establishing this mapping, the video needs to be framed at equal intervals, that is, the four videos are converted into four ordered image sets, and then each group of images of the four ordered image sets (a group of images contains four images of the top, bottom, left and right of this point position) is mapped to the tunnel model through the video. This means that each group of images corresponds to a position in the tunnel. At this point, the mapping between the tunnel model and the image is completed. The next task is to identify the four dimensions of the top, bottom, left and right of the robot's current position, and compare it with each group of images in the ordered image set. From the pixel level, it is determined which group of all image sets is closest. After finding the closest image, the specific position of the tunnel model where the robot is currently located can be determined through the previously established mapping.

[0082] Then compare the image of the current position in four dimensions with all the images in the image set, and enter the positioning reasoning part. This part mainly uses the cosine similarity calculation method innovatively to calculate the image similarity. Compared with the Euclidean distance, the cosine similarity method has the following advantages: cosine similarity is not affected by the size of the vector. Cosine similarity measures the direction of the vector rather than the size, so it is insensitive to the absolute size of the vector. This is particularly useful in processing tasks such as text classification and recommendation systems that need to consider the direction of the text rather than the length of the text. Cosine similarity is suitable for high-dimensional sparse data. In high-dimensional sparse data sets, cosine similarity can usually more accurately reflect the similarity between vectors because it focuses on the angle between vectors rather than the specific value. Since the essence of an image is that each pixel is a 3-channel 0-255 number, that is, a 3-channel digital matrix, and a high-similarity image means that the information is sparse, and after the image matrix is ​​transformed into a very high-dimensional data, it is more appropriate to use cosine similarity for calculation. The next question is how to map each image into a high-dimensional space. Here we use convolution, aggregation, and neural networks to compress information and transform dimensions to extract the features of each image.

[0083] Through cosine similarity calculation, we find the maximum similarity score, find the corresponding ordered image group through the maximum score, and locate the only point of the tunnel model through the ordered image group, that is, the absolute position of the robot in the tunnel model.

[0084] This embodiment relates to a specific image-based tunnel inspection self-positioning method, such as Figure 3 This solution solves the difficulty of absolute position judgment through computer vision technology and overcomes the challenges brought by the high similarity between points in the cable tunnel scene. First, it collects data through four-dimensional shooting and fixed viewing angle through modeling and mapping, and corresponds the video data to the tunnel model one by one to build a mapping relationship. Then, the current image data is collected through four-dimensional shooting and fixed viewing angle. In subsequent processing, the image similarity between the current collected image and the video frame data is calculated at the pixel dimension level, and high-dimensional space mapping is performed using cosine similarity and other methods. The image features are extracted and analyzed, and the absolute position of the robot in the tunnel model is located through the calculation results and the previously established mapping. The whole process covers tunnel modeling, data collection, mapping establishment, cosine similarity calculation, mapping reverse deduction and other links, thereby realizing the precise self-positioning of the inspection robot in a complex and high-similarity environment. It can be divided into the following steps:

[0085] Step 1: Tunnel modeling. The present invention is aimed at the high-similarity cable tunnel self-positioning task, so it is necessary to model the working scene of the inspection robot to provide a basic space for the positioning task. Since we mainly locate the absolute position of the inspection robot in the tunnel, the tunnel model only needs to display the feature of the tunnel length.

[0086] Step 2: Constant-distance and uniform-speed four-dimensional video recording. Constant-distance and multi-dimensional shooting requires ensuring several key points to ensure data accuracy and safe and efficient operation of the robot. First, distance sensors A and B are installed on the front and rear ends of one side of the inspection robot. The distance sensor must measure the distance of one side of the tunnel in real time. The distance between the distance sensor and the side of the tunnel must always maintain a preset distance, which can ensure that the distance between the robot and the side of the tunnel is constant during the entire shooting process, keep the video shooting angle unchanged, and keep the shooting angle consistent for the subsequent positioning and reasoning process, which helps to improve the accuracy of image similarity calculation. Secondly, the entire shooting process should be kept at a uniform speed as much as possible. In the subsequent mapping process, the time and point of the video recording should be as accurate as possible. The point of the tunnel model should correspond to the meaning, which requires the video recording process to have good continuity and uniformity, and to ensure that the time spent on each unit distance is consistent, so that it will be more accurate when the model is established and mapped with the video recording through time. Finally, we record videos in four directions. Multi-dimensional video can collect features of the location from more angles. Combined with the score aggregation algorithm, we can better identify specific scene features from scenes with high similarity, and flexibly adjust the score weight of each dimension to increase the accuracy of recognition and positioning.

[0087] Through the above method, the tunnel real scene information is collected, and finally 4 video files are generated, that is, the tunnel multi-dimensional real scene video. Establish a multi-dimensional frame image set. The essence of a video file is an ordered and finite set of multi-needle images, so we can collect the corresponding frames in the video at equal intervals to form an ordered image set. Each dimension collects frames according to this rule, and four image sets will be generated. The four images collected at the same frame time form an image group, and this image group corresponds to time one by one.

[0088] Step 3: Create a multi-dimensional frame-sampling image set. The essence of a video file is an ordered and finite set of multi-needle images, so we can collect the corresponding frames in the video at equal intervals to form an ordered image set. If each dimension is framed according to this rule, four image sets will be generated. The four images collected at the same frame-sampling time form an image group, which corresponds one-to-one to time.

[0089] Step 4: Establish a mapping between the image group and the tunnel model. The length of the tunnel S divided by the recording time t is the robot's travel speed v. By completing the frame collection time interval, the frame collection image group can be mapped to the corresponding tunnel model location. Here we set the frame collection time interval to m seconds:

[0090] The tunnel position corresponding to the first image group is 0;

[0091] The tunnel position corresponding to the nth image group is (n-1)*m*v, n≥2, and n is an integer, wherein, when n=2, the tunnel position corresponding to the second image group is m*v. ;

[0092] The mapping between the image group and the tunnel model is established through the above steps.

[0093] The data collection part is completed through the above four steps. Next, the positioning reasoning process is described.

[0094] Step 5: Constant-distance multi-dimensional shooting of the robot inspection process. In order to improve the accuracy of similarity calculation and to ensure that the robot's route is consistent with the construction route and shooting angle of the acquisition process, the shooting dimension is also 4D, shooting up, down, left and right to form an image group during the movement process.

[0095] Step 6: Convolution and dimensionality reduction. The purpose of this step is to convert the multi-channel and multi-dimensional matrix into one-dimensional data that can be calculated using cosine similarity. It can also be understood as extracting the features of the image. It is mainly divided into the following steps:

[0096] Convert the image into a 3-channel digital matrix, and set the matrix dimension to m*n. Next, extract image features from this 3*m*n matrix. Set a convolution kernel of appropriate size and convolve each m*n matrix. Through the convolution operation, compress the information of the sparse matrix to form three new matrices x*y. Aggregate channels, add the three x*y matrices, reduce the matrix dimension, and turn 3*x*y into an x*y matrix. This step is essentially to compress information. Convert the x*y matrix into a 1-dimensional matrix, stretch the matrix horizontally, and finally become a 1*p matrix, p=x×y. This 1-dimensional array of length p is the digital form of the features of the image we want to calculate cosine similarity.

[0097] Step 7: Calculate image cosine similarity. At present, we need to compare and calculate the similarity of two sets of data. One set is the image data set that has been collected. This data set is composed of multiple data sets (we assume that there are z image sets in this image data set). The other set is the image data set taken from a certain position a in the process of robot operation and inspection (set as data set K). We need to use data set K and each data set in the data set to correspond to the corresponding dimensions for cosine similarity calculation.

[0098] The cosine similarity calculation formula is cosine_score=(A·B) / (||A||||B||), where A and B are two vectors, and (||A|| and ||B||) are the moduli of A and B respectively.

[0099] Where A·B represents the inner product of vector A and vector B, expressed as: A·B=∑(i=1)^(n)A i *B i .

[0100] By calculating the cosine similarity, we can get z groups of scores. Each group has 4 scores (up, down, left, and right). Next, we need to aggregate the four scores in each group.

[0101] Step 8: The score aggregation algorithm includes an image recognition model. The main task of the image recognition model is to identify some feature devices around. The feature devices here refer to devices that can clearly distinguish the point from other points. When the model recognizes the device we specified in any dimension, we set this situation as case 2, otherwise it is case 1.

[0102] Case 1 is the majority of cases in the inspection process. In this case, the scores of the four dimensions of top, left, right, and bottom calculated based on cosine similarity are score1, score2, score3, and score4. We use the following formula to complete score aggregation.

[0103] Formula: Final_score=h1*score1+h2*score2+h3*score3+h4*score4, (h1=h2=h3=h4=0.25).

[0104] In case 2, when a feature device is recognized in the image, the weight corresponding to the dimension of the recognized feature device is increased to increase the accuracy of position recognition. For example, when a feature device is recognized on the left, h2 = 0.4, h1 = h3 = h4 = 0.2.

[0105] In this way, the scores of the four dimensions are aggregated into one score, that is, each image group corresponds to one score.

[0106] Step 9: Mapping and inference, sort the scores of all the image groups obtained, find the image group corresponding to the largest score, and then infer the tunnel position by mapping the image group with the tunnel model.

[0107] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0108] The embodiment of the present application also provides an image-based tunnel inspection self-positioning device. It should be noted that the image-based tunnel inspection self-positioning device of the embodiment of the present application can be used to execute the image-based tunnel inspection self-positioning method provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been described will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0109] The following is an introduction to the image-based tunnel inspection self-positioning device provided in an embodiment of the present application.

[0110] Figure 4 is a structural block diagram of an image-based tunnel inspection self-positioning device according to an embodiment of the present application. Figure 4 As shown, the device includes: a first acquisition unit 10, a first calculation unit 20 and a second acquisition unit 30.

[0111] The first acquisition unit 10 is used to control the inspection robot to perform constant-distance multi-dimensional shooting along the wall on any side of the tunnel in real time to obtain a first image group. The constant-distance multi-dimensional shooting is to keep a constant distance from the tunnel wall and shoot in multiple directions at the same time;

[0112] A first calculation unit 20 is used to perform cosine similarity calculation based on the first image group and the images in the plurality of second image groups, respectively, to obtain a plurality of similarity score groups, to perform aggregation based on the respective similarity score groups, to obtain corresponding first similarity scores, and to determine the second image group corresponding to the maximum value of the first similarity score as the target image group, the second image group is obtained by the inspection robot traveling at a uniform speed along a preset trajectory and performing constant-distance multi-dimensional photography in advance, and the plurality of first image groups correspond to different acquisition moments;

[0113] The second acquisition unit 30 is used to query the target mapping relationship according to the target image group, obtain the first position information, and complete the self-positioning of the inspection robot. The target mapping relationship is the mapping relationship between the second image group and the absolute position of the inspection robot in the tunnel. The absolute position is expressed as the distance from the tunnel entrance.

[0114] The embodiment of the present application provides an image-based tunnel inspection self-positioning device, which includes: a first acquisition unit, a first calculation unit, and a second acquisition unit. The device controls the inspection robot to perform constant-distance multi-dimensional shooting along the tunnel wall, and then calculates the cosine similarity between the first image group and multiple preset trajectory image groups, and determines the most similar image group by aggregating similarity scores. By querying the target mapping relationship, the absolute position information of the inspection robot is obtained. The inspection robot can rely on its own sensors and computer vision algorithms to complete the precise positioning of the absolute position in the tunnel, and realizes efficient self-positioning in tunnel inspection. The problem of inaccurate positioning of tunnel inspection robots in complex and high-similarity environments in the prior art is solved.

[0115] As a possible implementation manner, the first calculation unit includes: a first conversion module, a second conversion module and a calculation module.

[0116] A first conversion module, used to convert the images corresponding to each dimension in the first image group into matrix representations, and perform convolution and dimensionality reduction to obtain multiple first target matrices;

[0117] A second conversion module is used to convert the images corresponding to each dimension in the second image group into matrix representations, and perform convolution and dimensionality reduction to obtain multiple second target matrices;

[0118] Specifically, the purpose of convolution and dimensionality reduction is to convert multi-channel and multi-dimensional matrices into one-dimensional data that can be calculated using cosine similarity, which can also be understood as extracting image features.

[0119] The calculation module is used to calculate the cosine similarity based on the first target matrix corresponding to the first image group and the second target matrix corresponding to each second image group to obtain multiple similarity score groups.

[0120] In summary, for each pair of paired matrices, the cosine similarity formula is used to calculate the similarity between them. Cosine similarity measures the similarity of two vectors in direction, and its value range is between -1 and 1. The closer the value is to 1, the more similar the two vectors are. The convolution and dimensionality reduction steps reduce the complexity of the data while extracting image features, making the calculation of cosine similarity more efficient and accurate.

[0121] As a possible implementation manner, the first conversion module includes: a conversion submodule, a feature extraction submodule, an addition submodule and a stretch conversion submodule.

[0122] A conversion submodule, used for performing matrix conversion on the image based on RGB channels to obtain three third target matrices;

[0123] A feature extraction submodule, used for performing convolution operations on each third target matrix respectively to extract features from the image and obtain a corresponding fourth target matrix;

[0124] A summing submodule, used for summing the three fourth target matrices to obtain a fifth target matrix;

[0125] The stretching conversion submodule is used to perform horizontal stretching on the fifth target matrix and convert it into a one-dimensional matrix to obtain a first target matrix.

[0126] As a possible implementation manner, the first calculation unit further includes: an identification module, a weight module and a weighting module.

[0127] A recognition module, used to recognize images in the first image group to determine whether there is a preset feature device in the image;

[0128] A weight module, for increasing the initial weight corresponding to the image with the preset feature device when there is an image with the preset feature device in the first image group, and reducing the initial weight corresponding to other images in the first image group without the preset feature device, and performing weighted calculation based on the similarity score group corresponding to the first image group according to the updated weight to obtain a first similarity score, wherein the initial weights corresponding to the images are equal and the sum is 1;

[0129] The weighting module is used to obtain a first similarity score by performing weighted calculation based on an initial weight based on a similarity score group corresponding to the first image group when there is no image of a preset feature device in the first image group.

[0130] Through the above steps, the present application introduces cosine similarity calculation into image similarity calculation. This similarity calculation method is more suitable for the task scenario of image similarity calculation, and can greatly improve the accuracy of image comparison. It can effectively calculate and aggregate the cosine similarity between different data groups, and help the robot to perform more effective image evaluation and processing during operation and inspection.

[0131] As a possible implementation manner, the device further includes: a trajectory determination unit, a control unit, an image acquisition unit and a division and recombination unit.

[0132] A trajectory determination unit, configured to determine a preset trajectory based on the same side wall and the same distance during the constant distance multi-dimensional shooting of the first image group, wherein the starting point of the preset trajectory is the tunnel entrance and the end point of the preset trajectory is the tunnel exit;

[0133] A control unit, used to control the inspection robot to move at a uniform speed in the same dimension as the first image group, and to perform constant-distance multi-dimensional shooting to obtain a plurality of tunnel real-scene videos, wherein the tunnel real-scene videos correspond to the dimensions one by one;

[0134] An image acquisition unit, used to acquire images of each tunnel real-scene video at preset time intervals to obtain a corresponding third image group;

[0135] The dividing and reorganizing unit is used to divide the images in the third image group that are collected at the same time into the same group, and reorganize all the third image groups to obtain multiple second image groups.

[0136] By collecting tunnel real-life information in the above way, four video files are finally generated, namely, multi-dimensional real-life video of the tunnel. A multi-dimensional frame image set is established. The essence of a video file is an ordered and finite set of multi-needle images. The corresponding frames in the video can be collected at equal intervals to form an ordered image set. Each dimension collects frames according to this rule, and four image sets will be generated. The four images collected at the same frame time form an image group, which corresponds to time one by one.

[0137] As a possible implementation manner, the device further includes: a modeling unit and a mapping unit.

[0138] A modeling unit, used for performing three-dimensional modeling of the tunnel to obtain a tunnel model;

[0139] The mapping unit is used to determine the absolute position of each second image group in the tunnel model based on the total length of the tunnel and the total acquisition time of the second image groups, and obtain a target mapping relationship.

[0140] Specifically, this application targets the task of self-positioning in a highly similar cable tunnel, so it is necessary to model the working scene of the inspection robot to provide a basic space for the positioning task. Since we mainly locate the absolute position of the inspection robot in the tunnel, the tunnel model only needs to display the feature of the tunnel length.

[0141] As a possible implementation manner, the mapping unit includes: a calculation module and an absolute position module.

[0142] A calculation module is used to calculate the ratio between the total length S and the total acquisition time t to obtain the traveling speed v of the inspection robot;

[0143] The absolute position module is used to sort the second image groups based on the time sequence of acquisition, and determine the absolute position x=(n-1)*m*v corresponding to each second image group based on the travel speed v and the preset time interval m, where n is the serial number corresponding to the second image group.

[0144] Through the above steps, the absolute position of each second image group in the tunnel can be accurately calculated according to the robot's travel speed and time interval in the tunnel. The key to this process is to reasonably use speed, time and sequence information to ensure that the obtained image groups can correspond to the corresponding positions one by one. The inspection robot can complete position recognition at any position and at any time in a highly similar tunnel. The position recognition task becomes very coherent and flexible, and has continuous positioning capabilities.

[0145] The image-based tunnel inspection self-positioning device includes a processor and a memory. The first acquisition unit, the first calculation unit, and the second acquisition unit are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0146] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the communication efficiency can be improved by adjusting the kernel parameters.

[0147] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0148] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the image-based tunnel inspection self-positioning method.

[0149] An embodiment of the present invention provides an inspection robot, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the above-mentioned image-based tunnel inspection self-positioning method.

[0150] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0151] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0152] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0153] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0155] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0156] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0157] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0158] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0159] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0160] 1) The image-based tunnel inspection self-positioning method of the present application controls the inspection robot to perform constant-distance multi-dimensional shooting along the tunnel wall, and then calculates the cosine similarity between the first image group and multiple preset trajectory image groups, and determines the most similar image group by aggregating similarity scores. By querying the target mapping relationship, the absolute position information of the inspection robot is obtained. The inspection robot can rely on its own sensors and computer vision algorithms to complete the precise positioning of the absolute position in the tunnel, and realizes efficient self-positioning in tunnel inspection. The problem of inaccurate positioning of tunnel inspection robots in complex and high-similarity environments in the prior art is solved.

[0161] 2) The image-based tunnel inspection self-positioning device of the present application includes: a first acquisition unit, a first calculation unit, and a second acquisition unit. The device controls the inspection robot to perform constant-distance multi-dimensional shooting along the tunnel wall, and then calculates the cosine similarity between the first image group and multiple preset trajectory image groups, and determines the most similar image group by aggregating similarity scores. By querying the target mapping relationship, the absolute position information of the inspection robot is obtained. The inspection robot can rely on its own sensors and computer vision algorithms to complete the precise positioning of the absolute position in the tunnel, and realizes efficient self-positioning in tunnel inspection. The problem of inaccurate positioning of tunnel inspection robots in complex and high-similarity environments in the prior art is solved.

[0162] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image-based tunnel inspection self-positioning method, characterized in that: include: Control the inspection robot to perform constant-distance multi-dimensional shooting in real time along the wall on any side of the tunnel to obtain a first image group, wherein the constant-distance multi-dimensional shooting is to keep a constant distance from the tunnel wall while shooting in multiple directions; Based on the first image group and the images in the plurality of second image groups, cosine similarity calculation is performed respectively to obtain a plurality of similarity score groups, and based on the respective similarity score groups, corresponding first similarity scores are obtained by aggregation, and the second image group corresponding to the maximum value of the first similarity score is determined as the target image group, the second image group is obtained by the inspection robot moving at a uniform speed along a preset trajectory and performing the constant distance multi-dimensional shooting in advance, and the plurality of the first image groups correspond to different acquisition times; The target mapping relationship is queried according to the target image group to obtain the first position information and complete the self-positioning of the inspection robot. The target mapping relationship is the mapping relationship between the second image group and the absolute position of the inspection robot in the tunnel, and the absolute position is expressed as the distance from the tunnel entrance.

2. The method according to claim 1, characterized in that Based on the images in the first image group and the images in the plurality of second image groups, cosine similarity calculation is performed respectively to obtain a plurality of similarity score groups, including: Converting the images corresponding to each dimension in the first image group into matrix representations respectively, and performing convolution and dimensionality reduction to obtain a plurality of first target matrices; Converting the images corresponding to each dimension in the second image group into matrix representations respectively, and performing convolution and dimensionality reduction to obtain multiple second target matrices; The cosine similarity is calculated based on the first target matrix corresponding to the first image group and the second target matrix corresponding to each of the second image groups to obtain a plurality of similarity score groups.

3. The method according to claim 2, characterized in that The images corresponding to each dimension in the first image group are respectively converted into matrix representations, and convolution and dimensionality reduction are performed to obtain multiple first target matrices, including: Perform matrix transformation on the image based on the RGB channels to obtain three third target matrices; Performing convolution operations on each of the third target matrices respectively to extract features from the image to obtain a corresponding fourth target matrix; Adding the three fourth target matrices to obtain a fifth target matrix; The fifth target matrix is ​​laterally stretched and converted into a one-dimensional matrix to obtain the first target matrix.

4. The method according to claim 1, characterized in that: Aggregating each of the similarity score groups to obtain a corresponding first similarity score includes: Identify the images in the first image group to determine whether there is a preset feature device in the image; In the case where there is an image with the preset feature device in the first image group, the initial weight corresponding to the image with the preset feature device is increased, and the initial weight corresponding to other images in the first image group without the preset feature device is reduced, and based on the similarity score group corresponding to the first image group, weighted calculation is performed according to the updated weight to obtain the first similarity score, and the initial weights corresponding to the images are equal and the sum is 1; In the case that there is no image of the preset feature device in the first image group, a weighted calculation is performed based on the initial weight on the similarity score group corresponding to the first image group to obtain the first similarity score.

5. The method according to claim 1, characterized in that Before controlling the inspection robot to perform real-time constant-distance multi-dimensional photography along the wall on either side of the tunnel, the method further includes: Determine the preset trajectory based on the same side wall and the same distance during the constant distance multi-dimensional shooting of the first image group, wherein the starting point of the preset trajectory is the tunnel entrance and the end point of the preset trajectory is the tunnel exit; Controlling the inspection robot to move at a constant speed according to the same dimension as the first image group, and performing the constant-distance multi-dimensional shooting to obtain a plurality of tunnel real-scene videos, wherein the tunnel real-scene videos correspond to the dimensions one by one; Capturing images of each tunnel real-scene video at preset time intervals to obtain a corresponding third image group; The images in the third image groups having the same acquisition time are divided into the same group, and all the third image groups are reorganized to obtain a plurality of the second image groups.

6. The method according to claim 5, characterized in that Before querying the target mapping relationship according to the target image group to obtain the first position information, the method further includes: Carry out three-dimensional modeling of the tunnel to obtain a tunnel model; Based on the total length of the tunnel and the total acquisition time of the second image groups, the absolute position of each of the second image groups in the tunnel model is determined to obtain the target mapping relationship.

7. The method according to claim 6, characterized in that Determining the absolute position of each of the second image groups in the tunnel model based on the total length of the tunnel and the number of the second image groups includes: Calculate the ratio of the total length S to the total acquisition time t to obtain the traveling speed v of the inspection robot; The second image groups are sorted based on the time sequence of acquisition, and the absolute position x=(n-1)*m*v corresponding to each second image group is determined based on the travel speed v and the preset time interval m, where n is the sequence number corresponding to the second image group.

8. An image-based tunnel inspection self-positioning device, characterized in that: The device comprises: A first acquisition unit is used to control the inspection robot to perform constant-distance multi-dimensional shooting along the wall on any side of the tunnel in real time to obtain a first image group, wherein the constant-distance multi-dimensional shooting is to keep a constant distance from the tunnel wall while shooting in multiple directions; a first calculation unit, configured to perform cosine similarity calculations based on the first image group and images in the plurality of second image groups to obtain a plurality of similarity score groups, perform aggregation based on the respective similarity score groups to obtain corresponding first similarity scores, and determine the second image group corresponding to the maximum value of the first similarity score as the target image group, wherein the second image group is obtained by the inspection robot traveling at a uniform speed along a preset trajectory and performing the constant-distance multi-dimensional shooting in advance, and the plurality of first image groups correspond to different acquisition moments; The second acquisition unit is used to query the target mapping relationship according to the target image group, obtain the first position information, and complete the self-positioning of the inspection robot. The target mapping relationship is the mapping relationship between the second image group and the absolute position of the inspection robot in the tunnel, and the absolute position is expressed as the distance from the tunnel entrance.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

10. A patrol robot, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of claims 1 to 7.

Images