Method, device, computer equipment and storage medium for extracting trails in tunnels

Abstract

The present disclosure relates to a method, device, computer equipment, and storage medium for extracting trails in tunnels. The method comprises: rendering a pre-created virtual tunnel scene that matches the actual tunnel scene, obtaining a tunnel image sequence after rendering, the tunnel image sequence comprising: a virtual tunnel sequence and a trail sequence in the virtual tunnel scene; inputting the actual tunnel scene into a pre-trained U-shaped network trail extraction model, outputting the trails in the actual tunnel scene through the U-shaped network trail extraction model, the U-shaped network trail extraction model comprising: inputting the virtual tunnel sequence and the trail sequence in the virtual tunnel into the U-shaped network model, and obtaining a model by adjusting the objective function of the U-shaped network model. The present method can accurately identify and automatically extract trails in tunnels, saving a lot of manpower and reducing manual workload.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, computer equipment, and storage medium for extracting a trail in a tunnel. Background Art

[0002] With the continuous development of my country's electric power industry, the construction of underground power grids in cities is becoming more and more popular. Water seepage, cable aging, corrosion, breakage and cable ignition occur frequently in tunnels. Therefore, tunnel inspections are necessary to detect and eliminate faults in tunnel cables in a timely manner. However, due to the special environment of tunnels, manual inspections are costly and difficult.

[0003] Then inspection robots emerged. Inspection robot technology is a robot that replaces manual inspection of cables in tunnels. The modules in the inspection robots can accurately locate the location of cable faults, reduce cable fires and similar accidents, ensure the safe and stable operation of tunnel cables, and promote the construction of smart grids.

[0004] At present, the inspection of cable tunnels usually uses quadruped robots that can walk directly on the trails. However, when quadruped robots are used for inspection, due to the narrowness of the tunnels, the robots need to be able to accurately identify the trails in the tunnels during inspection. However, the traditional method of identifying trails requires collecting a large number of tunnel images and manually marking the trails. In addition, due to the changes in the light in the tunnel, the accuracy of the trails identified by the quadruped robots is not high, which may cause safety hazards. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, computer equipment, and storage medium for extracting trails in tunnels that can accurately identify and automatically extract trails in tunnels in response to the above-mentioned technical problems.

[0006] A method for extracting a trail in a tunnel, the method comprising:

[0007] Rendering a pre-created virtual tunnel scene that matches the actual tunnel scene, and obtaining a tunnel image sequence after rendering, wherein the tunnel image sequence includes: a virtual tunnel sequence and a trail sequence in the virtual tunnel scene;

[0008] The actual tunnel scene is input into the pre-trained U-shaped network trail extraction model, and the trail in the actual tunnel scene is output through the U-shaped network trail extraction model. The U-shaped network trail extraction model includes: inputting the virtual tunnel sequence and the trail sequence in the virtual tunnel into the U-shaped network model, and obtaining the model by adjusting the objective function of the U-shaped network model.

[0009] In one embodiment, the U-shaped network trail extraction model is obtained by:

[0010] Input the trail sequence and the virtual tunnel sequence in the virtual tunnel into the U-type network model, and input the multi-dimensional vector of the objective function to calculate the value of the objective function of the U-type network model;

[0011] The multidimensional vector of the objective function is adjusted to maximize the value of the objective function of the U-shaped network model. When the value of the objective function is maximized, the U-shaped network model corresponding to the multidimensional vector of the objective function is determined to be the trained U-shaped network trail extraction model.

[0012] In one embodiment, the objective function includes:

[0013]

[0014] Where m is the total number of samples of the virtual tunnel sequence and the trail sequence in the virtual tunnel, y (i) represents the category of the i-th sample, x (i) represents the i-th sample, J (θ) represents the probability of the U-shaped network model extracting the trail in the virtual tunnel, h (θ) Represents the multidimensional vector that needs to be adjusted.

[0015] In one embodiment, the process of training the U-shaped network trail extraction model further includes:

[0016] Visualize the process of training the U-shaped network trail extraction model through the deep learning training visualization tool;

[0017] When the parameter variation range of the U-type network model is greater than a preset threshold, the learning rate of the U-type network model during training is adjusted, and the parameters of the U-type network model include weights and bias values.

[0018] In one of the embodiments, when the actual tunnel scene changes, the environmental parameters of the virtual tunnel scene that matches the actual tunnel scene are adjusted so that the virtual tunnel scene after adjusting the environmental parameters matches the changed actual tunnel scene, and the environmental parameters include at least one of lighting parameters, trail color, and trail material map.

[0019] In one embodiment, rendering a pre-created virtual tunnel scene that matches the actual tunnel scene also includes:

[0020] The image of the actual tunnel scene is acquired, and a virtual tunnel scene matching the actual tunnel scene is created through image processing algorithm.

[0021] In a second aspect, the present disclosure also provides a device for extracting a trail in a tunnel, the device comprising:

[0022] A scene rendering module is used to render a pre-created virtual tunnel scene that matches the actual tunnel scene, and obtain a tunnel image sequence after rendering. The tunnel image sequence includes: a virtual tunnel sequence and a trail sequence in the virtual tunnel;

[0023] A model training module is used to input the virtual tunnel sequence and the trail sequence in the virtual tunnel into the U-shaped network model, and obtain the U-shaped network trail extraction model by adjusting the objective function of the U-shaped network model;

[0024] The trail output module is used to input the actual tunnel scene into the U-shaped network trail extraction model, and output the trail in the actual tunnel scene through the U-shaped network trail extraction model.

[0025] In a third aspect, the present disclosure further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0026] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0027] In a fifth aspect, the present disclosure further provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0028] The above-mentioned method, device, computer equipment, and storage medium for extracting trails in tunnels are trained by using virtual tunnel scenes. There is no need to manually shoot tunnel scenes, and then manually mark the trails frame by frame as training samples. The virtual tunnel scenes can be directly rendered to obtain them, which saves a lot of manpower and reduces manual workload. The U-shaped network is trained through the virtual tunnel scene and the trails corresponding to the virtual tunnel scene, and then the objective function is adjusted to obtain the U-shaped network trail extraction model, which can ensure the accuracy of the results extracted by the U-shaped network trail extraction model. While many other networks can only mark rectangular areas, the embodiment scheme provided in the present disclosure can adapt to the U-shaped network, i.e., the Unet network, which can mark networks in irregular areas, and the stability of the Unet network is higher than that of other networks, so the U-shaped network trail extraction model can stably identify trails in real tunnel images, thereby solving the problem of low accuracy in identifying trails, which may cause safety hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of an application environment of a method for extracting a trail in a tunnel in an embodiment;

[0030] Figure 2A schematic diagram of a process for extracting a trail in a tunnel in one embodiment;

[0031] Figure 3 A schematic diagram of a process for obtaining a U-shaped network trail extraction model in one embodiment;

[0032] Figure 4 A schematic diagram of a U-shaped network structure in an embodiment;

[0033] Figure 5 A schematic diagram of a process flow of training the U-shaped network trail extraction model in one embodiment;

[0034] Figure 6 A schematic diagram of a flow chart of a method for extracting a trail in a tunnel in another embodiment;

[0035] Figure 7 It is a schematic block diagram of the structure of a device for extracting a walkway in a tunnel in one embodiment;

[0036] Figure 8 Schematic diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0038] At present, when using track-mounted robots for inspection, there are problems of high cost and large amount of work, and it is necessary to enter the construction site when the new cable tunnel is close to completion. It is difficult to do this work for the already running tunnel. For tunnels longer than 10 kilometers, in addition to the robot body, the cost of track laying accounts for a large proportion, and it also lacks flexibility. Since the track itself also needs to occupy a certain space, it is impossible to build tracks for relatively short tunnels. When using wheeled robots for inspection, although the flexibility is better, there are walkways and stairs of a certain height in the tunnel, so a quadruped robot that can walk directly on the walkway is a better choice, but because the tunnel is relatively narrow, the robot needs to be able to accurately identify the walkway. The traditional recognition method requires the collection of a large number of tunnel images, and the recognition and manual annotation are performed based on the tunnel color, texture and other features. The workload is very huge, but this method has a low recognition rate and poor stability. For tunnel scenes with high recognition accuracy requirements, tens of thousands or even hundreds of thousands of images need to be annotated, and when the scene changes, the tunnel image needs to be re-annotated.

[0039] Therefore, in order to solve the above problems, the present invention provides a method for extracting a trail in a tunnel, which can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the terminal 102 and the server 104 need to process, which may include actual tunnel scene data and a virtual tunnel scene that matches the actual tunnel scene. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 renders a pre-created virtual tunnel scene that is stored in the data storage system and matches the actual tunnel scene. After the terminal 102 renders, a tunnel image sequence is obtained. The tunnel image sequence may include: a virtual tunnel sequence and a trail sequence in the virtual tunnel scene. The terminal 102 inputs the actual tunnel scene in the data storage system into the U-shaped network trail extraction model pre-trained by the server 104, and the U-shaped network trail extraction model pre-trained by the server 104 outputs the trail in the actual tunnel scene. The U-shaped network trail extraction model pre-trained by the server 104 includes: the virtual tunnel sequence obtained after the terminal 102 renders and the trail sequence in the virtual tunnel scene are input into the U-shaped network model, and the model obtained by adjusting the objective function of the U-shaped network model is the U-shaped network trail extraction model. It is understandable that the method can also be applied to a server or a terminal alone. The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0040] In one embodiment, Figure 2 As shown in FIG. 1 , a method for extracting a trail in a tunnel is provided, and the method is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0041] S202, rendering a pre-created virtual tunnel scene that matches the actual tunnel scene, and obtaining a tunnel image sequence after rendering, wherein the tunnel image sequence includes: a virtual tunnel sequence and a trail sequence in the virtual tunnel scene.

[0042] Among them, rendering can be the stage of making the image conform to the 3D scene. Rendering has many applications, such as: each CG software comes with a rendering engine, and there are also such as RenderMan. After the 3D scene is made using 3DS MAX, MAYA and other software for architectural design, animation production, etc., the designed content is made into the final effect map or animation using the application itself or auxiliary applications (lightscape, vray, etc.). The tunnel image sequence can be an image obtained in sequence from the rendered virtual tunnel scene at different times and different orientations. The virtual tunnel scene can be a tunnel scene whose appearance is very close to the real actual tunnel. The trail sequence in the virtual tunnel scene can be an image sequence that only retains the tunnel trail in the virtual tunnel scene, and the rest is hidden.

[0043] Specifically, the rendering requirement set by the terminal 102 is used to render a pre-created virtual tunnel scene that is very close to the actual scene through an image processing algorithm. The image processing algorithm in this embodiment may be an application that can realize the rendering function. After rendering, a tunnel image sequence that meets the rendering requirement is obtained. The rendering requirement may be to sequentially generate a trail sequence in a corresponding virtual tunnel scene from virtual tunnel scenes in different orientations. Therefore, the tunnel image sequence that meets the rendering requirement may include: virtual tunnel sequences in different orientations and trail sequences in virtual tunnel scenes corresponding to virtual tunnels in different orientations, which may include one or more groups of virtual tunnel sequences and trail sequences in virtual tunnel scenes corresponding to virtual tunnels.

[0044] S204, inputting the actual tunnel scene into a pre-trained U-shaped network trail extraction model, and outputting the trail in the actual tunnel scene through the U-shaped network trail extraction model, wherein the U-shaped network trail extraction model comprises: inputting the virtual tunnel sequence and the trail sequence in the virtual tunnel into the U-shaped network model, and obtaining a model by adjusting the objective function of the U-shaped network model.

[0045] The U-shaped network trail extraction model may be a model that can extract actual tunnel scenes. The U-shaped network may be a Unet network, which is a classic network design method and has a large number of applications in image segmentation tasks. The objective function of the U-shaped network model is usually a function that can calculate the probability of extracting trails in a virtual tunnel.

[0046] Specifically, terminal 102 inputs the actual tunnel scene from which trails need to be extracted into the pre-trained U-shaped network trail extraction model, and inputs the scene in which only trail images are retained in the actual tunnel scene through the U-shaped network trail extraction model, thereby extracting the trails in the actual tunnel scene.

[0047] The U-shaped network trail extraction model includes: inputting the virtual tunnel sequence and the trail sequence in the virtual tunnel into the U-shaped network model, i.e., as the training set of the model, and obtaining the model by adjusting the objective function of the U-shaped network model. After the U-shaped network trail extraction model is trained, it has the ability to identify trails from tunnel images. This process can be vividly regarded as an image classification process. The training process of the U-shaped network trail extraction model enables the U-shaped network trail extraction model to have the ability to recognize trails in images. And the trail position identified by the U-shaped network trail extraction model is consistent with the actual trail.

[0048] In the above-mentioned method for extracting trails in tunnels, a virtual tunnel scene is used for training, and there is no need to manually shoot the tunnel scene, and then manually mark the trails frame by frame as training samples. The virtual tunnel scene can be directly rendered to obtain the trails, which saves a lot of manpower and reduces the workload of manual labor. The U-shaped network is trained through the virtual tunnel scene and the trails corresponding to the virtual tunnel scene, and then the objective function is adjusted to obtain the U-shaped network trail extraction model, which can ensure the accuracy of the results extracted by the U-shaped network trail extraction model. Many other networks can only annotate rectangular areas, while the U-shaped network, namely Unet, is a network that can annotate irregular areas, and its stability is higher than other networks. Therefore, the U-shaped network trail extraction model can stably identify trails in real tunnel images, thereby solving the problem of low accuracy in identifying trails, which may cause safety hazards.

[0049] The above embodiment only roughly describes how the U-shaped network trail extraction model is obtained. The following is a detailed description of the training process of the U-shaped network trail extraction model. In one embodiment, Figure 3 As shown, the U-shaped network trail extraction model is obtained by the following methods:

[0050] S302, inputting the trail sequence in the virtual tunnel and the virtual tunnel sequence into a U-shaped network model, and inputting a multi-dimensional vector of an objective function to calculate the value of the objective function of the U-shaped network model.

[0051] The multidimensional vector of the objective function may be a variable that needs to be adjusted in the objective function, and the output value of the objective function may be changed by adjusting the variable.

[0052] S304, adjusting the multidimensional vector of the objective function so that the value of the objective function of the U-shaped network model is maximized. When the value of the objective function is maximized, the U-shaped network model corresponding to the multidimensional vector of the objective function is determined to be the U-shaped network trail extraction model obtained through training.

[0053] Specifically, the U-type network model is essentially a mathematical calculation formula. A fixed input will get a fixed output, and different outputs will be obtained by adjusting different multidimensional vectors. In this embodiment, the input of the U-type network model is the trail sequence in the virtual tunnel and the corresponding virtual tunnel sequence, and the output is also a picture sequence. When the U-type network model is not trained, the trail sequence of the virtual tunnel sequence corresponding to the input virtual tunnel sequence output will be very different from the trail sequence of the virtual tunnel sequence obtained by actual rendering, and this difference can be calculated through the objective function. Each time the virtual tunnel sequence output and the trail sequence in the virtual tunnel are input, a difference will be obtained. The multidimensional vector in the objective function can be continuously adjusted by the gradient descent method to make this difference optimal, that is, to maximize the objective function value and maximize the probability of correct classification. When the value of the objective function is the largest, the original multidimensional vector of the U-type network model is replaced with the multidimensional vector that maximizes the value of the objective function. The replaced U-type network model is the U-type network trail extraction model.

[0054] U-shaped network structure diagram, such as Figure 4 As shown in the figure, it consists of a contracting path (left) and an expansive path (right). The contracting path is used to obtain context information, and the expansive path is used for precise positioning. The two paths are symmetrical to each other. The contracting path follows the typical architecture of a convolutional network. It consists of repeated use of two 3x3 convolutions (unfilled convolutions), each followed by a rectified linear unit (ReLU) and a 2x2 max pooling operation, and downsampling with a stride of 2. In each downsampling step, we double the number of feature channels. The expanding path also uses the same arrangement pattern. Each step includes upsampling of the feature map, performing a 2x2 convolution ("deconvolution"), halving the number of feature channels, doubling the size of the feature map, and concatenating with the corresponding clipped feature map from the contracting path, followed by two 3x3 convolutions, each followed by a ReLU. Since each convolution will lose boundary pixels, clipping is necessary, and they are spliced ​​after clipping. In the last layer, a 1x1 convolution is used to map each resulting feature vector to the number of specific categories required, resulting in the number of classifications (binary classification).

[0055] like Figure 4As shown, in some embodiments, when the size of the input image is 572×572, after a 3×3 convolution, a linear correction is performed using a linear rectified unit (ReLU). After each 3×3 convolution, the size of the image is reduced by 2. After two convolutions in the first layer, the second layer is entered, and a 2x2 maximum pooling operation is required at this time, and downsampling with a step size of 2 is performed. After each downsampling, the size of the feature channel is doubled, and the maximum value of the feature in the image is calculated, thereby obtaining an image of 284×284 size, and then the 284×284 size image is used for subsequent training, and then two 3×3 convolution operations are continued. The above operations are repeated until an image of a preset standard size is obtained. Then, the obtained image of the preset standard size is deconvolved. The deconvolution is used to halve the number of feature channels and double the size of the feature map, that is, the size of the obtained image is multiplied by 2. The deconvolution operation corresponds to the maximum pooling operation. After each deconvolution operation, a copy and cut is performed to fuse the features of the image on the left and the corresponding image on the right. The copy and cut is to make the length and width of the two consistent. Then, two corresponding 3×3 convolution operations are performed. The top layer of the right output image is convolved with 1×1, which is mainly used to classify the image and convert the obtained feature channels into the number of specific classifications to obtain the classification results.

[0056] When the actual tunnel scene extraction effect of the trained U-shaped network trail extraction model is not good, such as inaccurate recognition, it is necessary to carefully check whether there are errors in the trail sequence in the virtual tunnel in the training data.

[0057] In some embodiments, if there are many trail sequences in a virtual tunnel and corresponding virtual tunnel sequences, which form a collection, the probability of identifying the correct trail when the collection is trained is the product of the trail sequence in each virtual tunnel in the collection and the value of the objective function calculated for the corresponding virtual tunnel sequence. The multidimensional vector of the objective function is adjusted to maximize the product of the value of the objective function, and a U-shaped network trail extraction model is obtained. The specific steps for obtaining the U-shaped network trail extraction model can be referred to the above embodiment, and will not be elaborated here.

[0058] In this embodiment, the probability of the trail to be identified can be solved through the objective function, and the greater the probability, the better. When the probability is the largest, it proves that the corresponding multidimensional vector can make the model recognition effect the best, thereby obtaining the model with the best recognition effect. And because the input training data is the trail sequence in the virtual tunnel and the corresponding virtual tunnel sequence, the training data is accurate, so the U-shaped network trail extraction trail after training can achieve a better extraction effect.

[0059] In one embodiment, the objective function includes:

[0060]

[0061] Wherein, m is the total number of samples of the virtual tunnel sequence and the trail sequence in the virtual tunnel, y (i) represents the category of the i-th sample, x (i) represents the i-th sample, J (θ) represents the probability of the U-shaped network model extracting the trail in the virtual tunnel, h (θ) Represents the multidimensional vector that needs to be adjusted. Satisfying J (θ) The largest h (θ) The value is the multidimensional vector corresponding to the model we need to solve.

[0062] In the above embodiment, it is mentioned how the U-shaped network trail extraction model is obtained. In this embodiment, the other parts of the processing process of training the U-shaped network trail extraction model are described below. In one embodiment, Figure 5 As shown, the process of training the U-shaped network trail extraction model also includes:

[0063] S502, visualizing the process of training the U-shaped network trail extraction model through a deep learning training visualization tool.

[0064] Among them, the deep learning training visualization tool can be a tool that can visualize the model training process and can visualize the dynamic change process when training the model.

[0065] Specifically, the process of training the U-shaped network trail extraction model is visualized through deep learning training visualization tools such as CNN Explainer and Visual DL. It is possible to intuitively feel how the U-shaped network gradually fits the data during the model training process, and to see the gradient descent process, the direction of the loss function, and the accuracy of the model during the model training process.

[0066] S504, when the parameter variation range of the U-type network model is greater than a preset threshold, adjusting the learning rate of the U-type network model during training, the parameters of the U-type network model including weights and bias values.

[0067] The parameters of the U-shaped network model can be the variables that need to be trained in the model. The learning rate is a hyperparameter set manually, which is mainly used to control the speed of adjusting the model parameters during the training phase. The bias value can be a value that enables the U-shaped network model to correctly classify. The weight is the value obtained by training the U-shaped network model, and the purpose is to allow the U-shaped network model to learn useful information during the training process.

[0068] Specifically, if a very large learning rate is used to train the model, the loss value will always be at a relatively large position and the model cannot converge. If a relatively large learning rate is used to train the model, the loss value will drop very quickly and ultimately a relatively small loss value cannot be obtained, so the result is not ideal. If a relatively small learning rate is used to train the model, the model will converge very slowly and it will take a long time for the model to converge. Therefore, after visualization through the deep learning training visualization tool, when the parameter variation range of the U-shaped network model is greater than the preset threshold, it proves that the learning rate is set too small or too large. It can be discovered and adjusted in time during the training process, which can improve the training efficiency of the network and shorten the training time.

[0069] In one embodiment, when the actual tunnel scene changes, it is necessary to extract a completely new tunnel. At this time, when the U-shaped network trail extraction model trained by the original unchanged scene is used, the extraction effect will be poor, so it is necessary to create a new virtual scene corresponding to the changed time tunnel scene. Many scenes in the cable tunnel are similar, so the previously produced virtual tunnel scene is a one-time production and repeated use. When the actual tunnel scene changes, it is only necessary to adjust the environmental parameters of the previously produced virtual tunnel scene, such as lighting parameters, trail color, trail material map, etc. At least one of the above-mentioned parameters needs to be adjusted. It should be noted that the environmental parameters here are only exemplified by the above three types. Technical personnel in this field can adjust other parameters in the actual process of adjusting the virtual tunnel scene so that the adjusted virtual tunnel scene is the same as the changed actual tunnel scene.

[0070] The virtual tunnel scene after adjusting the environmental parameters is matched with the actual tunnel scene after the change, so as to obtain the corresponding virtual tunnel scene, and then the U-shaped network trail extraction model mentioned in the above embodiment is used, and the trail extraction method using the U-shaped network trail extraction model is used to extract the trail in the changed actual tunnel scene. For the U-shaped network trail extraction model and the trail extraction method using the U-shaped network trail extraction model, please refer to the above embodiment, which will not be repeated here.

[0071] In some implementations, for example, for an actual tunnel scene with dim lighting, it is only necessary to adjust the lighting parameters in the virtual scene, and when the color material of the trail in the actual tunnel scene changes, it is only necessary to change the trail color and material mapping.

[0072] In this embodiment, by adjusting the link parameters, when the actual tunnel scene changes, the previously created virtual scene is reused and the corresponding environmental parameters are adjusted so that the adjusted virtual scene corresponds to the changed actual tunnel scene, thereby saving workload.

[0073] In some embodiments, the rendering of a pre-created virtual tunnel scene that matches the actual tunnel scene also includes:

[0074] An image of an actual tunnel scene is acquired, and a virtual tunnel scene matching the actual tunnel scene is created through an image processing algorithm.

[0075] The image processing algorithm may be an algorithm or application for creating a 3D scene. In this embodiment, the image processing algorithm may be a 3Dmax application, which acquires an image of an actual tunnel scene and creates a virtual tunnel scene close to the actual tunnel scene through the 3Dmax application. The image of the actual tunnel scene may be an image taken by a quadruped robot walking in a tunnel.

[0076] In this embodiment, if a virtual scene is not produced, the tunnel scene can only be photographed manually, and then the trails in each photographed scene must be manually marked frame by frame, which will increase the workload. Creating a virtual tunnel scene that matches the actual tunnel scene through an image processing algorithm can solve the current problem of heavy workload in manual collection and labeling, reduce the workload, and thus improve the efficiency of identifying trails in tunnels.

[0077] In another embodiment, the present disclosure also provides a method for extracting a trail in a tunnel, such as Figure 6 As shown, the method includes:

[0078] S602, acquiring an image of an actual tunnel scene, and creating a virtual tunnel scene matching the actual tunnel scene through an image processing algorithm.

[0079] S604, rendering a pre-created virtual tunnel scene that matches the actual tunnel scene, and obtaining a tunnel image sequence after rendering, wherein the tunnel image sequence includes: a virtual tunnel sequence and a trail sequence in the virtual tunnel scene.

[0080] S606, inputting the virtual tunnel sequence and the trail sequence in the virtual tunnel into a U-shaped network model, and obtaining a U-shaped network trail extraction model by adjusting the objective function of the U-shaped network model.

[0081] S608, inputting the actual tunnel scene into the pre-trained U-shaped network trail extraction model, and outputting the trail in the actual tunnel scene through the U-shaped network trail extraction model.

[0082] For the specific implementation of this embodiment, please refer to the above embodiment and will not be elaborated in detail here.

[0083] It should be understood that, although the steps in the flowcharts in the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts of the accompanying drawings may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0084] In one embodiment, Figure 7 As shown, a device 700 for extracting a trail in a tunnel is provided, comprising: a scene rendering module 702, a model training module 704 and a trail output module 706, wherein:

[0085] The scene rendering module 702 is used to render a pre-created virtual tunnel scene that matches the actual tunnel scene, and obtain a tunnel image sequence after rendering. The tunnel image sequence includes: a virtual tunnel sequence and a trail sequence in the virtual tunnel.

[0086] The model training module 704 is used to input the virtual tunnel sequence and the trail sequence in the virtual tunnel into the U-shaped network model, and obtain the U-shaped network trail extraction model by adjusting the objective function of the U-shaped network model.

[0087] The trail output module 706 is used to input the actual tunnel scene into the U-shaped network trail extraction model, and output the trail in the actual tunnel scene through the U-shaped network trail extraction model.

[0088] In one embodiment of the device, the model training module 704 includes: an objective function calculation module, an objective function adjustment module and a trail model determination module, wherein:

[0089] The objective function calculation module is used to input the trail sequence in the virtual tunnel and the virtual tunnel sequence into the U-shaped network model, and input the multi-dimensional vector of the objective function to calculate the value of the objective function of the U-shaped network model.

[0090] The objective function adjustment module is used to adjust the multidimensional vector of the objective function so as to maximize the value of the objective function of the U-shaped network model.

[0091] The trail model determination module is used to determine the U-shaped network model corresponding to the multidimensional vector of the objective function as the trained U-shaped network trail extraction model when the value of the objective function is the largest.

[0092] In one embodiment of the apparatus, the objective function comprises:

[0093]

[0094] Wherein, m is the total number of samples of the virtual tunnel sequence and the trail sequence in the virtual tunnel, y (i) represents the category of the i-th sample, x (i) represents the i-th sample, J (θ) represents the probability of the U-shaped network model extracting the trail in the virtual tunnel, h (θ) Represents the multidimensional vector that needs to be adjusted.

[0095] In one embodiment of the device, the model training module 704 is also used to visualize the process of training the U-shaped network trail extraction model through a deep learning training visualization tool; when the parameter variation range of the U-shaped network model is greater than a preset threshold, the learning rate of the U-shaped network model during training is adjusted, and the parameters of the U-shaped network model include weights and bias values.

[0096] In one embodiment of the device, the device also includes: a scene adjustment module, which is used to adjust the environmental parameters of the virtual tunnel scene that matches the actual tunnel scene when the actual tunnel scene changes, so that the virtual tunnel scene after adjusting the environmental parameters matches the actual tunnel scene after the change, and the environmental parameters include at least one of lighting parameters, trail color, and trail material map.

[0097] In one embodiment of the device, the device further comprises: a scene creation module, which is used to acquire an image of an actual tunnel scene and create a virtual tunnel scene matching the actual tunnel scene through an image processing algorithm.

[0098] The specific implementation of the device for extracting the trail in the tunnel can be referred to the embodiment of the method for extracting the trail in the tunnel described above, which will not be described in detail here. Each module in the above-mentioned device for extracting the trail in the tunnel can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0099] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for extracting a trail in a tunnel is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0100] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0101] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0102] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0103] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0104] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0105] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The above-mentioned embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the patent of the present disclosure shall be subject to the attached claims.

Claims

1. A method for extracting trails in tunnels, It is characterized in that The method comprises: Rendering a pre-created virtual tunnel scene that matches the actual tunnel scene, and obtaining a tunnel image sequence after rendering, wherein the tunnel image sequence includes: a virtual tunnel sequence and a trail sequence in the virtual tunnel scene; Inputting the actual tunnel scene into a pre-trained U-shaped network trail extraction model, and outputting the trail in the actual tunnel scene through the U-shaped network trail extraction model, wherein the U-shaped network trail extraction model comprises: inputting the virtual tunnel sequence and the trail sequence in the virtual tunnel into the U-shaped network model, and obtaining a model by adjusting the objective function of the U-shaped network model; The objective function includes: Wherein, m is the total number of samples of the virtual tunnel sequence and the trail sequence in the virtual tunnel, y (i) represents the category of the i-th sample, x (i) represents the i-th sample, J (θ) represents the probability of the U-shaped network model extracting the trail in the virtual tunnel, h (θ) Represents the multidimensional vector that needs to be adjusted.

2. The method for extracting a trail in a tunnel according to claim 1, It is characterized in that The U-shaped network trail extraction model is obtained by the following methods: Inputting the trail sequence in the virtual tunnel and the virtual tunnel sequence into a U-shaped network model, and inputting a multidimensional vector of an objective function to calculate a value of an objective function of the U-shaped network model; The multidimensional vector of the objective function is adjusted to maximize the value of the objective function of the U-shaped network model. When the value of the objective function is maximized, the U-shaped network model corresponding to the multidimensional vector of the objective function is determined to be the U-shaped network trail extraction model obtained through training.

3. The method for extracting a trail in a tunnel according to claim 1, It is characterized in that The process of training the U-shaped network trail extraction model also includes: Visualize the process of training the U-shaped network trail extraction model through a deep learning training visualization tool; When the parameter variation range of the U-shaped network model is greater than a preset threshold, the learning rate of the U-shaped network model during training is adjusted, and the parameters of the U-shaped network model include weights and bias values.

4. The method for extracting a trail in a tunnel according to claim 1, It is characterized in that When the actual tunnel scene changes, the environmental parameters of the virtual tunnel scene that matches the actual tunnel scene are adjusted so that the virtual tunnel scene after adjusting the environmental parameters matches the changed actual tunnel scene, and the environmental parameters include at least one of lighting parameters, trail color, and trail material mapping.

5. The method for extracting a trail in a tunnel according to claim 1, It is characterized in that The rendering of the pre-created virtual tunnel scene matching the actual tunnel scene also includes: An image of an actual tunnel scene is acquired, and a virtual tunnel scene matching the actual tunnel scene is created through an image processing algorithm.

6. A device for extracting a walkway in a tunnel, It is characterized in that The device comprises: A scene rendering module is used to render a pre-created virtual tunnel scene that matches the actual tunnel scene, and obtain a tunnel image sequence after rendering, wherein the tunnel image sequence includes: a virtual tunnel sequence and a trail sequence in the virtual tunnel; A model training module, used for inputting the virtual tunnel sequence and the trail sequence in the virtual tunnel into a U-shaped network model, and obtaining a U-shaped network trail extraction model by adjusting the objective function of the U-shaped network model; A trail output module, used to input the actual tunnel scene into the U-shaped network trail extraction model, and output the trail in the actual tunnel scene through the U-shaped network trail extraction model; The objective function includes: Wherein, m is the total number of samples of the virtual tunnel sequence and the trail sequence in the virtual tunnel, y (i) represents the category of the i-th sample, x (i) represents the i-th sample, J (θ) represents the probability of the U-shaped network model extracting the trail in the virtual tunnel, h (θ) Represents the multidimensional vector that needs to be adjusted.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images