Cable tunnel navigation path identification method and device, terminal equipment and storage medium

Abstract

The invention discloses a cable tunnel navigation path identification method and apparatus, a terminal device and a storage medium. The method comprises the steps of obtaining an original cable tunnel image of a to-be-identified cable tunnel; inputting the original cable tunnel image into a preset road boundary detection model to obtain a cable tunnel image with a road boundary frame; inputting the cable tunnel image into a preset image segmentation model to obtain a road boundary image in the to-be-identified cable tunnel; processing the road boundary image to obtain coordinate points of the road boundary; fitting according to a least square method and the coordinate points to obtain a road navigation line; and finally, coordinate information of the road navigation line is extracted, the road navigation line is drawn on the original cable tunnel image according to the coordinate information, and a navigation path of the cable tunnel to be identified is generated. By implementing the method and the device, the equipment cost of navigation path identification can be reduced while the calculation precision is met.

Description

Technical Field

[0001] The present invention relates to the technical field of path recognition, and particularly to a cable tunnel navigation path recognition method, device, terminal device, and storage medium. Background Art

[0002] In recent years, with the rapid growth of the volume of high-voltage cables, the manpower required to complete the necessary inspection work has increased significantly. Therefore, the intelligent replacement of inspection work is essential. Complementary to the fixed online monitoring equipment responsible for real-time monitoring of important nodes all day long, inspection robots can meet mobile monitoring work such as full-length temperature monitoring of cables and tunnel environment monitoring.

[0003] In the prior art, lidar is usually used to navigate and position a robot dog. The lidar needs to quickly acquire a large amount of laser point cloud data, usually generating hundreds of thousands to millions of points per second. These data need to be processed in real time to form a three-dimensional model of the environment. Therefore, in order to avoid recognition errors, a processor with high precision and high computing power is required to process it, and then the equipment cost is relatively high. Summary of the Invention

[0004] The present invention provides a cable tunnel navigation path recognition method, device, terminal device, and storage medium, which can reduce the equipment cost of the entire navigation path recognition process while meeting the calculation accuracy.

[0005] An embodiment of the present invention provides a cable tunnel navigation path recognition method, including:

[0006] Obtain the original cable tunnel image of the cable tunnel to be recognized; wherein, the original cable tunnel image carries the road information in the cable tunnel to be recognized, and the original cable tunnel image is obtained through a vision sensor;

[0007] Input the original cable tunnel image into a preset road boundary detection model, so that the preset road boundary detection model detects the road boundary in the original cable tunnel image to obtain a cable tunnel image with a road boundary box; wherein, the preset road boundary detection model is constructed based on the improved YOLOv8 network, and the improved YOLOv8 network is a YOLOv8 network with a multi-head self-attention module;

[0008] Input the cable tunnel image into a preset image segmentation model, so that the image segmentation model performs image segmentation on the cable tunnel image according to the road boundary box to obtain the road boundary image in the cable tunnel to be recognized;

[0009] Process the road boundary image to obtain the coordinate points of the road boundary;

[0010] According to the least squares method and the above coordinate points, a road navigation line is fitted;

[0011] Extract the coordinate information of the above road navigation line, and draw the above road navigation line on the above original cable tunnel image according to the above coordinate information to generate the navigation path of the above cable tunnel to be recognized.

[0012] Furthermore, the training of the above preset road boundary detection model includes:

[0013] Obtain a number of cable tunnel sample images with real labels; wherein, the above real label is a real road boundary box used to represent the road boundary information in the above cable tunnel sample image;

[0014] Input the above cable tunnel sample images into the road boundary detection model to be trained, so that the road boundary detection model detects the road boundary in the above cable tunnel sample images to obtain detected road boundary boxes;

[0015] Calculate the loss function value according to the above detected road boundary boxes and the above real labels;

[0016] Judge whether the current training times are not less than the preset training times; if so, the road boundary detection model training is completed to obtain the above preset road boundary detection model; otherwise, adjust the parameters in the road boundary detection model according to the above loss function value and continue to train the road boundary detection model.

[0017] Furthermore, the above preset road boundary detection model includes: a convolutional layer, a C2f network, a multi-head self-attention network, a neck network, and a head network;

[0018] The above inputting the above original cable tunnel image into the preset road boundary detection model, so that the preset road boundary detection model detects the road boundary in the above original cable tunnel image to obtain a cable tunnel image with a road boundary box, includes:

[0019] Input the above original cable tunnel image into the preset road boundary detection model for preprocessing to obtain a preprocessed first cable tunnel image, and input the above first cable tunnel image into the above convolutional layer;

[0020] The above convolutional layer performs convolution on the above first cable tunnel image, extracts a first cable tunnel feature map with the local features of the above first cable tunnel image, and inputs the above first cable tunnel feature map into the above C2f network;

[0021] The above C2f network enhances the features of the above first cable tunnel feature map through gradient flow optimization to obtain a second cable tunnel feature map, and inputs the above second cable tunnel feature map into the above multi-head self-attention network;

[0022] After the above multi-head self-attention network performs linear transformation and mapping on the above second cable tunnel feature map, a query matrix, a key matrix, and a value matrix are obtained;

[0023] According to the above query matrix, key matrix, value matrix, and the multi-head attention mechanism, a feature fusion operation is performed to obtain a third cable tunnel feature map with enhanced features, and the above third cable tunnel feature map is input into the above neck network;

[0024] The above neck network sequentially performs multi-scale feature extraction and feature fusion on the above third cable tunnel feature map to obtain a fourth cable tunnel feature map, and inputs the above fourth cable tunnel feature map into the above head network;

[0025] The above head network performs object detection targeting the above road boundary on the above fourth cable tunnel feature map to obtain a first cable tunnel image with several road bounding boxes; among them, the confidence levels of different road bounding boxes are different; the above road bounding boxes include road boundary information in the above first cable tunnel image;

[0026] The road bounding box with the highest confidence level is retained as the target road bounding box, and the coordinates of the above target road bounding box are calculated according to the road boundary information in the above first cable tunnel image;

[0027] According to the above target road bounding box and the above coordinates, the above cable tunnel image is obtained.

[0028] Further, the above inputting the above original cable tunnel image into a preset road boundary detection model for preprocessing to obtain a preprocessed first cable tunnel image includes:

[0029] Adjust the size of the above original cable tunnel image to a preset image size to obtain a first original cable tunnel image;

[0030] Normalize the above first original cable tunnel image to obtain the above first cable tunnel image.

[0031] Further, the above processing the above road boundary image to obtain the coordinate points of the above road boundary includes:

[0032] Perform mask detection on the above road boundary image to obtain a mask array of the above road boundary;

[0033] Identify the above mask array to obtain a matte image of the above road boundary;

[0034] Extract the contour point array of the road boundary on the above mask image to obtain the above coordinate points.

[0035] Further, the above road navigation line is fitted according to the least squares method and the above coordinate points, including:

[0036] Extract the left coordinate points on the left boundary of the above road boundary and the right coordinate points on the right boundary of the above road boundary from the above coordinate points;

[0037] According to the above left coordinate points and the least squares method, calculate the first fitting parameter when the sum of the squares of the differences between the ordinate of the above left coordinate points and the fitted ordinate of the left coordinate points is minimized; wherein, the fitted ordinate of the above left coordinate points is obtained by fitting the above left coordinate points according to the least squares method;

[0038] According to the above right coordinate points and the least squares method, calculate the second fitting parameter when the sum of the squares of the differences between the ordinate of the above right coordinate points and the fitted ordinate of the right coordinate points is minimized; wherein, the fitted ordinate of the above right coordinate points is obtained by fitting the above right coordinate points according to the least squares method;

[0039] Calculate the first fitted coordinates of each point on the above left boundary and the second fitted coordinates of each point on the above right boundary respectively according to the above first fitting parameter and the above second fitting parameter;

[0040] Calculate the average abscissa according to the abscissa of the above first fitted coordinates and the abscissa of the second fitted coordinates, and use the above average abscissa as the navigation fitted abscissa of the above road navigation line;

[0041] Calculate the average ordinate according to the ordinate of the above first fitted coordinates and the ordinate of the second fitted coordinates, and use the above average ordinate as the navigation fitted ordinate of the above road navigation line;

[0042] Generate the above road navigation line according to the above navigation fitted abscissa and the above navigation fitted ordinate.

[0043] Based on the above method item embodiment, the present invention correspondingly provides an apparatus item embodiment;

[0044] The present invention provides a cable tunnel navigation path recognition device, including:

[0045] An image acquisition module, a road boundary detection module, a mask array acquisition module, a coordinate point calculation module, a road navigation line fitting module, and a navigation path generation module;

[0046] The above-mentioned image acquisition module is used to acquire the original cable tunnel image of the cable tunnel to be recognized; wherein, the above-mentioned original cable tunnel image carries the road information in the above-mentioned cable tunnel to be recognized, and the above-mentioned original cable tunnel image is acquired through a vision sensor;

[0047] The above-mentioned road boundary detection module is used to input the above-mentioned original cable tunnel image into a preset road boundary detection model, so that the preset road boundary detection model detects the road boundary in the above-mentioned original cable tunnel image to obtain a cable tunnel image with a road boundary box; wherein, the above-mentioned preset road boundary detection model is constructed based on the improved YOLOv8 network, and the above-mentioned improved YOLOv8 network is a YOLOv8 network with a multi-head self-attention module;

[0048] The above-mentioned mask array acquisition module is used to input the above-mentioned cable tunnel image into a preset image segmentation model, so that the image segmentation model performs image segmentation on the above-mentioned cable tunnel image according to the above-mentioned road boundary box to obtain the road boundary image in the above-mentioned cable tunnel to be recognized;

[0049] The above-mentioned coordinate point calculation module is used to process the above-mentioned road boundary image to obtain the coordinate points of the above-mentioned road boundary;

[0050] The above-mentioned road navigation line fitting module is used to fit a road navigation line according to the least squares method and the above-mentioned coordinate points;

[0051] The above-mentioned navigation path generation module is used to extract the coordinate information of the above-mentioned road navigation line and draw the above-mentioned road navigation line on the above-mentioned original cable tunnel image according to the above-mentioned coordinate information to generate the navigation path of the above-mentioned cable tunnel to be recognized.

[0052] Further, the above-mentioned road boundary detection module includes:

[0053] A sample image acquisition unit, a sample image road boundary detection unit, a loss function value calculation unit, and a training times judgment unit;

[0054] The above-mentioned sample image acquisition unit is used to acquire a number of cable tunnel sample images with real labels; wherein, the above-mentioned real label is a real road boundary box used to represent the road boundary information in the above-mentioned cable tunnel sample image;

[0055] The above-mentioned sample image road boundary detection unit is used to input the above-mentioned cable tunnel sample image into a road boundary detection model to be trained, so that the road boundary detection model detects the road boundary in the above-mentioned cable tunnel sample image to obtain a detected road boundary box;

[0056] The above loss function value calculation unit is used to calculate the loss function value according to the above detected road bounding box and the above ground truth label;

[0057] The above training times judgment unit is used to judge whether the current training times is not less than the preset training times; if so, the road boundary detection model training is completed to obtain the above preset road boundary detection model; otherwise, the parameters in the above road boundary detection model are adjusted according to the above loss function value, and the above road boundary detection model continues to be trained.

[0058] Based on the above method item embodiments, the present invention correspondingly provides a terminal device item embodiment;

[0059] The present invention provides a terminal device, including a processor, a memory, and a computer program stored in the above memory and configured to be executed by the above processor. When the above processor executes the above computer program, it implements the above cable tunnel navigation path recognition method according to any embodiment of the present invention.

[0060] Based on the above method item embodiments, the present invention correspondingly provides a storage medium item embodiment;

[0061] The present invention provides a storage medium, including a processor, a memory, and a computer program stored in the above memory and configured to be executed by the above processor. When the above processor executes the above computer program, it implements the above cable tunnel navigation path recognition method according to any embodiment of the present invention.

[0062] The embodiments of the present invention have the following beneficial effects:

[0063] The present invention provides a method, apparatus, terminal device and storage medium for identifying a navigation path of a cable tunnel. The above method includes: obtaining an original cable tunnel image of the cable tunnel to be identified; wherein, the above original cable tunnel image carries road information in the above cable tunnel to be identified, and the above original cable tunnel image is obtained through a vision sensor; subsequently, inputting the above original cable tunnel image into a preset road boundary detection model, so that the above preset road boundary detection model detects the road boundary in the above original cable tunnel image to obtain a cable tunnel image with a road boundary box; wherein, the above preset road boundary detection model is constructed based on the improved YOLOv8 network, and the above improved YOLOv8 network is a YOLOv8 network with a multi-head self-attention module; then inputting the above cable tunnel image into a preset image segmentation model, so that the above image segmentation model performs image segmentation on the above cable tunnel image according to the above road boundary box to obtain a road boundary image in the above cable tunnel to be identified; processing the above road boundary image to obtain coordinate points of the above road boundary; subsequently, fitting a road navigation line according to the least squares method and the above coordinate points; finally, extracting coordinate information of the above road navigation line and drawing the above road navigation line on the above original cable tunnel image according to the above coordinate information to generate a navigation path of the above cable tunnel to be identified. Therefore, the present invention obtains an original cable tunnel image through a vision sensor, and then uses a preset road boundary detection model constructed based on the improved YOLOv8 network to process the cable tunnel image, identifies the road boundary of the cable tunnel, then performs image segmentation on it and uses the least squares method for fitting, and finally obtains a navigation path. Therefore, the present invention uses a vision sensor to replace obtaining the original cable tunnel image to obtain the image inside the cable tunnel, and at the same time performs related image processing based on the road boundary detection model constructed based on the YOLOv8 network with the characteristics of small volume and fast calculation speed, reducing the equipment cost of the entire navigation path identification process while meeting the calculation accuracy. Description of the Drawings

[0064] Figure 1 FIG. is a schematic flowchart of a method for identifying a navigation path of a cable tunnel provided by an embodiment of the present invention.

[0065] Figure 2 FIG. is a schematic structural diagram of a preset road boundary detection model provided by an embodiment of the present invention.

[0066] Figure 3 FIG. is a schematic diagram of the road boundary detection result before improvement provided by an embodiment of the present invention.

[0067] Figure 4 FIG. is a schematic diagram of the road boundary detection result after improvement provided by an embodiment of the present invention.

[0068] Figure 5 It is an iterative graph of algorithm network performance metrics provided by an embodiment of the present invention.

[0069] Figure 6 It is a road boundary mask image before improvement provided by an embodiment of the present invention.

[0070] Figure 7 It is a road boundary mask image after improvement provided by an embodiment of the present invention.

[0071] Figure 8 It is a schematic diagram of a navigation path before improvement provided by an embodiment of the present invention.

[0072] Figure 9 It is a schematic diagram of a navigation path after improvement provided by an embodiment of the present invention.

[0073] Figure 10 It is a schematic structural diagram of a cable tunnel navigation path recognition device provided by an embodiment of the present invention. Detailed implementation manners

[0074] Next, the technical solutions in the present invention will be clearly and completely described in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0075] As Figure 1 shown, a cable tunnel navigation path recognition method provided by an embodiment of the present invention includes:

[0076] Step S101: Obtain the original cable tunnel image of the cable tunnel to be recognized; wherein, the above original cable tunnel image carries the road information in the above cable tunnel to be recognized, and the above original cable tunnel image is obtained through a vision sensor;

[0077] Specifically, obtain the actual scene image of the internal environment of the cable tunnel to be recognized through a vision sensor, and use this actual scene image as the above original cable tunnel image. The actual scene image needs to include the road information inside the cable tunnel.

[0078] Step S102: Input the above original cable tunnel image into a preset road boundary detection model, so that the preset road boundary detection model detects the road boundary in the above original cable tunnel image to obtain a cable tunnel image with a road boundary box; wherein, the above preset road boundary detection model is constructed based on the improved YOLOv8 network, and the above improved YOLOv8 network is a YOLOv8 network with a multi-head self-attention module;

[0079] In a preferred embodiment, the training of the above-mentioned preset road boundary detection model includes:

[0080] Obtain a number of cable tunnel sample images with real labels; wherein, the above real labels are real road boundary boxes used to represent the road boundary information in the cable tunnel sample images;

[0081] Input the above cable tunnel sample images into the road boundary detection model to be trained, so that the road boundary detection model detects the road boundaries in the cable tunnel sample images and obtains detected road boundary boxes;

[0082] Calculate the loss function value according to the above detected road boundary boxes and the above real labels;

[0083] Judge whether the current training times are not less than the preset training times; if so, the road boundary detection model training is completed to obtain the above-mentioned preset road boundary detection model; otherwise, adjust the parameters in the road boundary detection model according to the above loss function value and continue to train the road boundary detection model.

[0084] Specifically, collect the actual scene video of the internal environment of the cable tunnel through the visual sensor on the camera, and perform frame division on the video to obtain tunnel sample images with the internal road information of the cable tunnel.

[0085] Specifically, use the image augmentation method to augment the data of these tunnel sample images, use labelimg to annotate the road boundary information in the augmented images, and finally divide the annotated images into a training set and a test set. For example, after augmentation, 92 tunnel sample images are obtained. After dividing the training set and the test set, the above-mentioned cable tunnel sample images for training are obtained, that is, 82 in the training set and 10 in the test set for testing the preset road boundary detection model obtained after training.

[0086] Specifically, when training the road boundary detection model, the GPU used has 8G of memory RTX4060, the preset training times are set to 100, the Batch size (batch size) is set to 4, and the initial learning rate is 0.01.

[0087] Specifically, for each training, the loss function value will be calculated once, and when the current training times are less than the preset training times, the parameters in the road boundary detection model will be adjusted and optimized once according to the current loss function value, and then the next training will be carried out.

[0088] In this preferred embodiment, a preset road boundary detection model is obtained by training the road boundary detection model.

[0089] In another preferred embodiment, the above-mentioned preset road boundary detection model includes: a convolutional layer, a C2f network, a multi-head self-attention network, a neck network, and a head network;

[0090] The above-mentioned step of inputting the above-mentioned original cable tunnel image into the preset road boundary detection model to enable the preset road boundary detection model to detect the road boundary in the above-mentioned original cable tunnel image and obtain a cable tunnel image with a road boundary box includes:

[0091] Input the above-mentioned original cable tunnel image into the preset road boundary detection model for preprocessing to obtain a first cable tunnel image after preprocessing, and input the above-mentioned first cable tunnel image into the above-mentioned convolutional layer;

[0092] The above-mentioned convolutional layer performs convolution on the above-mentioned first cable tunnel image to extract a first cable tunnel feature map with the local features of the above-mentioned first cable tunnel image, and input the above-mentioned first cable tunnel feature map into the above-mentioned C2f network;

[0093] The above-mentioned C2f network enhances the features of the above-mentioned first cable tunnel feature map through gradient flow optimization to obtain a second cable tunnel feature map, and input the above-mentioned second cable tunnel feature map into the above-mentioned multi-head self-attention network;

[0094] After the above-mentioned multi-head self-attention network performs linear transformation and mapping on the above-mentioned second cable tunnel feature map, a query matrix, a key matrix, and a value matrix are obtained;

[0095] According to the above-mentioned query matrix, key matrix, value matrix, and the multi-head attention mechanism, a feature fusion operation is performed to obtain a third cable tunnel feature map with enhanced features, and input the above-mentioned third cable tunnel feature map into the above-mentioned neck network;

[0096] Specifically, first input the original cable tunnel image to be detected into a preset road boundary detection model for preprocessing, such as resizing, normalizing, etc., to meet the input requirements of the model. Subsequently, use the backbone network in the model to extract features from the preprocessed image. The backbone network consists of the above-mentioned convolutional layers, C2f network, and multi-head self-attention network. The feature extraction process of the backbone network is as follows: First, through a series of convolutional layers, these convolutional layers use convolutional kernels of different sizes and numbers to extract local features in the image. After the convolutional operation, an activation function (such as ReLU) is used to increase the non-linearity of the model, and batch normalization is used to accelerate the training process and stabilize the model performance. Subsequently, the obtained first cable tunnel feature map is fed into the C2f network. In the C2f network, by parallelizing more gradient flow branches, richer gradient flow information can be obtained while ensuring light weight. The C2f network contains multiple convolutional layers and bottleneck structures. By optimizing the gradient flow, the model performance is enhanced, and the second cable tunnel feature map is obtained. Subsequently, the multi-head self-attention network receives the second cable tunnel feature map as input, calculates the query, key, and value matrices, and uses the multi-head attention mechanism for feature fusion. The features for feature fusion include the road boundary inside the cable tunnel. At the same time, position embedding is introduced to capture the position information in the feature map, thereby enhancing the model's understanding of spatial relationships. Finally, the multi-head self-attention network outputs the third cable tunnel feature map with enhanced features to the neck network.

[0097] The above-mentioned neck network sequentially performs multi-scale feature extraction and feature fusion on the above-mentioned third cable tunnel feature map to obtain the fourth cable tunnel feature map, and inputs the above-mentioned fourth cable tunnel feature map into the above-mentioned head network;

[0098] Specifically, the neck network receives the third cable tunnel feature map output from the backbone network as input. These third cable tunnel feature maps contain information at different levels and scales in the image. The third cable tunnel feature map first undergoes multi-scale feature extraction through the SPP structure. The SPP structure converts the third cable tunnel feature map into a fixed-length representation through pooling operations at different scales, thereby enhancing the model's robustness to targets at different scales. After being processed by the SPP structure, the feature map is fed into the PANet for feature fusion. The PANet structure realizes cross-scale information transmission and fuses feature maps at different scales through bottom-up and top-down paths. After the feature fusion and processing by the neck network, a series of fused fourth cable tunnel feature maps are finally output. These fourth cable tunnel feature maps contain richer and more discriminative information, providing better input for subsequent detection tasks.

[0099] The above-mentioned head network performs object detection on the above-mentioned fourth cable tunnel feature map with the above-mentioned road boundary as the target, and obtains a first cable tunnel image with a number of road bounding boxes; among them, the confidence levels of different road bounding boxes are different; the above-mentioned road bounding boxes include road boundary information in the above-mentioned first cable tunnel image;

[0100] Retain the road bounding box with the highest confidence level as the target road bounding box, and calculate the coordinates of the above-mentioned target road bounding box according to the road boundary information in the above-mentioned first cable tunnel image;

[0101] Obtain the above-mentioned cable tunnel image according to the above-mentioned target road bounding box and the above-mentioned coordinates.

[0102] Specifically, the head network receives the fourth cable tunnel feature map output from the neck network for object detection. These fourth cable tunnel feature maps have been fused and enhanced at different scales, containing rich image information and object features. The object detection process mainly includes four parts: multi-scale detection head, decoupled structure, feature map processing and prediction output, to obtain the above-mentioned first cable tunnel image. Finally, post-processing is performed on the output of the head network to remove redundant detection boxes and determine the final detection result. The post-processing steps usually include operations such as coordinate transformation and NMS. Coordinate transformation is used to convert the detected road bounding box into coordinates relative to the original cable tunnel image. NMS is used to remove road bounding boxes with high overlap degrees, and only retain the road bounding box with the highest confidence level as the final result. Finally, the above-mentioned cable tunnel image with road bounding boxes is obtained.

[0103] Schematically, the structure of the preset road boundary detection model is as Figure 2 shown. Without changing the original network structure of YOLOv8, a multi-head self-attention mechanism is added, that is, Figure 2 "MHSA" in Figure 2 to achieve the modification of the Backbone in the original YOLOv8 network. Subsequently, the neck network receives the third cable tunnel feature map output by the backbone network, processes it in the neck network, and outputs it to the head network (i.e.,

[0104] Add "MHSA" to `_all_` in `conv.py` of the original YOLOv8 network code. Then modify the `init.py` file in the `modules` folder, import the MHSA function in the file, and declare the function in `_all_`. Then create a new file `yolov8_MHSA.yaml` in the `V8` folder and configure the relevant parameter details in the file. Then register it in `task.py` and add MHSA. Finally, create a new `train.py` in the `ultralytics` file and set the parameter path of the model to the path of `yolov8_MHSA.yaml`. Among them, the configured relevant parameters are: embedding dimension, number of heads, and Dropout rate (dropout rate). The embedding dimension is used to define the dimension of the query vector, key vector, and value vector of each attention head in MHSA. The number of heads is used to define the number of attention heads in MHSA. The Dropout rate is applied to the Dropout operation of the attention head output to prevent overfitting.

[0105] Specifically, obtain the second original cable tunnel image at the same time, and use the road boundary detection model constructed based on the original YOLOv8 network to perform road boundary detection on the second original cable tunnel image. Schematically, using the road boundary detection model constructed based on the original YOLOv8 network, the schematic diagram of the road boundary detection result before improvement is as Figure 3 shown. Using the above-mentioned preset road boundary detection model constructed based on the improved YOLOv8 network, the schematic diagram of the road boundary detection result after improvement is as Figure 4 shown.

[0106] Specifically, the comparison results of the performance indicators of the road boundary detection models before and after improvement are shown in the following table:

[0107] Table of comparison results of performance indicators

[0108]

[0109] It can be clearly seen from the above table that the performance indicators of the road boundary detection model based on the improvement are significantly higher than those of the road boundary detection model before improvement.

[0110] Schematically, the iterative graph of the algorithm network performance indicators of the above-mentioned preset road boundary detection model constructed based on the improved YOLOv8 network is as Figure 5 shown. Figure 5The image corresponding to "train / box_loss" represents the bounding box loss on the training set. It can be seen that during the training process, the change of its loss with the number of iterations is as follows: the loss value becomes smaller and smaller, that is, the model's prediction of the bounding box is more and more accurate. The image corresponding to "train / cls_loss" represents the classification loss on the training set. It can be seen that as the iteration progresses, this loss gradually decreases, indicating that the model's classification ability is improving. The image corresponding to "train / dfl_loss" represents the distribution focal loss on the training set. The image corresponding to "metrics / precision(B)" represents the precision index on the training set. The image corresponding to "metrics / recall(B)" represents the recall rate on the training set. The image corresponding to "val / box_loss" represents the bounding box loss on the test set. The image corresponding to "val / cls_loss" represents the classification loss on the test set. The image corresponding to "val / dfl_loss" represents the distribution focal loss on the test set. The image corresponding to "metrics / mAP50(B)" represents the average distribution mean with an IoU (Intersection over Union) threshold of 50% on the test set. The image corresponding to "metrics / mAP50-95(B)" represents the average precision mean when the IoU threshold ranges from 50% to 95% on the test set.

[0111] Preferably, after adding the MHSA multi-head self-attention mechanism, by calculating multiple attention heads in parallel, the representation ability of the model is effectively enhanced, enabling it to better capture various features in the image, enhance the representation ability, and alleviate the instability problem that may occur in single-head attention.

[0112] In this preferred embodiment, after the original cable tunnel image is input into the preset road boundary detection model, through the processing of the convolutional layer, C2f network, multi-head self-attention network, neck network, and head network in the preset road boundary detection model, a cable tunnel image with road bounding boxes is finally obtained.

[0113] In another preferred embodiment, the above-mentioned preprocessing of the original cable tunnel image by inputting it into the preset road boundary detection model to obtain the first preprocessed cable tunnel image includes:

[0114] Adjust the size of the above-mentioned original cable tunnel image to the preset image size to obtain the first original cable tunnel image;

[0115] Normalize the above-mentioned first original cable tunnel image to obtain the above-mentioned first cable tunnel image.

[0116] Specifically, the size of the original cable tunnel image is adjusted to a preset size and normalized to meet the processing requirements of the preset road boundary detection model for the image.

[0117] In this preferred embodiment, a first original cable tunnel image is obtained by preprocessing the original cable tunnel image.

[0118] Step S103: Input the above cable tunnel image into a preset image segmentation model, so that the image segmentation model performs image segmentation on the above cable tunnel image according to the above road boundary box to obtain the road boundary image in the above cable tunnel to be recognized;

[0119] Specifically, the above preset image segmentation model is constructed based on the YOLOv8 network, and its object segmentation code is YOLOv8n-seg.

[0120] Step S104: Process the above road boundary image to obtain the coordinate points of the above road boundary;

[0121] In a preferred embodiment, the processing of the above road boundary image to obtain the coordinate points of the above road boundary includes:

[0122] Perform mask detection on the above road boundary image to obtain the mask array of the above road boundary;

[0123] Identify the above mask array to obtain the mask image of the above road boundary;

[0124] Specifically, use Pillow image to process the mask array to identify the mask image.

[0125] Schematically, based on Figure 3 the road boundary detection result before improvement shown, the road boundary mask image before improvement is as Figure 6 shown, based on Figure 4 the schematic diagram of the road boundary detection result after improvement shown, the road boundary mask image after improvement is as Figure 7 shown.

[0126] Extract the contour point array of the road boundary on the above mask image to obtain the above coordinate points.

[0127] Specifically, use the polygon object, that is, view the contour point array of the mask image, and then extract the contour point array to obtain the above coordinate points.

[0128] In this preferred embodiment, the coordinate points of the road boundary are obtained by processing the road boundary image.

[0129] Step S105: According to the least squares method and the above coordinate points, fit to obtain a road navigation line;

[0130] In a preferred embodiment, the above-mentioned fitting to obtain a road navigation line according to the least squares method and the above coordinate points includes:

[0131] Extract the left coordinate points on the left boundary of the above road boundary and the right coordinate points on the right boundary of the above road boundary from the above coordinate points;

[0132] According to the above left coordinate points and the least squares method, calculate the first fitting parameter when the sum of the squares of the differences between the ordinate of the above left coordinate points and the fitted ordinate of the left coordinate points is minimized; wherein, the fitted ordinate of the above left coordinate points is obtained by fitting the above left coordinate points according to the least squares method;

[0133] Specifically, the principle of the above least squares method is that given a series of coordinate points, assuming that there is a linear relationship between the x-axis and y-axis of the coordinate points, that is, it can be fitted in the form of y = Kx + B. Subsequently, the first fitting parameter is calculated according to the following formula:

[0134]

[0135] In the formula, f1 represents the sum of the squares of the differences between the ordinate of the left coordinate points and the fitted ordinate of the left coordinate points, N1 represents the total number of left coordinate points, y 1i represents the ordinate of the i-th left coordinate point, K1 and B1 represent the first fitting parameter, x 1i represents the abscissa of the i-th left coordinate point, and the value of the formula K1x 1i + B1 represents the fitted ordinate of the above left coordinate points.

[0136] According to the above right coordinate points and the least squares method, calculate the second fitting parameter when the sum of the squares of the differences between the ordinate of the above right coordinate points and the fitted ordinate of the right coordinate points is minimized; wherein, the fitted ordinate of the above right coordinate points is obtained by fitting the above right coordinate points according to the least squares method;

[0137] Specifically, the second fitting parameter is calculated according to the following formula:

[0138]

[0139] In the formula, f2 represents the sum of the squares of the differences between the ordinate of the right coordinate points and the fitted ordinate of the right coordinate points, N2 represents the total number of right coordinate points, y 2i represents the ordinate of the i-th right coordinate point, K2 and B2 represent the second fitting parameter, x 2irepresents the abscissa of the i-th right coordinate point, and the value of the expression K2x 2i + B2 represents the fitted ordinate of the above-mentioned right coordinate point.

[0140] Respectively according to the above first fitting parameter and the above second fitting parameter, calculate the first fitting coordinates of each point on the above left boundary and the second fitting coordinates of each point on the above right boundary;

[0141] Specifically, according to the abscissa of the left coordinate point and the first fitting parameter, calculate the first fitting coordinates, and according to the abscissa of the right coordinate point and the second fitting parameter, calculate the second fitting coordinates.

[0142] According to the abscissa of the above first fitting coordinates and the abscissa of the second fitting coordinates, calculate the average abscissa value, and use the above average abscissa value as the navigation fitting abscissa of the above road navigation line;

[0143] Specifically, calculate the average abscissa value according to the following formula:

[0144]

[0145] In the formula, X p represents the average abscissa value, x1 represents the abscissa of the first fitting coordinates, and x2 represents the abscissa of the second fitting coordinates.

[0146] According to the ordinate of the above first fitting coordinates and the ordinate of the second fitting coordinates, calculate the average ordinate value, and use the above average ordinate value as the navigation fitting ordinate of the above road navigation line;

[0147] Specifically, calculate the average ordinate value according to the following formula:

[0148]

[0149] In the formula, Y p represents the average ordinate value, y1 represents the ordinate of the first fitting coordinates, and y2 represents the ordinate of the second fitting coordinates.

[0150] Generate the above road navigation line according to the above navigation fitting abscissa and the above navigation fitting ordinate.

[0151] In this preferred embodiment, the road navigation line is fitted according to the least squares method and the coordinate points.

[0152] Step S106: Extract the coordinate information of the above road navigation line, and draw the above road navigation line on the above original cable tunnel image according to the above coordinate information to generate the navigation path of the above cable tunnel to be recognized.

[0153] Specifically, based on the coordinates of the generated road navigation line, the road navigation line is drawn on the original cable tunnel image to generate a navigation path.

[0154] Schematically, based on the above Figure 6 The schematic diagram of the navigation path before improvement obtained based on the road boundary mask image before improvement shown above is as Figure 8 shown. Based on the above Figure 7 The schematic diagram of the navigation path after improvement obtained based on the road boundary mask image after improvement shown above is as Figure 9 shown.

[0155] Based on the above method item embodiments, the present invention correspondingly provides device item embodiments.

[0156] As Figure 10 shown, an embodiment of the present invention provides a cable tunnel navigation path recognition device, including:

[0157] An image acquisition module, a road boundary detection module, a mask array acquisition module, a coordinate point calculation module, a road navigation line fitting module, and a navigation path generation module;

[0158] The above image acquisition module is used to acquire the original cable tunnel image of the cable tunnel to be recognized; wherein, the original cable tunnel image carries the road information in the cable tunnel to be recognized, and the original cable tunnel image is acquired through a vision sensor;

[0159] The above road boundary detection module is used to input the original cable tunnel image into a preset road boundary detection model, so that the preset road boundary detection model detects the road boundary in the original cable tunnel image to obtain a cable tunnel image with a road boundary box; wherein, the preset road boundary detection model is constructed based on the improved YOLOv8 network, and the improved YOLOv8 network is a YOLOv8 network with a multi-head self-attention module;

[0160] The above mask array acquisition module is used to input the cable tunnel image into a preset image segmentation model, so that the image segmentation model performs image segmentation on the cable tunnel image according to the road boundary box to obtain the road boundary image in the cable tunnel to be recognized;

[0161] The above coordinate point calculation module is used to process the road boundary image to obtain the coordinate points of the road boundary;

[0162] The above road navigation line fitting module is used to fit the road navigation line according to the least squares method and the above coordinate points;

[0163] The above navigation path generation module is used to extract the coordinate information of the above road navigation line, and draw the above road navigation line on the above original cable tunnel image according to the above coordinate information to generate the navigation path of the above cable tunnel to be recognized.

[0164] In a preferred embodiment, the above road boundary detection module includes:

[0165] A sample image acquisition unit, a sample image road boundary detection unit, a loss function value calculation unit, and a training times judgment unit;

[0166] The above sample image acquisition unit is used to acquire a plurality of cable tunnel sample images with real labels; wherein, the above real label is a real road boundary box used to represent the road boundary information in the above cable tunnel sample image;

[0167] The above sample image road boundary detection unit is used to input the above cable tunnel sample image into the road boundary detection model to be trained, so that the road boundary detection model detects the road boundary in the above cable tunnel sample image to obtain a detected road boundary box;

[0168] The above loss function value calculation unit is used to calculate a loss function value according to the above detected road boundary box and the above real label;

[0169] The above training times judgment unit is used to judge whether the current training times is not less than the preset training times; if so, the road boundary detection model training is completed to obtain the above preset road boundary detection model; otherwise, adjust the parameters in the above road boundary detection model according to the above loss function value, and continue to train the above road boundary detection model.

[0170] It should be noted that the device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement without creative efforts. The above schematic diagram is only an example of a cable tunnel navigation path recognition device, and does not constitute a limitation on a cable tunnel navigation path recognition device, and may include more or fewer components than shown in the figure, or combine some components, or different components.

[0171] Based on the above method embodiment, the present invention correspondingly provides a terminal device embodiment.

[0172] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the above memory and configured to be executed by the above processor. When the above processor executes the above computer program, it implements a cable tunnel navigation path recognition method in any one of the embodiments of the present invention.

[0173] Exemplarily, in this embodiment, the above computer program may be divided into one or more modules. The above one or more modules are stored in the above memory and executed by the above processor to complete the present invention. The above one or more module elements may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the above computer program in the above device;

[0174] The above terminal device may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The above device may include, but is not limited to, a processor and a memory;

[0175] The so-called processor may be a central processing module (Central Processing Unit, CPU), or may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The above processor is the control center of the above device, and connects various parts of the entire device through various interfaces and lines;

[0176] The above-mentioned memory can be used to store the above-mentioned computer programs and / or modules. By running or executing the computer programs and / or modules stored in the above-mentioned memory, and by invoking the data stored in the memory, the above-mentioned processor realizes various functions of the above-mentioned device. The above-mentioned memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, etc. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0177] Based on the above method embodiment, the present invention correspondingly provides a storage medium embodiment.

[0178] Another embodiment of the present invention provides a storage medium. The above-mentioned storage medium includes a stored computer program. When the above-mentioned computer program runs, it controls the device where the above-mentioned storage medium is located to execute the cable tunnel navigation path recognition method according to any one of the above-mentioned embodiments of the present invention.

[0179] In this embodiment, the above-mentioned storage medium is a computer-readable storage medium. The above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The above-mentioned computer-readable medium can include: any entity or device capable of carrying the above-mentioned computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0180] Compared with the prior art, by implementing the above-mentioned various embodiments of the present invention, while meeting the calculation accuracy, the device cost of the entire navigation path recognition process is reduced.

[0181] The above is the preferred implementation manner of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present invention.

Claims

1. A method for identifying a navigation path in a cable tunnel, characterized in that: include: Acquire an original cable tunnel image of a cable tunnel to be identified; wherein the original cable tunnel image carries road information in the cable tunnel to be identified, and the original cable tunnel image is acquired by a visual sensor; Inputting the original cable tunnel image into a preset road boundary detection model, so that the preset road boundary detection model detects the road boundary in the original cable tunnel image, and obtains a cable tunnel image with a road boundary box; wherein the preset road boundary detection model is constructed based on an improved YOLOv8 network, and the improved YOLOv8 network is a YOLOv8 network with a multi-head self-attention module; Inputting the cable tunnel image into a preset image segmentation model so that the image segmentation model performs image segmentation on the cable tunnel image according to the road boundary box to obtain a road boundary image in the cable tunnel to be identified; Processing the road boundary image to obtain coordinate points of the road boundary; According to the least square method and the coordinate points, a road navigation line is obtained by fitting; The coordinate information of the road navigation line is extracted, and the road navigation line is drawn on the original cable tunnel image according to the coordinate information to generate a navigation path of the cable tunnel to be identified.

2. A cable tunnel navigation path identification method according to claim 1, characterized in that: The training of the preset road boundary detection model includes: Acquire a number of cable tunnel sample images with real labels; wherein the real labels are real road boundary boxes used to represent road boundary information in the cable tunnel sample images; Inputting the cable tunnel sample image into a road boundary detection model to be trained, so that the road boundary detection model detects the road boundary in the cable tunnel sample image to obtain a detected road boundary box; Calculating a loss function value according to the detected road boundary box and the true label; Determine whether the current number of training times is not less than the preset number of training times; if so, the road boundary detection model training is completed and the preset road boundary detection model is obtained; otherwise, adjust the parameters in the road boundary detection model according to the loss function value, and continue to train the road boundary detection model.

3. A cable tunnel navigation path identification method according to claim 2, characterized in that: The preset road boundary detection model includes: a convolutional layer, a C2f network, a multi-head self-attention network, a neck network, and a head network; The step of inputting the original cable tunnel image into a preset road boundary detection model so that the preset road boundary detection model detects the road boundary in the original cable tunnel image to obtain the cable tunnel image with a road boundary frame includes: Inputting the original cable tunnel image into a preset road boundary detection model for preprocessing to obtain a preprocessed first cable tunnel image, and inputting the first cable tunnel image into the convolution layer; The convolution layer convolves the first cable tunnel image to extract a first cable tunnel feature map having local features of the first cable tunnel image, and inputs the first cable tunnel feature map into the C2f network; The C2f network performs feature enhancement on the first cable tunnel feature map through gradient flow optimization to obtain a second cable tunnel feature map, and inputs the second cable tunnel feature map into the multi-head self-attention network; After the multi-head self-attention network performs linear transformation and mapping on the second cable tunnel feature map, a query matrix, a key matrix and a value matrix are obtained; Performing a feature fusion operation according to the query matrix, the key matrix, the value matrix, and the multi-head attention mechanism to obtain a feature-enhanced third cable tunnel feature map, and inputting the third cable tunnel feature map into the neck network; The neck network sequentially performs multi-scale feature extraction and feature fusion on the third cable tunnel feature map to obtain a fourth cable tunnel feature map, and inputs the fourth cable tunnel feature map into the head network; The head network performs target detection on the fourth cable tunnel feature image with the road boundary as the target, and obtains a first cable tunnel image with a plurality of road boundary boxes; wherein the confidence levels of different road boundary boxes are different; and the road boundary boxes include road boundary information in the first cable tunnel image; retaining the road boundary box with the highest confidence as the target road boundary box, and calculating the coordinates of the target road boundary box according to the road boundary information in the first cable tunnel image; The cable tunnel image is obtained according to the target road boundary box and the coordinates.

4. A cable tunnel navigation path identification method according to claim 3, characterized in that: The step of inputting the original cable tunnel image into a preset road boundary detection model for preprocessing to obtain a preprocessed first cable tunnel image includes: Adjusting the size of the original cable tunnel image to a preset image size to obtain a first original cable tunnel image; The first original cable tunnel image is normalized to obtain the first cable tunnel image.

5. A cable tunnel navigation path identification method according to claim 3, characterized in that: The processing of the road boundary image to obtain the coordinate points of the road boundary includes: Performing mask detection on the road boundary image to obtain a mask array of the road boundary; Identifying the mask array to obtain a mask image of the road boundary; The contour point array of the road boundary on the mask image is extracted to obtain the coordinate points.

6. A cable tunnel navigation path identification method according to claim 5, characterized in that: The step of fitting a road navigation line according to the least square method and the coordinate points includes: Extracting from the coordinate points a left coordinate point located at a left boundary of the road boundary and a right coordinate point located at a right boundary of the road boundary; According to the left coordinate point and the least square method, a first fitting parameter is calculated when the sum of squares of the difference between the ordinate of the left coordinate point and the fitting ordinate of the left coordinate point is minimized; wherein the fitting ordinate of the left coordinate point is obtained by fitting the left coordinate point according to the least square method; According to the right-side coordinate point and the least square method, a second fitting parameter is calculated when the sum of squares of the difference between the ordinate of the right-side coordinate point and the fitting ordinate of the right-side coordinate point is minimized; wherein the fitting ordinate of the right-side coordinate point is obtained by fitting the right-side coordinate point according to the least square method; Calculating first fitting coordinates of each point on the left boundary and second fitting coordinates of each point on the right boundary according to the first fitting parameters and the second fitting parameters respectively; According to the abscissa of the first fitting coordinate and the abscissa of the second fitting coordinate, an average abscissa value is calculated, and the average abscissa value is used as the navigation fitting abscissa of the road navigation line; According to the ordinate of the first fitting coordinate and the ordinate of the second fitting coordinate, an average ordinate value is calculated, and the average ordinate value is used as the navigation fitting ordinate of the road navigation line; The road navigation line is generated according to the navigation fitting abscissa and the navigation fitting ordinate.

7. A cable tunnel navigation path identification device, characterized in that: include: Image acquisition module, road boundary detection module, mask array acquisition module, coordinate point calculation module, road navigation line fitting module and navigation path generation module; The image acquisition module is used to acquire an original cable tunnel image of the cable tunnel to be identified; wherein the original cable tunnel image carries road information in the cable tunnel to be identified, and the original cable tunnel image is acquired by a visual sensor; The road boundary detection module is used to input the original cable tunnel image into a preset road boundary detection model, so that the preset road boundary detection model detects the road boundary in the original cable tunnel image to obtain a cable tunnel image with a road boundary box; wherein the preset road boundary detection model is constructed based on an improved YOLOv8 network, and the improved YOLOv8 network is a YOLOv8 network with a multi-head self-attention module; The mask array acquisition module is used to input the cable tunnel image into a preset image segmentation model, so that the image segmentation model performs image segmentation on the cable tunnel image according to the road boundary box to obtain the road boundary image in the cable tunnel to be identified; The coordinate point calculation module is used to process the road boundary image to obtain the coordinate points of the road boundary; The road navigation line fitting module is used to fit the road navigation line according to the least square method and the coordinate points; The navigation path generation module is used to extract the coordinate information of the road navigation line, and draw the road navigation line on the original cable tunnel image according to the coordinate information to generate the navigation path of the cable tunnel to be identified.

8. A cable tunnel navigation path identification device according to claim 7, characterized in that: The road boundary detection module comprises: A sample image acquisition unit, a sample image road boundary detection unit, a loss function value calculation unit, and a training times determination unit; The sample image acquisition unit is used to acquire a number of cable tunnel sample images with real labels; wherein the real labels are real road boundary boxes used to represent road boundary information in the cable tunnel sample images; The sample image road boundary detection unit is used to input the cable tunnel sample image into the road boundary detection model to be trained, so that the road boundary detection model detects the road boundary in the cable tunnel sample image to obtain a detected road boundary box; The loss function value calculation unit is used to calculate the loss function value according to the detected road boundary box and the true label; The training times judgment unit is used to judge whether the current training times is not less than the preset training times; if so, the road boundary detection model training is completed and the preset road boundary detection model is obtained; otherwise, the parameters in the road boundary detection model are adjusted according to the loss function value, and the road boundary detection model continues to be trained.

9. A terminal device, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements a cable tunnel navigation path identification method as described in any one of claims 1 to 6.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a cable tunnel navigation path identification method as described in any one of claims 1 to 6.

Images