Cable tunnel disease detection method and device, computer equipment and readable storage medium

Abstract

The invention relates to a cable tunnel disease detection method and device, computer equipment and a readable storage medium. The method comprises the following steps: acquiring a plurality of cable tunnel images of a cable tunnel acquired in a target time period, and acquiring a plurality of sensor data of the cable tunnel in the target time period; performing disease category identification on the plurality of cable tunnel images based on a cable tunnel identification model to obtain a disease category corresponding to each cable tunnel image output by the cable tunnel identification model; converting the disease categories corresponding to the plurality of cable tunnel images into a first matrix; converting the plurality of sensor data into a second matrix; splicing the first matrix and the second matrix to obtain a plurality of data sets; and fitting the plurality of data sets, and obtaining the cable tunnel disease grade according to a fitting result. By adopting the method, the detection efficiency and accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tunnel structure disease detection, in particular to a cable tunnel disease detection method and device, computer equipment and a readable storage medium. BACKGROUND

[0002] With the rapid development of urban infrastructure, the safety and stability of cable tunnels, which are carriers of important urban power supply and communication facilities, are increasingly valued. Cable tunnel structures may develop diseases (such as cracking, water seepage, and lining peeling, etc.) due to environmental changes, material aging, or external factors during long-term use. These diseases affect the reliability of power and communication and pose serious safety hazards. Therefore, it is particularly important to detect and assess the structural health status of cable tunnels in a timely and accurate manner.

[0003] Traditional detection methods rely on manual inspection or simple sensor monitoring, which have low detection efficiency and accuracy. SUMMARY

[0004] Therefore, it is necessary to provide a cable tunnel disease detection method, device, computer equipment and readable storage medium that can improve detection efficiency and accuracy to solve the above technical problems.

[0005] In a first aspect, the present application provides a cable tunnel disease detection method, which comprises:

[0006] obtaining multiple cable tunnel images of the cable tunnel collected in a target time period, and obtaining multiple sensor data of the cable tunnel in the target time period, the multiple cable tunnel images including multiple target visible light images and multiple target thermal imaging images;

[0007] performing disease category identification on the multiple cable tunnel images based on the trained cable tunnel recognition model to obtain the disease categories corresponding to each cable tunnel image output by the cable tunnel recognition model;

[0008] converting the disease categories corresponding to the multiple cable tunnel images into a first matrix according to the time sequence and the instance spatial positions of each cable tunnel image; and converting the multiple sensor data into a second matrix according to the time sequence and the positions of the sensors collecting the sensor data;

[0009] splicing the first matrix and the second matrix to obtain multiple data sets; fitting the multiple data sets, and obtaining the cable tunnel disease grade according to the fitting result.

[0010] In one of the embodiments, the acquiring multiple cable tunnel images of the cable tunnel collected in a target time period includes: acquiring multiple visible light images and multiple thermal imaging images of the cable tunnel collected in the target time period; for each visible light image, calculating the number of pixels of each gray level in the visible light image, and obtaining a cumulative distribution function based on the number of pixels of each gray level; performing gray level mapping on the visible light image based on the cumulative distribution function to obtain an initial visible light image corresponding to the visible light image; inputting the initial visible light images corresponding to the multiple visible light images into the trained panoramic segmentation residual model to obtain multiple target visible light images output by the panoramic segmentation residual model; and inputting the multiple thermal imaging images into the panoramic segmentation residual model to obtain multiple target thermal imaging images output by the panoramic segmentation residual model.

[0011] In one of the embodiments, the training method of the cable tunnel identification model includes: acquiring cable tunnel sample images and disease categories of the cable tunnel sample images; inputting the cable tunnel sample images into an initial cable tunnel identification model to obtain predicted disease categories of the cable tunnel sample images output by the initial cable tunnel identification model; training the initial cable tunnel identification model based on the disease categories of the cable tunnel sample images and the difference between the predicted disease categories of each cable tunnel sample image; determining whether the accuracy of the initial cable tunnel identification model reaches a first preset accuracy, if yes, the training is completed, and the cable tunnel identification model is obtained; and if no, re-executing the step of acquiring the cable tunnel sample images and the disease categories of the cable tunnel sample images.

[0012] In one of the embodiments, the acquiring cable tunnel sample images and disease categories of the cable tunnel sample images includes: acquiring initial cable tunnel sample images; performing panoramic segmentation on the initial cable tunnel sample images based on the trained panoramic segmentation residual model to obtain the cable tunnel sample images, and performing category labeling on the cable tunnel sample images to obtain the disease categories of the cable tunnel sample images.

[0013] In one of the embodiments, the training method of the panoramic segmentation residual model includes: acquiring target sample images and segmentation images of the target sample images, the target sample images being target visible light sample images of a cable tunnel or thermal imaging sample images of the cable tunnel; inputting the target sample images into a pooling layer of an initial panoramic segmentation residual model to obtain sample features; inputting the sample features into a fusion network of the initial panoramic segmentation residual model to obtain sample fusion features; inputting the sample fusion features into a full connection layer of the initial panoramic segmentation residual model to obtain predicted segmentation images; training the initial panoramic segmentation residual model based on the difference between the segmentation images and the predicted segmentation images; determining whether the accuracy of the initial panoramic segmentation residual model reaches a second preset accuracy, if yes, the training is completed, and the panoramic segmentation residual model is obtained; and if no, re-executing the step of acquiring the target sample images and the segmentation images of the target sample images.

[0014] In one of the embodiments, the fusion network comprises N layers of network layers connected in sequence, the i-th layer of network layers comprises an i-th layer of residual blocks and an i-th layer of attention modules; N is a positive integer greater than 3, i is a positive integer greater than or equal to 1 and less than or equal to N;

[0015] The sample feature is input into the fusion network of the initial panoramic segmentation residual model to obtain a sample fusion feature, comprising:

[0016] For the first layer of network layers, the sample feature is input into the first layer of residual blocks, the output of the first layer of residual blocks is input into the first layer of attention modules, and the sample feature and the output of the first layer of attention modules are fused to obtain the output of the first layer of network layers;

[0017] For the second layer of network layers, the output of the first layer of residual blocks is input into the second layer of residual blocks, the output of the second layer of residual blocks and the output of the first layer of attention modules are input into the second layer of attention modules, and the output of the first layer of network layers and the output of the second layer of attention modules are fused to obtain the output of the second layer of network layers;

[0018] For the N-th layer of network layers, the output of the N-1-th layer of residual blocks and the output of the N-2-th layer are input into the N-th layer of residual blocks, the output of the N-th layer of residual blocks and the output of the N-1-th layer of attention modules are input into the N-th layer of attention modules, and the output of the N-1-th layer of network layers, the output of the N-1-th layer of residual blocks and the output of the N-th layer of attention modules are fused to obtain the output of the N-th layer of network layers;

[0019] For any i-th layer of network layers other than the first layer of network layers, the second layer of network layers and the N-th layer of network layers, the output of the i-2-th layer of residual blocks and the output of the i-1-th layer of residual blocks are input into the i-th layer of residual blocks, the output of the i-th layer of residual blocks and the output of the i-1-th layer of attention modules are input into the i-th layer of attention modules, and the output of the i-1-th layer of network layers and the output of the i-th layer of attention modules are fused to obtain the output of the i-th layer of network layers;

[0020] The outputs of the N layers of network layers are fused to obtain the sample fusion feature.

[0021] In one of the embodiments, the i-th layer of attention modules comprises a first attention module and a second attention module; wherein the first attention module comprises a first channel attention processing layer and a first spatial attention processing layer; the second attention module comprises a second spatial attention processing layer; when the target sample image is a target visible light sample image, the feature input into the i-th layer of attention modules is input into the first attention module; when the target sample image is a thermal imaging sample image, the feature input into the i-th layer of attention modules is input into the second attention module.

[0022] In a second aspect, the present application also provides a cable tunnel disease detection device, the device comprising:

[0023] an acquisition module configured to acquire a plurality of cable tunnel images of the cable tunnel collected in a target time period and a plurality of sensor data of the cable tunnel in the target time period, the plurality of cable tunnel images comprising a plurality of target visible light images and a plurality of thermal imaging images;

[0024] a recognition module configured to perform disease category recognition on the plurality of cable tunnel images based on the trained cable tunnel recognition model, to obtain disease categories corresponding to each of the cable tunnel images output by the cable tunnel recognition model;

[0025] a conversion module configured to convert the disease categories corresponding to the plurality of cable tunnel images into a first matrix according to a time sequence and an instance space position corresponding to each of the cable tunnel images, and convert the plurality of sensor data into a second matrix according to the time sequence and a position of a sensor collecting each of the sensor data;

[0026] a determination module configured to splice the first matrix and the second matrix to obtain a plurality of data sets, and fit the plurality of data sets to obtain a cable tunnel disease grade according to a fitting result.

[0027] In a third aspect, the present application also provides a computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing steps of the method of any one of the first aspect when executing the computer program.

[0028] In a fourth aspect, the present application also provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement steps of the method of any one of the first aspect.

[0029] In a fifth aspect, the present application also provides a computer program product comprising a computer program, the computer program being executed by a processor to implement steps of the method of any one of the first aspect.

[0030] The cable tunnel disease detection method, device, computer equipment and readable storage medium, by acquiring a plurality of cable tunnel images of the cable tunnel collected in a target time period and acquiring a plurality of sensor data of the cable tunnel in the target time period, the plurality of cable tunnel images include a plurality of target visible light images and a plurality of thermal imaging images; then, based on the trained cable tunnel recognition model, the disease category of each cable tunnel image is identified, and the disease category corresponding to each cable tunnel image output by the cable tunnel recognition model is obtained. Then, according to the time sequence and the instance space position corresponding to each cable tunnel image, the disease categories corresponding to the plurality of cable tunnel images are converted into a first matrix; according to the time sequence and the position of the sensor collecting each sensor data, the plurality of sensor data are converted into a second matrix; the first matrix and the second matrix are spliced to obtain a plurality of data sets; the plurality of data sets are fitted, and the cable tunnel disease grade is obtained according to the fitting result. In one aspect, the disease category of the cable tunnel image is identified by the trained cable tunnel recognition model, which greatly improves the processing efficiency and further improves the detection efficiency. On the other hand, the disease category and the sensor data in the target time period are fitted, and the cable tunnel disease grade is obtained based on the fitting result, which can avoid accidental misjudgment of single-time detection, thereby improving the detection accuracy. In addition, the target visible light image, the thermal imaging image and the sensor data are used to obtain the cable tunnel disease grade, wherein the visible light image can provide appearance texture and structural damage information, the thermal imaging image can capture temperature anomalies, and the sensor data can provide information that cannot be captured by the image in the environment, so that the cable tunnel disease grade obtained based on the complementary of multiple data sources is further improved. The detection accuracy is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other related drawings without creative labor on the basis of these drawings.

[0032] Figure 1 An application environment diagram of the cable tunnel disease detection method in an embodiment;

[0033] Figure 2 A flowchart of the cable tunnel disease detection method in an embodiment;

[0034] Figure 3 A structure diagram of a time sequence prediction model in an embodiment;

[0035] Figure 4A structural schematic diagram of a cable tunnel identification model in one embodiment;

[0036] Figure 5 A structural schematic diagram of a panoramic segmentation residual model in one embodiment;

[0037] Figure 6 A structural schematic diagram of a first attention module and a second attention module in one embodiment;

[0038] Figure 7 A structural schematic diagram of an attention processing layer in one embodiment;

[0039] Figure 8 A structural block diagram of a cable tunnel disease detection device in one embodiment;

[0040] Figure 9 An internal structural diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0042] It should be noted that the terms "first", "second", etc. used in the present application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "a plurality of" used in the present application refers to two or more. The term "and / or" used in the present application refers to one of the options or any combination of multiple options.

[0043] The cable tunnel disease detection method provided by the embodiments of the present application can be applied to, for example, Figure 1The application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data required by the server 104 to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, unmanned aerial vehicles, low-altitude aerial vehicles, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart TV, a smart air conditioner, a smart vehicle device, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0044] In an exemplary embodiment, as shown in Figure 2 A cable tunnel disease detection method is provided, which is applied to the terminal in Figure 1 for example, including the following steps 201 to 204. Among them:

[0045] Step 201, acquiring a plurality of cable tunnel images of the cable tunnel collected in a target time period, and acquiring a plurality of sensor data of the cable tunnel in the target time period; the plurality of cable tunnel images include a plurality of target visible light images and a plurality of target thermal imaging images.

[0046] Among them, the target time period can refer to one year, one month or one day. Taking the target time period as one year as an example, acquiring a plurality of cable tunnel images of the cable tunnel collected in the target time period can be acquiring cable tunnel images of different positions of the cable tunnel collected every day in one year.

[0047] The cable tunnel image can be an image segmented by a trained panoramic segmentation residual model. The purpose of segmentation is to remove the background in the image and only keep the key part, so that the cable tunnel recognition model can more accurately identify the disease category.

[0048] The sensor data can be temperature data, humidity data, and strain force data, etc. Specifically, temperature and humidity sensors and strain force sensors can be installed at multiple different positions in the cable tunnel. The installation positions of the sensors are the same as the positions where the cable tunnel images are collected. The collection frequency and time of the temperature and humidity sensors and the strain force sensors distributed in the cable tunnel space are set to be consistent.

[0049] Multiple sensor data points for the cable tunnel within the target time period can be obtained using a trained time series prediction model. This time series prediction model can be an LSTM (Long Short-Term Memory) model, the structure of which is as follows: Figure 3 As shown.

[0050] Figure 3 middle This represents the input at the current moment; This represents the hidden state of the previous moment, which is short-term memory; It represents the state of the unit at the previous moment, which is long-term memory; Represents the forget gate, receiving... and As input, the signal passes through a sigmoid function, and the output is a value between 0 and 1, which determines the... Which information should be retained (close to 1) or forgotten (close to 0)? and The two parts constitute the input gate. It's a sigmoid function that determines which new information needs to be updated. It is a tanh function that processes the input and generates new candidate values ​​to be added to the cell state. Indicates the output gate, receiving gate. and As input, the signal passes through a sigmoid function, and the output is a value between 0 and 1. This value determines the cell state. What information is output to the hidden state? Cell state The update is based on the Forgotten Gate Input gate ( and ) and the cell state at the previous time step Update in progress. (Hidden status) The update is based on the output gate. and unit state Update.

[0051] In one example, multiple sensor data points of a cable tunnel within a historical time period are input into a trained time series prediction model to obtain multiple sensor data points of the cable tunnel within a target time period, as output by the time series prediction model. Since directly collected sensor data may contain anomalies or missing values, using a time series prediction model to predict sensor data within the target time period can remove outliers and fill in missing values.

[0052] Step 202: Based on the trained cable tunnel recognition model, identify the disease category of each cable tunnel image and obtain the disease category corresponding to each cable tunnel image output by the cable tunnel recognition model.

[0053] Among them, the disease category refers to the type of abnormality in the cable tunnel structure, such as cracking, water seepage, and lining spalling.

[0054] The cable tunnel identification model can use VIT-B as the backbone network. The structure of the cable tunnel identification model is as follows: Figure 4 As shown, the cable tunnel identification model processes multiple cable tunnel images as follows: For each cable tunnel image, mapping, pooling, and location embedding operations are performed to obtain the mapped first feature, the pooled feature, and the location-embedded feature; the mapped first feature, the pooled feature, and the location-embedded feature are then fused, and the fused feature is regularized. The regularized feature is then input into an encoder (Transformer encoder), and the encoder output is subsequently regularized and mapped to obtain the mapped second feature; the mapped second feature is then input into a multi-head attention mechanism to obtain the defect category corresponding to the cable tunnel image.

[0055] The multi-head attention mechanism includes the following formula:

[0056] , , ,

[0057] ,

[0058]

[0059] in, These three matrices are the inputs to the multi-head attention mechanism; These three are weight matrices; Indicates the first One point of attention; It consists of the query, index, and value after linear transformation. This describes the computational process of a single-head attention mechanism, for example, calculating... and The similarity between the two is scaled and then normalized using Softmax to obtain the attention weights. These attention weights are then compared with... Multiplying them gives the weighted value, which is also the result of multiplying the first product. The output of each attention head; , indicating the first the output of the attention head; denotes concatenating the outputs of all attention heads together; denotes an output weight matrix; denotes the output of the multi-head attention mechanism.

[0060] In step 203, the disease categories corresponding to the plurality of cable tunnel images are converted into a first matrix according to the time sequence and the instance space positions corresponding to the cable tunnel images, and the plurality of sensor data are converted into a second matrix according to the time sequence and the positions of the sensors collecting the sensor data.

[0061] The instance space positions corresponding to the cable tunnel images refer to the spatial coordinates of the objects in the cable tunnel images. The first matrix is a disease category matrix, that is, an image recognition result matrix, and the second matrix is a sensor data matrix.

[0062] The first matrix has the following expression form:

[0063]

[0064] wherein, denotes the instance space positions corresponding to the cable tunnel images; denotes a timestamp; denotes a disease category.

[0065] The second matrix has the following expression form:

[0066]

[0067] wherein, denotes the positions of the sensors; denotes a timestamp; denotes sensor data.

[0068] In step 204, the first matrix and the second matrix are spliced to obtain a plurality of data sets, and the plurality of data sets are fitted to obtain a cable tunnel disease level according to a fitting result.

[0069] The data sets each include disease categories and sensor data corresponding to different positions at the same time. The cable tunnel disease level includes normal, attention, abnormal, and serious, etc. The fitting result includes probabilities corresponding to the disease categories, so that the cable tunnel disease level can be determined by the probabilities corresponding to the disease categories.

[0070] In an optional implementation, the first matrix and the second matrix are spliced to obtain a plurality of data sets, including: merging the positions and time stamps in the first matrix and the second matrix, and then constructing a data set from the disease categories and the sensor data corresponding to different positions at the same time. The plurality of data sets are fitted, and a cable tunnel disease grade is obtained according to a fitting result, including: for each data set, a label annotation is performed on the data set according to the disease categories and the sensor data corresponding to different positions in the data set, to obtain a disease category corresponding to the data set; and the disease categories corresponding to the plurality of data sets are fitted based on a Bayesian classifier, and a cable tunnel disease grade is obtained according to a fitting result.

[0071] wherein a core formula of the Bayesian classifier is as follows:

[0072]

[0073] wherein, is a prior probability of the disease category , indicating a total proportion of the disease category ; is a conditional probability of the feature under the disease category ; is a posterior probability, indicating a probability of belonging to the disease category under the known observation data .

[0074] In the above cable tunnel disease detection method, a plurality of cable tunnel images of the cable tunnel collected in a target time period are obtained, and a plurality of sensor data of the cable tunnel in the target time period are obtained, the plurality of cable tunnel images including a plurality of target visible light images and a plurality of thermal imaging images; then, based on the trained cable tunnel recognition model, disease category recognition is performed on the plurality of cable tunnel images respectively, to obtain disease categories corresponding to each cable tunnel image output by the cable tunnel recognition model. Then, the disease categories corresponding to the plurality of cable tunnel images are converted into a first matrix according to a time sequence and an instance space position corresponding to each cable tunnel image; the plurality of sensor data are converted into a second matrix according to a time sequence and a position of a sensor collecting each sensor data; the first matrix and the second matrix are spliced to obtain a plurality of data sets, each data set including disease categories and sensor data; fitting is performed on the plurality of data sets, and a cable tunnel disease grade is obtained according to a fitting result. On the one hand, the disease category recognition is performed on the cable tunnel image by the trained cable tunnel recognition model, which greatly improves the processing efficiency and further improves the detection efficiency; on the other hand, the disease categories and the sensor data in the target time period are fitted, and the cable tunnel disease grade is obtained based on the fitting result, which can avoid accidental misjudgment of single-time detection, thereby improving the detection accuracy. In addition, the target visible light image, the thermal imaging image and the sensor data are used to obtain the cable tunnel disease grade, wherein the visible light image can provide appearance texture and structural damage information, the thermal imaging image can capture temperature abnormalities, and the sensor data can provide information that cannot be captured by the image in the environment, so that the cable tunnel disease grade obtained is based on the complement of multiple data sources, further improving the detection accuracy.

[0075] In one embodiment, the plurality of cable tunnel images of the cable tunnel collected in the target time period are obtained, including the following steps 2011 to 2014.

[0076] Step 2011, a plurality of visible light images and a plurality of thermal imaging images of the cable tunnel collected in the target time period are obtained.

[0077] The visible light image can be collected by a high-definition camera; the thermal imaging image can be collected by an infrared thermal imaging device.

[0078] Step 2012, for each visible light image, the number of pixels of each gray level in the visible light image is calculated, and a cumulative distribution function is obtained based on the number of pixels of each gray level; the visible light image is mapped based on the cumulative distribution function to obtain an initial visible light image corresponding to the visible light image.

[0079] In other words, for each gray level in the visible light image , the number of pixels of each gray level is counted The gray scale is calculated based on the following formula The corresponding probability is:

[0080]

[0081] wherein, represents the gray scale The corresponding probability is; represents the total number of pixels in the visible light image. That is, the more times each gray scale appears, the higher the probability value.

[0082] For each gray scale The gray scale The corresponding cumulative distribution function is:

[0083]

[0084] wherein, represents the corresponding cumulative distribution function. That is, the value of the cumulative distribution function is equal to the sum of the probabilities of all gray scales less than or equal to In simple terms, it is to add the probabilities from the darkest to the current gray scale.

[0085] Then, a gray scale mapping function is used to adjust the gray scale of the visible light image according to the calculated cumulative distribution function, so as to perform contrast stretching (image equalization), that is, equalization processing, and the gray scale mapping function is as follows:

[0086]

[0087] wherein, represents the gray scale after contrast stretching (after equalization processing); is the maximum gray scale number of the visible light image (for example, for an 8-bit image, ); The function represents a rounding operation. After the operation, the gray scale distribution of the visible light image is more uniform, thereby improving the contrast of the visible light image and making the visible light image clearer.

[0088] Step 2013, input the initial visible light images corresponding to the plurality of visible light images into the trained panoramic segmentation residual model to obtain a plurality of target visible light images output by the panoramic segmentation residual model.

[0089] Step 2014, input the plurality of thermal imaging images into the panoramic segmentation residual model to obtain a plurality of target thermal imaging images output by the panoramic segmentation residual model.

[0090] The panoramic segmentation residual model can be a Resnet-18 network as a backbone network, and the residual network is mixed with an attention module to enhance the panoramic segmentation performance. Based on this, the structure of the panoramic segmentation residual model can be as shown in Figure 5 The fusion network includes N network layers connected in sequence, the i-th network layer includes an i-th residual block and an i-th attention module; N is a positive integer greater than 3, and i is a positive integer greater than or equal to 1 and less than or equal to N. Figure 5 In an example, the fusion network includes 5 network layers, and the FAM is an attention module.

[0091] In one example, the initial visible light images corresponding to multiple visible light images are input into the trained panoramic segmentation residual model to obtain multiple target visible light images output by the panoramic segmentation residual model, including: for each initial visible light image, inputting the initial visible light image into the pooling layer to obtain the output of the pooling layer, inputting the output of the pooling layer into the fusion network to obtain the output of the fusion network, and then inputting the output of the fusion network into the full connection layer to obtain the target visible light image.

[0092] Inputting the output of the pooling layer into the fusion network to obtain the output of the fusion network includes:

[0093] For the first network layer, the output of the pooling layer is input into the first residual block, then the output of the first residual block is input into the first attention module, and then the output of the first attention module is fused with the output of the pooling layer to obtain the output of the first network layer.

[0094] For the second network layer, the output of the first residual block is input into the second residual block, then the output of the second residual block and the output of the first attention module are input into the second attention module, and then the output of the second attention module and the output of the first network layer are fused to obtain the output of the second network layer.

[0095] For the N-th network layer, the output of the N-1-th residual block and the output of the N-2-th layer are input into the N-th residual block, the output of the N-th residual block and the output of the N-1-th attention module are input into the N-th attention module, and the output of the N-1-th network layer, the output of the N-1-th residual block and the output of the N-th attention module are fused to obtain the output of the N-th network layer.

[0096] For any i-th network layer other than the first, second, and Nth network layers, the outputs of the (i-2)-th and (i-1)-th residual blocks are input into the i-th residual block. The outputs of the i-th residual block and (i-1)-th attention modules are input into the i-th attention module. The outputs of the (i-1)-th and i-th network layers are then fused to obtain the output of the i-th network layer. For example, when the fusion network includes 5 network layers, for the third network layer, the outputs of the second and first layer residual blocks are input into the third layer residual block. Then, the output of the third layer residual block and the output of the second layer attention module are input into the third layer attention module. Finally, the output of the third layer attention module and the output of the second layer network layer are fused to obtain the output of the third layer network layer. For the fourth network layer, the outputs of the third and second layer residual blocks are input into the fourth layer residual block. Then, the output of the fourth layer residual block and the output of the third layer attention module are input into the fourth layer attention module. Finally, the output of the fourth layer attention module and the output of the third layer network layer are fused to obtain the output of the fourth layer network layer.

[0097] The outputs of the N-layer network are fused to obtain the output of the fused network.

[0098] In another example, multiple thermal imaging images are input into a panoramic segmentation residual model to obtain multiple target thermal imaging images output by the panoramic segmentation residual model. This includes: for each thermal imaging image, inputting the thermal imaging image into a pooling layer to obtain the output of the pooling layer, inputting the output of the pooling layer into a fusion network to obtain the output of the fusion network, and then inputting the output of the fusion network into a fully connected layer to obtain the target thermal imaging image.

[0099] The specific processing procedure of the fusion network is similar to that described above and will not be repeated here. However, it should be noted that, in order to reduce computational complexity and improve segmentation accuracy, and to make reasonable use of the differences between human imaging images and visible light images, each attention module layer is designed to include a first attention module and a second attention module. The first attention module includes a channel attention processing layer and a first spatial attention processing layer; the second attention module includes a second spatial attention processing layer.

[0100] For the i-th layer attention module, when the input to the panoramic segmentation residual model is an initial visible light image, the features input to the i-th layer attention module are input to the first attention module; when the input to the panoramic segmentation residual model is a thermal imaging image, the features input to the i-th layer attention module are input to the second attention module.

[0101] In one example, such as Figure 6 As shown, Figure 6The (a) diagram in FIG. 1 provides a structural diagram of a first attention module. For the i-th layer attention module, the feature input channel attention processing layer of the i-th layer attention module is input, and then the output of the channel attention processing layer and the feature input into the i-th layer attention module are fused, and then the fused feature is input into the first spatial attention processing layer, and then the output of the first spatial attention processing layer and the fused feature are fused again to obtain the output of the first attention module in the i-th layer attention module.

[0102] Figure 6 The (b) diagram in FIG. 1 provides a structural diagram of a second attention module. For the i-th layer attention module, the feature input into the i-th layer attention module is input into the second spatial attention processing layer, and then the output of the second spatial attention processing layer and the feature input into the i-th layer attention module are fused, and then the fused feature is subjected to a tensor adjustment operation (reshape) to obtain the output of the second attention module in the i-th layer attention module.

[0103] It can be understood that the spatial attention processing layer and the channel attention processing layer differ in the dimension of the tensor adjustment operation link feature matrix.

[0104] It should be noted that the channel attention processing layer, the first spatial attention processing layer and the second spatial attention processing layer all include an attention processing layer, and the structure of the attention processing layer is as shown in FIG. 2. Figure 7 The attention processing layer includes a first convolutional layer (conv1), a second convolutional layer (conv2) and a third convolutional layer (conv3); the feature input into the attention processing layer is input into the first convolutional layer, the second convolutional layer and the third convolutional layer respectively, and the output of the first convolutional layer, the output of the second convolutional layer and the output of the third convolutional layer are all subjected to a tensor adjustment operation (reshape); the output of the first convolutional layer after the tensor adjustment operation and the output of the second convolutional layer after the tensor adjustment operation are fused to obtain first fused features, and the first fused features are subjected to a sigmoid function operation to obtain processed first fused features; the output of the third convolutional layer after the tensor adjustment operation and the processed first fused features are fused to obtain second fused features; the feature input into the attention processing layer and the second fused features are fused to obtain the output of the attention processing layer.

[0105] In the embodiment of the present application, the visible light image and the thermal imaging image are processed by the panoramic segmentation residual model, so that the segmented target visible light image and target thermal imaging image can be quickly obtained, and the detection efficiency can be improved.

[0106] In one of the embodiments, the training method of the cable tunnel recognition model includes the following steps 2021 to 2024.

[0107] At step 2021, the cable tunnel sample image and the disease category of the cable tunnel sample image are obtained.

[0108] The cable tunnel sample image can be a visible light image or a thermal imaging image. The cable tunnel sample image is obtained by panoramic segmentation of an initial cable tunnel sample image based on the trained panoramic segmentation residual model. The disease category of the cable tunnel sample image is the actual disease category of the cable tunnel sample image.

[0109] In an optional implementation, an initial cable tunnel sample image is obtained. The initial cable tunnel sample image is subjected to panoramic segmentation based on the trained panoramic segmentation residual model to obtain a cable tunnel sample image. The cable tunnel sample image is subjected to category labeling to obtain the disease category of the cable tunnel sample image.

[0110] The initial cable tunnel sample image is input into the trained panoramic segmentation residual model to obtain the cable tunnel sample image. For details, refer to the implementation of steps 2013 and 2014 described above, which will not be repeated here.

[0111] At step 2022, the cable tunnel sample image is input into the initial cable tunnel recognition model to obtain the predicted disease category of the cable tunnel sample image output by the initial cable tunnel recognition model.

[0112] It can be understood that the structure of the initial cable tunnel recognition model is the same as that of the cable tunnel recognition model. For details, refer to the structure of the cable tunnel recognition model described above. Figure 4 Therefore, the processing manner of the initial cable tunnel recognition model on the cable tunnel sample image can refer to the processing process of the cable tunnel recognition model on multiple cable tunnel images described above, which will not be repeated here.

[0113] At step 2023, the initial cable tunnel recognition model is trained based on the difference between the disease category of the cable tunnel sample image and the predicted disease category of each cable tunnel sample image. Gradient descent method can be used for iterative training.

[0114] In a possible implementation, the model parameters of the initial cable tunnel recognition model are updated based on the cross-entropy loss between the disease category of the cable tunnel sample image and the predicted disease category of each cable tunnel sample image.

[0115] At step 2024, it is determined whether the accuracy of the initial cable tunnel recognition model reaches a first preset accuracy. If yes, the training is completed, and the cable tunnel recognition model is obtained. If no, the steps of obtaining the cable tunnel sample image and the disease category of the cable tunnel sample image are re-executed.

[0116] The calculation method of the accuracy rate of the initial cable tunnel identification model is as follows:

[0117]

[0118] wherein, is the accuracy rate of the initial cable tunnel identification model; is a true positive example; is a true negative example; is a false positive example; is a false negative example.

[0119] When the accuracy rate of the initial cable tunnel identification model reaches the first preset accuracy rate, it is indicated that the model meets the requirement, and the current initial cable tunnel identification model is taken as the cable tunnel identification model; when the accuracy rate of the initial cable tunnel identification model has not reached the first preset accuracy rate, it is indicated that the model does not meet the requirement, and it is needed to return to execute the step 2021 again until the accuracy rate of the initial cable tunnel identification model reaches the first preset accuracy rate.

[0120] In one of the embodiments, the training method of the panoramic segmentation residual model includes the following steps 2031 to 2036, wherein:

[0121] In step 2031, a target sample image and a segmentation image of the target sample image are obtained, the target sample image being a target visible light sample image of a cable tunnel or a thermal imaging sample image of the cable tunnel.

[0122] The segmentation image of the target sample image is an image obtained by removing the background from the initial target sample image.

[0123] In step 2032, the target sample image is input into a pooling layer of an initial panoramic segmentation residual model to obtain a sample feature.

[0124] It can be understood that the structure of the initial panoramic segmentation residual model is the same as that of the panoramic segmentation residual model, and reference is made to FIGS. 1 to 3. Figure 5 Figure 6 and Figure 7 The initial panoramic segmentation residual model includes a pooling layer, a fusion network and a full connection layer.

[0125] In step 2033, the sample feature is input into the fusion network of the initial panoramic segmentation residual model to obtain a sample fusion feature.

[0126] In a possible implementation manner, the fusion network includes N network layers connected in sequence, the i-th network layer includes an i-th residual block and an i-th attention module; N is a positive integer greater than 3, and i is a positive integer greater than or equal to 1 and less than or equal to N.

[0127] ​For the first layer network layer, the sample feature is input into the first layer residual block, the output of the first layer residual block is input into the first layer attention module, and the sample feature and the output of the first layer attention module are fused to obtain the output of the first layer network layer.

[0128] For the second layer network layer, the output of the first layer residual block is input into the second layer residual block, the output of the second layer residual block and the output of the first layer attention module are input into the second layer attention module, and the output of the first layer network layer and the output of the second layer attention module are fused to obtain the output of the second layer network layer.

[0129] For the Nth layer network layer, the output of the N-1th layer residual block and the output of the N-2th layer are input into the Nth layer residual block, the output of the Nth layer residual block and the output of the N-1th layer attention module are input into the Nth layer attention module, and the output of the N-1th layer network layer, the output of the N-1th layer residual block and the output of the Nth layer attention module are fused to obtain the output of the Nth layer network layer.

[0130] For any ith layer network layer other than the first layer network layer, the second layer network layer and the Nth layer network layer, the output of the i-2th layer residual block and the output of the i-1th layer residual block are input into the ith layer residual block, the output of the ith layer residual block and the output of the i-1th layer attention module are input into the ith layer attention module, and the output of the i-1th layer network layer and the output of the ith layer attention module are fused to obtain the output of the ith layer network layer.

[0131] The outputs of the N layer network layers are fused to obtain the sample fusion feature.

[0132] From the processing process of the initial visible light image and the thermal imaging image by the panoramic segmentation residual model, it can be seen that the attention modules used in the panoramic segmentation residual model are different for different images, that is, the ith layer attention module includes a first attention module and a second attention module; wherein the first attention module includes a channel attention processing layer and a first spatial attention processing layer; the second attention module includes a second spatial attention processing layer; when the target sample image is a target visible light sample image, the feature input into the ith layer attention module is input into the first attention module; when the target sample image is a thermal imaging sample image, the feature input into the ith layer attention module is input into the second attention module.

[0133] Step 2034, input the sample fusion feature into the full connection layer of the initial panoramic segmentation residual model to obtain a predicted segmentation image.

[0134] Step 2035, training the initial panoramic segmentation residual model based on the difference between the segmentation image and the predicted segmentation image. Specifically, gradient descent method can be used for iterative training.

[0135] In a possible implementation, the model parameters of the initial panoramic segmentation residual model are updated based on a cross-entropy loss of the segmented image and the predicted segmented image.

[0136] In step 2036, it is determined whether the accuracy of the initial panoramic segmentation residual model reaches a second preset accuracy. If yes, the training is completed, and the panoramic segmentation residual model is obtained. If no, the steps of obtaining the target sample image and the segmented image of the target sample image are re-executed.

[0137] The calculation method of the accuracy of the initial panoramic segmentation residual model is as follows:

[0138]

[0139] Wherein, is the accuracy of the initial panoramic segmentation residual model; is a true positive; is a false positive; is a false negative; represents the predicted segmented image; represents the segmented image, that is, the real segmented image.

[0140] When the accuracy of the initial panoramic segmentation residual model reaches the second preset accuracy, it is indicated that the model meets the requirements, and the current initial panoramic segmentation residual model is taken as the panoramic segmentation residual model. When the accuracy of the initial panoramic segmentation residual model has not reached the second preset accuracy, it is indicated that the model does not meet the requirements, and the step 2031 needs to be re-executed until the accuracy of the initial panoramic segmentation residual model reaches the second preset accuracy.

[0141] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps has no strict sequence limitation, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps. It can be understood that each step in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination all belong to the scope protected by the present application.

[0142] Based on the same inventive concept, the embodiments of the present application also provide a cable tunnel disease detection device for implementing the above-mentioned cable tunnel disease detection method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more cable tunnel disease detection device embodiments provided below can refer to the limitations of the cable tunnel disease detection method in the foregoing, which will not be repeated here.

[0143] In one exemplary embodiment, as shown in Figure 8 A cable tunnel disease detection device is provided, and the cable tunnel disease detection device 800 includes an acquisition module 801, an identification module 802, a conversion module 803, and a determination module 804, wherein:

[0144] The acquisition module 801 is configured to acquire a plurality of cable tunnel images of a cable tunnel collected in a target time period and acquire a plurality of sensor data of the cable tunnel in the target time period, and the plurality of cable tunnel images include a plurality of target visible light images and a plurality of thermal imaging images.

[0145] The identification module 802 is configured to perform disease category identification on the plurality of cable tunnel images based on a trained cable tunnel identification model, to obtain disease categories corresponding to each cable tunnel image output by the cable tunnel identification model.

[0146] The conversion module 803 is configured to convert the disease categories corresponding to the plurality of cable tunnel images into a first matrix according to a time sequence and an instance space position corresponding to each cable tunnel image, and convert the plurality of sensor data into a second matrix according to a time sequence and a position of a sensor collecting each sensor data.

[0147] The determination module 804 is configured to splice the first matrix and the second matrix to obtain a plurality of data sets, fit the plurality of data sets, and obtain a cable tunnel disease grade according to a fitting result.

[0148] In one embodiment, the acquisition module 801 is specifically configured to acquire a plurality of visible light images and a plurality of thermal imaging images of a cable tunnel collected in a target time period, calculate the number of pixels of each gray level in each visible light image, obtain a cumulative distribution function based on the number of pixels of each gray level, perform gray level mapping on the visible light image based on the cumulative distribution function to obtain an initial visible light image corresponding to the visible light image, input the initial visible light images corresponding to the plurality of visible light images into a trained panoramic segmentation residual model to obtain a plurality of target visible light images output by the panoramic segmentation residual model, and input the plurality of thermal imaging images into the panoramic segmentation residual model to obtain a plurality of target thermal imaging images output by the panoramic segmentation residual model.

[0149] In one of the embodiments, the device further comprises a training module configured to: acquire a cable tunnel sample image and a disease category of the cable tunnel sample image; input the cable tunnel sample image into an initial cable tunnel recognition model to obtain a predicted disease category of the cable tunnel sample image output by the initial cable tunnel recognition model; train the initial cable tunnel recognition model based on a difference between the disease category of the cable tunnel sample image and the predicted disease category of each cable tunnel sample image; determine whether an accuracy of the initial cable tunnel recognition model reaches a first preset accuracy; if yes, the training is completed to obtain the cable tunnel recognition model; and if no, re-perform the steps of acquiring the cable tunnel sample image and the disease category of the cable tunnel sample image.

[0150] In one of the embodiments, the training module is specifically configured to: acquire an initial cable tunnel sample image; perform panoramic segmentation on the initial cable tunnel sample image based on the trained panoramic segmentation residual model to obtain a cable tunnel sample image; and perform category labeling on the cable tunnel sample image to obtain a disease category of the cable tunnel sample image.

[0151] In one of the embodiments, the training module is further configured to: acquire a target sample image and a segmentation image of the target sample image, the target sample image being a target visible light sample image of a cable tunnel or a thermal imaging sample image of the cable tunnel; input the target sample image into a pooling layer of the initial panoramic segmentation residual model to obtain a sample feature; input the sample feature into a fusion network of the initial panoramic segmentation residual model to obtain a sample fusion feature; input the sample fusion feature into a full connection layer of the initial panoramic segmentation residual model to obtain a predicted segmentation image; train the initial panoramic segmentation residual model based on a difference between the segmentation image and the predicted segmentation image; determine whether an accuracy of the initial panoramic segmentation residual model reaches a second preset accuracy; if yes, the training is completed to obtain the panoramic segmentation residual model; and if no, re-perform the steps of acquiring the target sample image and the segmentation image of the target sample image.

[0152] In one of the embodiments, the fusion network comprises N network layers connected in sequence, the i-th network layer comprises an i-th residual block and an i-th attention module; N is a positive integer greater than 3, i is a positive integer greater than or equal to 1 and less than or equal to N; the training module is specifically configured to: for the first network layer, input the sample feature into the first residual block, input an output of the first residual block into the first attention module, and fuse the sample feature and an output of the first attention module to obtain an output of the first network layer.

[0153] For the second layer network layer, input the output of the first layer residual block into the second layer residual block, input the output of the second layer residual block and the output of the first layer attention module into the second layer attention module, and fuse the output of the first layer network layer and the output of the second layer attention module to obtain the output of the second layer network layer;

[0154] For the Nth layer network layer, input the output of the (N-1)th layer residual block and the output of the (N-2)th layer into the Nth layer residual block, input the output of the Nth layer residual block and the output of the (N-1)th layer attention module into the Nth layer attention module, and fuse the output of the (N-1)th layer network layer, the output of the (N-1)th layer residual block and the output of the Nth layer attention module to obtain the output of the Nth layer network layer.

[0155] For any ith layer network layer other than the first layer network layer, the second layer network layer and the Nth layer network layer, input the output of the (i-2)th layer residual block and the output of the (i-1)th layer residual block into the ith layer residual block, input the output of the ith layer residual block and the output of the (i-1)th layer attention module into the ith layer attention module, and fuse the output of the (i-1)th layer network layer and the output of the ith layer attention module to obtain the output of the ith layer network layer.

[0156] Fuse the outputs of the N layer network layers to obtain the sample fusion features.

[0157] In one of the embodiments, the ith layer attention module includes a first attention module and a second attention module; wherein the first attention module includes a first channel attention processing layer and a first spatial attention processing layer; the second attention module includes a second spatial attention processing layer; when the target sample image is a target visible light sample image, the features input into the ith layer attention module are input into the first attention module; when the target sample image is a thermal imaging sample image, the features input into the ith layer attention module are input into the second attention module.

[0158] The above-mentioned various modules in the cable tunnel disease detection device can be realized by software, hardware and combinations thereof in whole or in part. The above-mentioned various modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above-mentioned various modules.

[0159] In an exemplary embodiment, a computer device, which can be a terminal, is provided, and an internal structure diagram thereof can be as shown in Figure 9As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with the external terminal in a wired or wireless manner. The wireless manner can be realized through WIFI, mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. The computer program is executed by the processor to realize a cable tunnel disease detection method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0160] Those skilled in the art can understand that, Figure 9 The skilled in the art can understand that,

[0161] In one exemplary embodiment, a computer device is provided, including a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any one of the above method embodiments.

[0162] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method in any one of the above method embodiments.

[0163] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by a processor to implement the steps of the method in any one of the above method embodiments.

[0164] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0165] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0166] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for detecting cable tunnel defects, characterized in that, The method includes: Acquire multiple cable tunnel images collected within a target time period, and acquire multiple sensor data of the cable tunnel within the target time period. The multiple cable tunnel images include multiple target visible light images and multiple target thermal imaging images. Based on the trained cable tunnel recognition model, the defect categories of multiple cable tunnel images are identified respectively, and the defect categories corresponding to each cable tunnel image output by the cable tunnel recognition model are obtained. According to the time series and the instance spatial location corresponding to each cable tunnel image, the defect categories corresponding to multiple cable tunnel images are converted into a first matrix; according to the time series and the location of the sensor that collected each sensor data, multiple sensor data are converted into a second matrix. The first matrix and the second matrix are concatenated to obtain multiple datasets; the multiple datasets are fitted, and the cable tunnel disease level is obtained based on the fitting results.

2. The method according to claim 1, characterized in that, The acquisition of multiple cable tunnel images collected within the target time period includes: Acquire multiple visible light images and multiple thermal images of the cable tunnel collected within the target time period; For each of the visible light images, the number of pixels at each gray level in the visible light image is calculated, and a cumulative distribution function is obtained based on the number of pixels at each gray level; the visible light image is then mapped to gray levels based on the cumulative distribution function to obtain an initial visible light image corresponding to the visible light image. The initial visible light image corresponding to multiple visible light images is input into the trained panoramic segmentation residual model to obtain multiple target visible light images output by the panoramic segmentation residual model; Multiple thermal imaging images are input into the panoramic segmentation residual model to obtain multiple target thermal imaging images output by the panoramic segmentation residual model.

3. The method according to claim 1, characterized in that, The training method for the cable tunnel identification model includes: Obtain sample images of cable tunnels and the types of defects in the cable tunnel sample images; The cable tunnel sample image is input into the initial cable tunnel identification model to obtain the predicted disease category of the cable tunnel sample image output by the initial cable tunnel identification model; The initial cable tunnel identification model is trained based on the differences between the defect categories of the cable tunnel sample images and the predicted defect categories of each cable tunnel sample image. Determine whether the accuracy of the initial cable tunnel identification model reaches the first preset accuracy. If yes, the training is complete and the cable tunnel identification model is obtained. If not, re-execute the steps of obtaining cable tunnel sample images and the disease categories of the cable tunnel sample images.

4. The method according to claim 3, characterized in that, The acquisition of cable tunnel sample images and the types of defects in the cable tunnel sample images include: Obtain initial sample images of the cable tunnel; The initial cable tunnel sample image is segmented using a trained panoramic segmentation residual model to obtain the cable tunnel sample image. The cable tunnel sample image is then labeled with categories to obtain the defect categories of the cable tunnel sample image.

5. The method according to claim 4, characterized in that, The training method for the panoramic segmentation residual model includes: Acquire a target sample image and a segmented image of the target sample image, wherein the target sample image is a visible light sample image of a cable tunnel or a thermal imaging sample image of a cable tunnel; The target sample image is input into the pooling layer of the initial panoramic segmentation residual model to obtain sample features; The sample features are input into the fusion network of the initial panoramic segmentation residual model to obtain sample fusion features; The sample fusion features are input into the fully connected layer of the initial panoramic segmentation residual model to obtain the predicted segmentation image; The initial panoramic segmentation residual model is trained based on the difference between the segmented image and the predicted segmented image; Determine whether the accuracy of the initial panoramic segmentation residual model reaches the second preset accuracy. If yes, the training is complete and the panoramic segmentation residual model is obtained. If no, the steps of obtaining the target sample image and the segmented image of the target sample image are executed again.

6. The method according to claim 5, characterized in that, The fusion network comprises N network layers connected in sequence, wherein the i-th network layer comprises an i-th residual block and an i-th attention module; N is a positive integer greater than 3, and i is a positive integer greater than or equal to 1 and less than or equal to N; The step of inputting the sample features into the fusion network of the initial panoramic segmentation residual model to obtain sample fusion features includes: For the first network layer, the sample features are input into the first layer residual block, the output of the first layer residual block is input into the first layer attention module, and the sample features and the output of the first layer attention module are fused to obtain the output of the first network layer. For the second network layer, the output of the first residual block is input into the second residual block, the output of the second residual block and the output of the first attention module are input into the second attention module, and the output of the first network layer and the output of the second attention module are fused to obtain the output of the second network layer. For the Nth network layer, the output of the (N-1)th residual block and the output of the (N-2)th layer are input into the Nth residual block. The output of the Nth residual block and the output of the (N-1)th attention module are input into the Nth attention module. The output of the (N-1)th network layer, the output of the (N-1)th residual block, and the output of the Nth attention module are fused to obtain the output of the Nth network layer. For any i-th network layer other than the first, second, and Nth network layers, the output of the (i-2)-th residual block and the output of the (i-1)-th residual block are input into the i-th residual block. The output of the i-th residual block and the output of the (i-1)-th attention module are input into the i-th attention module. The output of the (i-1)-th network layer and the output of the i-th attention module are fused to obtain the output of the i-th network layer. The outputs of the N-layer network are fused to obtain the sample fusion features.

7. The method according to claim 6, characterized in that, The i-th layer attention module includes a first attention module and a second attention module; wherein, the first attention module includes a channel attention processing layer and a first spatial attention processing layer; and the second attention module includes a second spatial attention processing layer. When the target sample image is the target visible light sample image, the features input to the i-th layer attention module are input to the first attention module; When the target sample image is the thermal imaging sample image, the features input to the i-th layer attention module are input to the second attention module.

8. A cable tunnel defect detection device, characterized in that, The device includes: The acquisition module is used to acquire multiple cable tunnel images collected within a target time period, and to acquire multiple sensor data of the cable tunnel within the target time period. The multiple cable tunnel images include multiple target visible light images and multiple target thermal imaging images. The identification module is used to identify the disease category of each of the multiple cable tunnel images based on the trained cable tunnel identification model, and to obtain the disease category corresponding to each cable tunnel image output by the cable tunnel identification model. The conversion module is used to convert the defect categories corresponding to multiple cable tunnel images into a first matrix according to the time series and the instance spatial location corresponding to each cable tunnel image; and to convert multiple sensor data into a second matrix according to the time series and the location of the sensor that collected each sensor data. The determination module is used to concatenate the first matrix and the second matrix to obtain multiple datasets; to fit the multiple datasets and to obtain the cable tunnel disease level based on the fitting results.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images