A method and system for underground cable safety assessment based on cloud-edge collaboration and multi-agent deep reinforcement learning

Abstract

This invention discloses a method and system for underground cable safety assessment based on cloud-edge collaboration and multi-agent deep reinforcement learning. The method includes: acquiring temperature, vibration, deformation, and image data from multiple acquisition nodes of the underground cable; uploading this data to edge service nodes, where a bidirectional long short-term memory network is used to extract spatiotemporal features from the video image data, temperature, vibration, and deformation data; uploading the preprocessed feature data from the edge service nodes to a cloud computing service center for multi-source feature data fusion; and training a cable safety status assessment model by constructing a multi-agent dual-adversarial deep Q-network, using the fused feature data as the state input of the network, thereby achieving the identification and assessment of the underground cable safety status. This invention achieves effective feature extraction and fusion of multi-source data from underground cables, and enables rapid and accurate assessment of the underground cable safety status based on the fused features.

Description

Technical Field

[0001] This invention relates to the field of cable safety assessment technology, and in particular to a method and system for underground cable safety assessment based on cloud-edge collaboration and multi-agent deep reinforcement learning. Background Technology

[0002] With the acceleration of urbanization in my country, the coverage and asset scale of power cables are increasing at an unprecedented rate. In recent years, the proportion of underground cable power supply in the power distribution system has been growing. However, cables are frequently damaged by external forces during use, seriously affecting the safety and reliability of cable power supply. The contradiction between social infrastructure construction and underground cable power supply is becoming increasingly prominent, and how to effectively protect the safety of underground cables during infrastructure construction has become an urgent problem to be solved.

[0003] For monitoring and preventing damage to power networks such as underground cable tunnels, it is difficult to address the issue of external force damage through routine line inspections. For randomly occurring external force damage, only by strengthening real-time monitoring of the lines can their resistance to external forces be improved. Some power departments have introduced technological methods to monitor the status of transmission lines and determine whether they are damaged, such as leakage current detection, power line carrier detection, and point strain sensor detection.

[0004] However, all of the above methods have obvious limitations: the leakage current detection method can only reflect results after the transmission line is damaged, and it cannot work during power outages; the power line carrier detection method faces problems such as high noise, large impedance changes, and large signal transmission losses during monitoring; the point strain sensor detection method can monitor the strain state of the transmission line in real time. When an external force damages the line, the transmission line will strain due to tension. The tension is transmitted through the line to the location of the sensor, causing it to also strain, thus determining whether the cable has been damaged by external force. However, the point strain sensor detection method also has shortcomings. It cannot locate the fault point and can only be placed in some special locations on the transmission line. The monitoring range is not comprehensive enough, and the strain detection value deviates from the actual value to a certain extent, failing to fully reflect the true situation.

[0005] Therefore, how to accurately locate the operating status of underground cables in real time to avoid large-scale power outages and achieve prior knowledge has become an urgent problem and a research hotspot for urban power supply safety. Summary of the Invention

[0006] Purpose of the invention: To address the problems of the prior art, this invention provides a method for safety assessment of underground cables based on cloud-edge collaboration and multi-agent deep reinforcement learning, which solves the problem that the operating status of underground cables cannot be located in real time and accurately during large-scale power outages. This invention also provides a safety assessment system for underground cables based on cloud-edge collaboration and multi-agent deep reinforcement learning.

[0007] Technical solution: According to a first aspect of the present invention, a method for safety assessment of underground cables based on cloud-edge collaboration and multi-agent deep reinforcement learning is provided, the method comprising:

[0008] Construct a cloud-edge collaborative service architecture for data transmission and processing in underground cable operations, including edge service nodes and a cloud computing service center;

[0009] Temperature data, vibration data, deformation data, and video image data of multiple monitoring nodes of underground cables are acquired. The number of monitoring nodes is denoted as M. A sensor with Raman optical time domain reflectance technology is used to collect cable temperature data at point M. A sensor with phase-sensitive time domain reflectance technology is used to collect cable vibration data at point M. A sensor with Brillouin optical time domain analysis technology is used to collect cable deformation data at point M. A high-definition full-time monitoring camera is used to collect key ground image data of the cable at point M.

[0010] Multi-source data obtained from each monitoring node is uploaded to the edge service node. Each edge service node constructs a long short-term memory neural network for the underground cable operation sequence data formed by temperature, vibration, and deformation, thereby achieving noise filtering, redundant data removal, and key feature extraction in the underground cable operation sequence data. At the same time, for underground cable operation data in the form of video images, key features of the image data are extracted by constructing a convolutional neural network and a bidirectional long short-term memory network.

[0011] Each edge service node uploads the multi-source data of underground cable operation after key feature extraction to the cloud computing service center. Based on a fully connected neural network, the multi-source data of underground cable operation after key feature extraction is mapped to a unified dimension to achieve data fusion processing.

[0012] A multi-agent dual-adversarial deep Q-network cable safety status assessment model with a feedback learning mechanism is constructed: an agent is designed for each data monitoring node of the underground cable, and the multi-source fusion feature data of each data monitoring node is used as the state input of the dual-adversarial deep Q-network of each agent. The underground cable safety status assessment model is obtained through extensive interaction training between the multi-agent and the data environment. The output of the cable safety status assessment model of the dual-adversarial deep Q-network of each agent is the assessment of the underground cable safety status.

[0013] Furthermore, the method also includes:

[0014] The multi-agent dual-adversarial deep Q network cable safety status assessment model obtained after training is deployed in a system. The specific deployment steps are as follows: save the network parameters of the dual-adversarial deep Q network in the agents corresponding to each monitoring node after training in the cloud computing service center, read the multi-source data fusion features transmitted by each edge service node in real time, and evaluate the safety status of each cable operation monitoring point.

[0015] Furthermore, including:

[0016] The cloud-edge collaborative service architecture for data transmission and processing in underground cable operations includes:

[0017] An edge service node is constructed, comprising an edge microserver and a macro base station. The edge microserver is used to collect and preprocess raw underground cable operation data from various data acquisition nodes, and the macro base station is used to communicate and forward the processed multi-source data features of underground cable operation to the cloud computing service center.

[0018] The cloud computing service center is used to uniformly manage the multi-source data features from various edge service nodes and to provide a platform for the training and application of underground cable safety status assessment algorithms.

[0019] Furthermore, including:

[0020] Each edge service node constructs a long short-term memory neural network based on the underground cable operation sequence data generated by temperature, vibration, and deformation. This enables noise filtering, redundant data removal, and key feature extraction from the underground cable operation sequence data, specifically including:

[0021] For the underground cable operation sequence data, the temperature, vibration, and deformation formatted data collected by each edge service node are extracted and normalized to obtain normalized data.

[0022] A two-layer long short-term neural network model is constructed to extract temporal features from temperature, vibration, and deformation data. Normalized data is used as the state input of the long short-term neural network. Each layer of the long short-term neural network controls the flow of information through forget gate, input gate, and output gate. The second layer of the long short-term neural network updates its internal state based on the information from the first layer and finally obtains the final feature data of temperature, vibration, and deformation.

[0023] Furthermore, including:

[0024] The process of mapping multi-source data on underground cable operation after key feature extraction to a unified dimension based on a fully connected neural network to achieve data fusion processing specifically includes:

[0025] The input data for the fully connected neural network consists of multi-source data features extracted from key features, while the output data consists of data features mapped to a unified dimension. The data features These represent the processed multi-source feature fusion data, namely, feature data related to cable temperature, feature data related to cable vibration, feature data related to cable deformation, and feature data related to cable ground images.

[0026] Furthermore, including:

[0027] The construction of a multi-agent, dual-adversarial deep Q-network cable security status assessment model with a feedback learning mechanism specifically includes:

[0028] The construction of a multi-agent dual-adversarial deep Q-network cable safety status assessment model with feedback learning mechanism is transformed into solving the problem of minimizing the cable safety status assessment error at point M, including decision variables and objective function;

[0029] The problem of minimizing the cable safety status assessment error at the above M points is remodeled as a Markov decision process, and M cable safety status assessment agents based on a multi-agent bi-adversarial deep Q network are constructed to solve the Markov decision process.

[0030] An intelligent agent model for assessing the safety status of underground cables is obtained through extensive interaction training between multiple agents and the data environment.

[0031] Furthermore, including:

[0032] The problem of minimizing the error in the cable safety status assessment at point M includes decision variables and an objective function;

[0033] The decision variable is: the safety status assessment of the cable data monitoring node at point M, a. i ={e low ,…,e high}, 1≤i≤M, where: i represents the i-th cable data acquisition point, and the safety status assessment is represented by multiple discrete values, e low ,e high These represent the minimum and maximum safety factors for the current cable, respectively.

[0034] The objective function is:

[0035]

[0036] Where: T represents the total number of time slots for safety status assessment of each cable, a′ i,t This indicates the true safety status of each cable.

[0037] Furthermore, including:

[0038] The problem of minimizing the cable safety status assessment error at point M is remodeled as a Markov decision process, expressed as:

[0039]

[0040] a i,t ={e low ,...,e high}, 1≤i≤M, 1≤t≤T

[0041] r i,t =-(a′) i,t -a i,t ) 2

[0042] Where: s i,t a i,t r i,t Let each represent the state, behavior, and reward function of the Markov decision process associated with the safety status assessment of each underground cable. These represent the processed multi-source feature fusion data, namely, feature data related to cable temperature, feature data related to cable vibration, feature data related to cable deformation, and feature data related to cable ground images.

[0043] Furthermore, including:

[0044] The construction of M cable safety state assessment agents based on a multi-agent bi-adversarial deep Q-network to solve the Markov decision process specifically includes:

[0045] The constructed M cable safety status assessment agents include value function networks with identical structures. and objective value function network The structure consists of two layers: the value function network and the target value function network. Both consist of one input layer, multiple hidden layers, and two output layers: the behavioral value function output layer and the target value function output layer. and the output layer of the advantage function

[0046] The output value of the value function network Output layer by behavioral value function and the output layer of the advantage function Joint decision, namely: Among them: U and These represent the number of discrete action dimensions that the agent can choose and the other security state evaluation levels besides the current evaluation output 'a', respectively.

[0047] Furthermore, including:

[0048] The method involves training an intelligent agent model for assessing the safety status of underground cables through extensive interactions between multiple agents and the data environment. The training steps include:

[0049] Initialize the data environment and repeat the iteration steps until the preset number of training segments K is reached. The length of each training segment is set to N, and the update frequency of the agent is set to K. update ;

[0050] The iterative steps include:

[0051] The processed multi-source feature fusion data is input into... The decision made by the intelligent agent i for assessing the cable safety status at each point is the current cable safety status e. i,t ;

[0052] The reward function r of the agent receiving feedback i,t Multi-source fusion characteristic data for the next cable operation Evaluation output a i,t and current multi-source fusion feature data Forming a quadruple (s i,t ,a i,t ,s i,t+1 ,r i,t ) Stored in the shared experience pool of the intelligent agent;

[0053] If the agent's update condition is met, that is, the number of segments K that satisfy the update frequency is reached. i %K update =0, where: K i Given the current number of segments, a priority experience replay mechanism combined with an agent update mechanism is proposed. Each agent draws a small batch of quadruplets (s, a, s, r) from the shared experience pool as training samples. The agents are then trained using a multi-agent bi-adversarial deep Q-network algorithm combined with a joint loss function based on the priority experience replay mechanism. The value function network is then updated accordingly. Update.

[0054] Furthermore, including:

[0055] The specific principle of the priority experience replay mechanism implemented during the agent update process is as follows:

[0056] Calculate the sampling weight of each empirical tuple in the empirical pool, and perform mini-batch quadruples sampling based on the sampling weight. The formula for calculating the sampling weight is as follows:

[0057]

[0058] Where: κ represents the sampling weight of the priority experience replay mechanism; This represents the total number of experience tuples stored for each agent in the shared experience pool. This indicates the priority of a particular empirical tuple among all currently stored empirical tuples; p represents the absolute value of the TD error between a particular empirical tuple and all other empirical tuples. Let sum(·) represent the set of absolute values ​​of the TD errors of a certain empirical tuple and all other empirical tuples, where sum(·) is a summation function; and φ represents a sampling weight coefficient. and Let s' and a' represent the value function and target value function of agent i, respectively, and let s' and a' represent the cable operation characteristic data and target action value of a certain empirical tuple in the next time slot. This represents the target value of agent i.

[0059] Furthermore, including:

[0060] The joint loss function utilizing the multi-agent dual-adversarial deep Q-network algorithm combined with the priority experience replay mechanism is expressed as:

[0061]

[0062] In each update round, the agent is trained by minimizing the joint loss function constructed above.

[0063] On the other hand, the present invention also provides an underground cable safety assessment system based on cloud-edge collaboration and multi-agent deep reinforcement learning, the system comprising:

[0064] The service architecture building module is used to build a cloud-edge collaborative service architecture for data transmission and processing in underground cable operations, including edge service nodes and cloud computing service centers;

[0065] The data acquisition module is used to acquire temperature data, vibration data, deformation data, and video image data of multiple monitoring nodes of the underground cable. The number of monitoring nodes is denoted as M. The temperature data of the cable at point M is acquired using a sensor with Raman optical time domain reflectance technology, the vibration data of the cable at point M is acquired using a sensor with phase-sensitive time domain reflectance technology, the deformation data of the cable at point M is acquired using a sensor with Brillouin optical time domain analysis technology, and key ground image data of the cable at point M is acquired using a high-definition full-time monitoring camera.

[0066] The preprocessing module uploads multi-source data from various monitoring nodes to edge service nodes. Each edge service node constructs a long short-term memory neural network for the underground cable operation sequence data formed by temperature, vibration, and deformation, thereby achieving noise filtering, redundant data removal, and key feature extraction in the underground cable operation sequence data. At the same time, for underground cable operation data in the form of video images, key features of the image data are extracted by constructing a convolutional neural network and a bidirectional long short-term memory network.

[0067] The fusion processing module is used to upload the multi-source data of underground cable operation after key features are extracted from each edge service node to the cloud computing service center. Based on a fully connected neural network, the multi-source data of underground cable operation after key features are extracted is mapped to a unified dimension to achieve data fusion processing.

[0068] The safety status assessment module is used to construct a multi-agent dual-adversarial deep Q-network cable safety status assessment model with a feedback learning mechanism: an agent is designed for each data monitoring node of the underground cable, and the multi-source fusion feature data of each data monitoring node is used as the state input of the dual-adversarial deep Q-network of each agent. The underground cable safety status assessment model is obtained through extensive interaction training between the multi-agent and the data environment. The output of the cable safety status assessment model of each agent dual-adversarial deep Q-network is the assessment of the safety status of the underground cable.

[0069] Furthermore, the system also includes:

[0070] The system deployment module is used to deploy the multi-agent dual-adversarial deep Q network cable safety status assessment model obtained after training. The specific deployment steps are as follows: save the network parameters of the dual-adversarial deep Q network in the corresponding agent of each monitoring node after training in the cloud computing service center, read the multi-source data fusion features transmitted by each edge service node in real time, and evaluate the safety status of each cable operation monitoring point.

[0071] Finally, the present invention also provides a computer-readable storage medium storing computer instructions that, when executed, perform the aforementioned method for safety assessment of underground cables based on cloud-edge collaboration and multi-agent deep reinforcement learning.

[0072] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0073] This invention provides a method and system for assessing the safety status of underground cables based on a multi-agent dual-adversarial deep Q-network, which considers multi-source fusion data feature input under a cloud-edge collaborative framework. Compared with centralized data collection and processing, the proposed cloud-edge collaborative architecture for underground cable operation data transmission and processing can effectively reduce computational burden and communication overload, and is suitable for scenarios such as real-time assessment of the safety status of underground cables.

[0074] Compared with methods based on single-type data input, the method and system of this application can achieve a more comprehensive safety status assessment. With the development of Internet of Things technology, data acquisition efficiency has been greatly improved. By using high-precision and highly intelligent sensors to collect massive amounts of data, it is not necessary to know the accurate dynamic model of the underground cable operation. The underground cable safety status assessment model can be efficiently constructed through a data-driven approach.

[0075] Compared with other deep learning-based methods that consider the features of multi-source data input, the proposed method transforms the data classification problem into a Markov decision process, fully tapping the potential of time-series data applications and improving data utilization efficiency. In summary, the method of this invention can achieve high-accuracy assessment of the safety status of underground cables.

[0076] With the rapid development of IoT technology, the widespread application of highly sensitive and precise sensors has made it possible to collect real-time and accurate information on the operating status of underground cables. Therefore, comprehensively analyzing the characteristics of multi-source sensing data and using artificial intelligence methods to intelligently assess the safety status of underground cables will provide significant support for improving the safety of urban power cables. Attached Figure Description

[0077] Figure 1 This is a flowchart of an underground cable safety assessment method based on cloud-edge collaboration and multi-agent deep reinforcement learning, as described in Embodiment 1 of the present invention.

[0078] Figure 2 This is a framework diagram of the multi-agent dual-adversarial deep Q-network algorithm that considers multi-source data feature processing and input as described in Embodiment 1 of the present invention;

[0079] Figure 3 This is a cloud-edge collaborative framework diagram for transmitting and processing multi-source fusion data features of underground cables, as described in Embodiment 1 of the present invention.

[0080] Figure 4 This is a flowchart illustrating the construction of a multi-agent dual-adversarial deep Q-network cable security status assessment model with a feedback learning mechanism, as described in Embodiment 1 of the present invention.

[0081] Figure 5 This is a schematic diagram of the underground cable safety assessment system based on cloud-edge collaboration and multi-agent deep reinforcement learning, as described in Embodiment 3 of the present invention. Detailed Implementation

[0082] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] Example 1

[0084] like Figure 1 As shown, this invention proposes a safety assessment method for underground cables based on cloud-edge collaboration and multi-agent deep reinforcement learning. In this embodiment, the temperature, deformation, vibration, and other data in the underground cable operation data are obtained by modeling the underground cable operation process using COMSOL and ANSYS simulation platforms. The ground operation status of the underground cable is obtained by capturing images from a real camera. The specific steps include:

[0085] S1. Construct a cloud-edge collaborative service architecture for data transmission and processing in underground cable operations, including edge service nodes and cloud computing service centers.

[0086] S1.1 Construct edge service nodes, including edge microservers and macro base stations. The edge microservers are used to collect and preprocess raw underground cable operation data from various data acquisition nodes. The macro base stations are used to communicate and forward the processed multi-source data features of underground cable operation to the cloud computing service center.

[0087] S1.2 Build a cloud computing service center to uniformly manage the multi-source data features from various edge service nodes, and provide a platform for the training and application of underground cable safety status assessment algorithms.

[0088] like Figure 3 As shown, sensors are used to collect data and the collection results are sent to an edge microserver. The edge microserver processes the data and then sends it to a macro base station.

[0089] S2. Acquire temperature, vibration, deformation, and video image data from multiple monitoring nodes of the underground cable. However, different data types suffer from noise and data loss due to the influence of the acquisition devices and communication links. Specifically, a sensor with Raman time-domain reflectometry (TDDR) technology is used to acquire cable temperature data at point M; a sensor with phase-sensitive TDDR technology is used to acquire cable vibration data at point M; a sensor with Brillouin time-domain analysis technology is used to acquire cable deformation data at point M; and a high-definition, real-time monitoring camera is used to acquire key ground image data of the cable at point M.

[0090] S3. Upload the multi-source data obtained from each acquisition node to the edge service node. Each edge service node constructs a long short-term memory neural network for underground cable operation sequence data such as temperature, vibration, and deformation to achieve noise filtering, redundant data removal, and key feature extraction. Simultaneously, for underground cable operation data in video image format, a convolutional neural network and a bidirectional long short-term memory network are constructed to extract key features from the image data, such as... Figure 2 As shown.

[0091] S3.1 First, the formatted data such as temperature, vibration, and deformation collected from each edge service node for time-series feature data will be normalized:

[0092]

[0093] Among them, D w D z and D x These are the raw data for temperature, vibration, and deformation, respectively; D w,norm D z,norm and D x,norm These are normalized data for temperature, vibration, and deformation, respectively; D w,max D z,max D x,max and D w,min D z,min D x,min These represent the maximum and minimum values ​​corresponding to the original temperature, vibration, and deformation data, respectively. A two-layer long short-term neural network model is constructed to extract temporal features from the temperature, vibration, and deformation data. Normalized data is used as the state input of the long short-term neural network. Each layer of the long short-term neural network controls the flow of information through forget gates, input gates, and output gates. The second layer of the long short-term neural network updates its internal state based on the information from the first layer and finally obtains the final feature data of temperature, vibration, and deformation.

[0094] The specific process can be represented as follows:

[0095] D t ={D w,norm D z,norm D x,norm}

[0096] i t =δ(W i ·[H t-1 D t ]+V i )

[0097] f t =δ(W f ·[H t-1D t ]+V f )

[0098]

[0099] O t =δ(W O ·[H t-1 D t ]+V O )

[0100] H t =O t *tanh(C t )

[0101] Where: i t f t O t These represent the forget gate, input gate, and output gate, respectively; C t Indicates the state of a memory unit; The values ​​of candidate states for memory cells are represented by δ and tanh, respectively, which represent the Sigmoid and tanh activation functions; W and V represent the weight matrix and bias matrix, respectively; D t To input underground cable operation data; H t Features of underground cable operation data extracted.

[0102] S3.2. The temporal features of the cable ground operation status data in the form of video images are initially extracted by a convolutional neural network containing multiple convolutional layers, and then input into a bidirectional long short-term memory neural network to further extract key features.

[0103] S4. Each edge service node uploads the pre-processed multi-source data of underground cable operation to the cloud computing service center. Based on a fully connected neural network, the pre-processed multi-source data is mapped to a unified dimension to achieve data fusion processing.

[0104] In this embodiment, the above steps are mainly completed by a deep neural network, whose input data is preprocessed multi-source data features, and whose output data is data features mapped to a unified dimension. These represent the processed multi-source feature fusion data, namely, feature data related to cable temperature, feature data related to cable vibration, feature data related to cable deformation, and feature data related to cable ground images.

[0105] S5. Construct a multi-agent, dual-adversarial deep Q-network cable safety status assessment model with a feedback learning mechanism. Design an agent for each data acquisition point of the underground cable. Use the multi-source fused feature data from each data acquisition point as the state input of each agent's dual-adversarial deep Q-network. Train the underground cable safety status assessment model through extensive interaction between the multi-agent and the data environment. The output of each agent's dual-adversarial deep Q-network model is the assessment of the underground cable's safety status. Figure 4 As shown, the specific implementation steps are as follows:

[0106] S5.1 First, the construction of a multi-agent dual-adversarial deep Q-network cable safety status assessment model with feedback learning mechanism is transformed into a problem of minimizing the cable safety status assessment error at point M, including decision variables and objective function;

[0107] The decision variable is: a) for the safety status assessment of the cable data acquisition point at location M. i ={e low ,...,e high}, 1≤i≤M;

[0108] Where: i represents the i-th cable data acquisition point, and the safety status assessment is represented by multiple discrete values, e low ,e high These represent the minimum and maximum safety factors for the current cable, respectively.

[0109] The objective function is: Where: T represents the total number of time slots for safety status assessment of each cable, a i ′ ,t This indicates the true safety status of each cable.

[0110] S5.2 Secondly, the problem of minimizing the cable safety status assessment error at point M is remodeled as a Markov decision process as follows:

[0111]

[0112] a i,t ={e low ,...,e high}, 1≤i≤M, 1≤t≤T;

[0113] r i,t =-(a′) i,t -a i,t ) 2 ;

[0114] Where: s i,t a i,t r i,tLet each represent the state, behavior, and reward function of the Markov decision process associated with the safety status assessment of each underground cable. These represent the processed multi-source feature fusion data, namely, feature data related to cable temperature, feature data related to cable vibration, feature data related to cable deformation, and feature data related to cable ground images.

[0115] S5.3 Then, construct M cable safety status assessment agents based on a multi-agent bi-adversarial deep Q-network to solve the above Markov decision process.

[0116] The constructed M agents are mainly composed of value function networks with identical structures. and objective value function network constitute.

[0117] Both the value function network and the target value function network consist of one input layer, multiple hidden layers, and two output layers, with the two output layers being the behavior value function output layers. and the output layer of the advantage function

[0118] Specifically, to avoid the overfitting problem of value function network outputs in existing deep reinforcement learning techniques, an adversarial network framework is proposed, namely, the output value of the value function network... Output layer by behavioral value function and the output layer of the advantage function Joint decision, namely:

[0119]

[0120] Among them: U and These represent the number of discrete action dimensions that the agent can choose and the other security state evaluation levels besides the current evaluation output 'a', respectively.

[0121] S5.4 Finally, through extensive interaction training between multiple agents and the data environment, an intelligent agent model for assessing the safety status of underground cables is obtained. The trained agent is then deployed in the underground cable safety status assessment system. The training steps mainly include:

[0122] Initialize the data environment and repeat the iteration steps until the preset number of training segments K is reached. The length of each training segment is set to N, and the update frequency of the agent is set to K. update ;

[0123] The iterative steps include:

[0124] The processed multi-source feature fusion data is input to... The decision made by the intelligent agent i for assessing the cable safety status at each point is the current cable safety status e.i,t ;

[0125] The reward function r that the agent receives feedback from i,t Multi-source fusion characteristic data for the next cable operation Evaluation output a i,t and current multi-source fusion feature data Forming a quadruple (s i,t ,a i,t ,s i,t+1 ,r i,t ) Stored in the shared experience pool of the intelligent agent;

[0126] If the agent's update condition is met, then the number of segments K that satisfy the update frequency has been reached. i %K update =0, where: K i Let be the current segment number. A priority experience replay mechanism combined with an agent update mechanism is proposed. Each agent draws a mini-batch of quadruplets (s, a, s, r) from the experience pool as training samples. The agent is trained using a joint loss function combining a multi-agent bi-adversarial deep Q-network algorithm and the priority experience replay mechanism. The value function network is then... Update.

[0127] The specific principle of the priority experience replay mechanism implemented during the agent update process is as follows: calculate the sampling weight of each experience tuple in the experience pool, and perform mini-batch quadruple sampling according to the sampling weight. The formula for calculating the sampling weight is as follows:

[0128]

[0129] Where: κ represents the sampling weight of the priority experience replay mechanism; This represents the total number of experience tuples stored for each agent in the shared experience pool. This indicates the priority of a particular empirical tuple among all currently stored empirical tuples; p represents the absolute value of the TD error between a particular empirical tuple and all other empirical tuples. Let sum(·) represent the set of absolute values ​​of the TD errors of a certain empirical tuple and all other empirical tuples, where sum(·) is a summation function; and φ represents a sampling weight coefficient. and Let s' and a' represent the value function and target value function of agent i, respectively, and let s' and a' represent the cable operation characteristic data and target action value of a certain empirical tuple in the next time slot. This represents the target value of agent i.

[0130] like Figure 2 As shown, the joint loss function of the multi-agent bi-adversarial deep Q-network algorithm proposed in the agent update stage during the implementation of this embodiment is as follows:

[0131]

[0132] In each update round, the agent is trained by minimizing the joint loss function constructed above.

[0133] Meanwhile, the objective value function network is updated using a soft update mechanism, namely: Where σ is the soft update mechanism coefficient, used to constrain the update magnitude of the objective value function network.

[0134] Example 2

[0135] Based on Example 1, the method of the present invention further includes:

[0136] S6. Deploy the obtained underground cable safety status assessment model systematically.

[0137] The specific deployment steps are as follows: save the network parameters of the dual adversarial deep Q network in the intelligent agents of each underground cable data monitoring point after training in the cloud computing service center, read the multi-source data fusion characteristics transmitted by each edge service node in real time, and evaluate the safety status of each cable operation monitoring point.

[0138] Example 3

[0139] like Figure 5 As shown, the present invention also provides an underground cable safety assessment system based on cloud-edge collaboration and multi-agent deep reinforcement learning, the system comprising:

[0140] The service architecture building module is used to build a cloud-edge collaborative service architecture for data transmission and processing in underground cable operations, including edge service nodes and cloud computing service centers;

[0141] The data acquisition module is used to acquire temperature data, vibration data, deformation data, and video image data of multiple monitoring nodes of the underground cable. The number of monitoring nodes is denoted as M. The temperature data of the cable at point M is acquired using a sensor with Raman optical time domain reflectance technology, the vibration data of the cable at point M is acquired using a sensor with phase-sensitive time domain reflectance technology, the deformation data of the cable at point M is acquired using a sensor with Brillouin optical time domain analysis technology, and key ground image data of the cable at point M is acquired using a high-definition full-time monitoring camera.

[0142] The preprocessing module uploads multi-source data from various monitoring nodes to edge service nodes. Each edge service node constructs a long short-term memory neural network for the underground cable operation sequence data formed by temperature, vibration, and deformation, thereby achieving noise filtering, redundant data removal, and key feature extraction in the underground cable operation sequence data. At the same time, for underground cable operation data in the form of video images, key features of the image data are extracted by constructing a convolutional neural network and a bidirectional long short-term memory network.

[0143] The fusion processing module is used to upload the multi-source data of underground cable operation after key features are extracted from each edge service node to the cloud computing service center. Based on a fully connected neural network, the multi-source data of underground cable operation after key features are extracted is mapped to a unified dimension to achieve data fusion processing.

[0144] The safety status assessment module is used to construct a multi-agent dual-adversarial deep Q-network cable safety status assessment model with a feedback learning mechanism: an agent is designed for each data monitoring node of the underground cable, and the multi-source fusion feature data of each data monitoring node is used as the state input of the dual-adversarial deep Q-network of each agent. The underground cable safety status assessment model is obtained through extensive interaction training between the multi-agent and the data environment. The output of the cable safety status assessment model of each agent dual-adversarial deep Q-network is the assessment of the safety status of the underground cable.

[0145] Example 4

[0146] Based on Example 3, the system further includes:

[0147] The system deployment module is used to deploy the multi-agent dual-adversarial deep Q network cable safety status assessment model obtained after training. The specific deployment steps are as follows: save the network parameters of the dual-adversarial deep Q network in the corresponding agent of each monitoring node after training in the cloud computing service center, read the multi-source data fusion features transmitted by each edge service node in real time, and evaluate the safety status of each cable operation monitoring point.

[0148] Finally, the present invention also provides a computer-readable storage medium storing computer instructions that, when executed, perform the aforementioned method for safety assessment of underground cables based on cloud-edge collaboration and multi-agent deep reinforcement learning.

[0149] The other technical features of the underground cable safety assessment system based on cloud-edge collaboration and multi-agent deep reinforcement learning described in this invention are similar to the corresponding underground cable safety assessment method based on cloud-edge collaboration and multi-agent deep reinforcement learning, and will not be repeated here.

[0150] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0152] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Therefore, if these modifications and variations to the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for safety assessment of underground cables based on cloud-edge collaboration and multi-agent deep reinforcement learning, characterized in that, The method includes: Construct a cloud-edge collaborative service architecture for data transmission and processing in underground cable operations, including edge service nodes and a cloud computing service center; Temperature data, vibration data, deformation data, and video image data of multiple monitoring nodes of underground cables are acquired. The number of monitoring nodes is denoted as M. A sensor with Raman optical time domain reflectance technology is used to collect cable temperature data at point M. A sensor with phase-sensitive time domain reflectance technology is used to collect cable vibration data at point M. A sensor with Brillouin optical time domain analysis technology is used to collect cable deformation data at point M. A high-definition full-time monitoring camera is used to collect key ground image data of the cable at point M. Multi-source data obtained from each monitoring node is uploaded to the edge service node. Each edge service node constructs a long short-term memory neural network for the underground cable operation sequence data formed by temperature, vibration, and deformation, thereby achieving noise filtering, redundant data removal, and key feature extraction in the underground cable operation sequence data. At the same time, for underground cable operation data in the form of video images, key features of the image data are extracted by constructing a convolutional neural network and a bidirectional long short-term memory network. Each edge service node uploads the multi-source data of underground cable operation after key feature extraction to the cloud computing service center. Based on a fully connected neural network, the multi-source data of underground cable operation after key feature extraction is mapped to a unified dimension to achieve data fusion processing. Construct a multi-agent dual-adversarial deep Q-network cable safety status assessment model with feedback learning mechanism: Design an agent for each data monitoring node of the underground cable, use the multi-source fusion feature data of each data monitoring node as the state input of the dual-adversarial deep Q-network of each agent, and obtain the underground cable safety status assessment model through extensive interaction training between the multi-agent and the data environment. The output of the dual-adversarial deep Q-network cable safety status assessment model of each agent is the assessment of the underground cable safety status. The construction of a multi-agent, dual-adversarial deep Q-network cable security status assessment model with a feedback learning mechanism specifically includes: The construction of a multi-agent dual-adversarial deep Q-network cable safety status assessment model with feedback learning mechanism is transformed into solving the problem of minimizing the cable safety status assessment error at point M, including decision variables and objective function; The problem of minimizing the cable safety status assessment error at the above M points is remodeled as a Markov decision process, and M cable safety status assessment agents based on a multi-agent bi-adversarial deep Q network are constructed to solve the Markov decision process. An intelligent agent model for assessing the safety status of underground cables was obtained through extensive interaction training between multiple agents and the data environment. The problem of minimizing the error in the cable safety status assessment at point M includes decision variables and an objective function; The decision variable is: the safety status assessment of the cable data monitoring node at point M, a. i ={e low ,...,e high }, 1≤i≤M, where: i represents the i-th cable data acquisition point, and the safety status assessment is represented by multiple discrete values, e low ,e high These represent the minimum and maximum safety factors for the current cable, respectively. The objective function is: Where: T represents the total number of time slots for safety status assessment of each cable, a′ i,t This indicates the true safety status of each cable. The problem of minimizing the cable safety status assessment error at point M is remodeled as a Markov decision process, expressed as: to i,t ={and low ,...,And high },1≤i≤M,1≤t≤T r i,t =-(a′ i,t -a i,t ) 2 Where: s i,t a i,t r i,t Let each represent the state, behavior, and reward function of the Markov decision process associated with the safety status assessment of each underground cable. These represent the processed multi-source feature fusion data, namely, feature data related to cable temperature, feature data related to cable vibration, feature data related to cable deformation, and feature data related to cable ground images; The construction of M cable safety state assessment agents based on a multi-agent bi-adversarial deep Q-network to solve the Markov decision process specifically includes: The constructed M cable safety status assessment agents include value function networks with identical structures. and objective value function network The structure consists of two layers: the value function network and the target value function network. Both consist of one input layer, multiple hidden layers, and two output layers: the behavioral value function output layer and the target value function output layer. and the output layer of the advantage function The output value of the value function network Output layer by behavioral value function and the output layer of the advantage function Joint decision, namely: Among them: U and These represent the number of discrete action dimensions that the agent can choose and the evaluation levels of other security states besides the current evaluation output 'a', respectively. The method involves training an intelligent agent model for assessing the safety status of underground cables through extensive interactions between multiple agents and the data environment. The training steps include: Initialize the data environment and repeat the iteration steps until the preset number of training segments K is reached. The length of each training segment is set to N, and the update frequency of the agent is set to K. update ; The iterative steps include: The processed multi-source feature fusion data is input into... The decision made by the intelligent agent i for assessing the cable safety status at each point is the current cable safety status e. i,t ; The reward function r of the agent receiving feedback i,t Multi-source fusion characteristic data for the next cable operation Evaluation output a i,t and current multi-source fusion feature data Forming a quadruple (s i,t ,a i,t ,s i,t+1 ,r i,t ) Stored in the shared experience pool of the intelligent agent; If the agent's update condition is met, that is, the number of segments K that satisfy the update frequency is reached. i %K update =0, where: K i Given the current number of segments, a priority experience replay mechanism combined with an agent update mechanism is proposed. Each agent draws a small batch of quadruplets (s, a, s, r) from the shared experience pool as training samples. The agents are then trained using a multi-agent bi-adversarial deep Q-network algorithm combined with a joint loss function based on the priority experience replay mechanism. The value function network is then updated accordingly. Update; The specific principle of the priority experience replay mechanism implemented during the agent update process is as follows: Calculate the sampling weight of each empirical tuple in the empirical pool, and perform mini-batch quadruples sampling based on the sampling weight. The formula for calculating the sampling weight is as follows: Where: κ represents the sampling weight of the priority experience replay mechanism; This represents the total number of experience tuples stored for each agent in the shared experience pool. This indicates the priority of a particular empirical tuple among all currently stored empirical tuples; p represents the absolute value of the TD error between a particular empirical tuple and all other empirical tuples. Let sum(·) represent the set of absolute values ​​of the TD errors of a certain empirical tuple and all other empirical tuples, where sum(·) is a summation function; and φ represents a sampling weight coefficient. and Let s' and a' represent the value function and target value function of agent i, respectively, and let s' and a' represent the cable operation characteristic data and target action value of a certain empirical tuple in the next time slot. This represents the target value of agent i; The joint loss function utilizing the multi-agent dual-adversarial deep Q-network algorithm combined with the priority experience replay mechanism is expressed as: In each update round, the agent is trained by minimizing the joint loss function constructed above.

2. The underground cable safety assessment method based on cloud-edge collaboration and multi-agent deep reinforcement learning as described in claim 1, characterized in that, The method also includes: The multi-agent dual-adversarial deep Q network cable safety status assessment model obtained after training is deployed in a system. The specific deployment steps are as follows: save the network parameters of the dual-adversarial deep Q network in the agents corresponding to each monitoring node after training in the cloud computing service center, read the multi-source data fusion features transmitted by each edge service node in real time, and evaluate the safety status of each cable operation monitoring point.

3. The safety assessment method as described in claim 1 or 2, characterized in that, The cloud-edge collaborative service architecture for data transmission and processing in underground cable operations includes: An edge service node is constructed, comprising an edge microserver and a macro base station. The edge microserver is used to collect and preprocess raw underground cable operation data from various data acquisition nodes, and the macro base station is used to communicate and forward the processed multi-source data features of underground cable operation to the cloud computing service center. The cloud computing service center is used to uniformly manage the multi-source data features from various edge service nodes and to provide a platform for the training and application of underground cable safety status assessment algorithms.

4. The security assessment method as described in claim 1 or 2, characterized in that, Each edge service node constructs a long short-term memory neural network based on the underground cable operation sequence data generated by temperature, vibration, and deformation. This enables noise filtering, redundant data removal, and key feature extraction from the underground cable operation sequence data, specifically including: For the underground cable operation sequence data, the temperature, vibration, and deformation formatted data collected by each edge service node are extracted and normalized to obtain normalized data. A two-layer long short-term neural network model is constructed to extract temporal features from temperature, vibration, and deformation data. Normalized data is used as the state input of the long short-term neural network. Each layer of the long short-term neural network controls the flow of information through forget gate, input gate, and output gate. The second layer of the long short-term neural network updates its internal state based on the information from the first layer and finally obtains the final feature data of temperature, vibration, and deformation.

5. The safety assessment method as described in claim 1 or 2, characterized in that, The process of mapping multi-source data on underground cable operation after key feature extraction to a unified dimension based on a fully connected neural network to achieve data fusion processing specifically includes: The input data for the fully connected neural network consists of multi-source data features extracted from key features, while the output data consists of data features mapped to a unified dimension. The data features These represent the processed multi-source feature fusion data, namely, feature data related to cable temperature, feature data related to cable vibration, feature data related to cable deformation, and feature data related to cable ground images.

6. A safety assessment system for underground cables based on cloud-edge collaboration and multi-agent deep reinforcement learning, characterized in that, The system includes: The service architecture building module is used to build a cloud-edge collaborative service architecture for data transmission and processing in underground cable operations, including edge service nodes and cloud computing service centers; The data acquisition module is used to acquire temperature data, vibration data, deformation data, and video image data of multiple monitoring nodes of the underground cable. The number of monitoring nodes is denoted as M. The temperature data of the cable at point M is acquired using a sensor with Raman optical time domain reflectance technology, the vibration data of the cable at point M is acquired using a sensor with phase-sensitive time domain reflectance technology, the deformation data of the cable at point M is acquired using a sensor with Brillouin optical time domain analysis technology, and key ground image data of the cable at point M is acquired using a high-definition full-time monitoring camera. The preprocessing module uploads multi-source data from various monitoring nodes to edge service nodes. Each edge service node constructs a long short-term memory neural network for the underground cable operation sequence data formed by temperature, vibration, and deformation, thereby achieving noise filtering, redundant data removal, and key feature extraction in the underground cable operation sequence data. At the same time, for underground cable operation data in the form of video images, key features of the image data are extracted by constructing a convolutional neural network and a bidirectional long short-term memory network. The fusion processing module is used to upload the multi-source data of underground cable operation after key features are extracted from each edge service node to the cloud computing service center. Based on a fully connected neural network, the multi-source data of underground cable operation after key features are extracted is mapped to a unified dimension to achieve data fusion processing. The safety status assessment module is used to construct a multi-agent dual-adversarial deep Q network cable safety status assessment model with a feedback learning mechanism: an agent is designed for each data monitoring node of the underground cable, and the multi-source fusion feature data of each data monitoring node is used as the state input of the dual-adversarial deep Q network of each agent. The underground cable safety status assessment model is obtained through extensive interaction training between the multi-agent and the data environment. The output of the cable safety status assessment model of each agent dual-adversarial deep Q network is the assessment of the safety status of the underground cable. The construction of a multi-agent, dual-adversarial deep Q-network cable security status assessment model with a feedback learning mechanism specifically includes: The construction of a multi-agent dual-adversarial deep Q-network cable safety status assessment model with feedback learning mechanism is transformed into solving the problem of minimizing the cable safety status assessment error at point M, including decision variables and objective function; The problem of minimizing the cable safety status assessment error at the above M points is remodeled as a Markov decision process, and M cable safety status assessment agents based on a multi-agent bi-adversarial deep Q network are constructed to solve the Markov decision process. An intelligent agent model for assessing the safety status of underground cables was obtained through extensive interaction training between multiple agents and the data environment. The problem of minimizing the error in the cable safety status assessment at point M includes decision variables and an objective function; The decision variable is: the safety status assessment of the cable data monitoring node at point M, a. i ={e low ,...,e high }, 1≤i≤M, where: i represents the i-th cable data acquisition point, and the safety status assessment is represented by multiple discrete values, e low ,e high These represent the minimum and maximum safety factors for the current cable, respectively. The objective function is: Where: T represents the total number of time slots for safety status assessment of each cable, a′ i,t This indicates the true safety status of each cable. The problem of minimizing the cable safety status assessment error at point M is remodeled as a Markov decision process, expressed as: to i,t ={and low ,...,And high },1≤i≤M,1≤t≤T r i,t =-(a′ i,t -a i,t ) 2 Where: s i,t a i,t r i,t Let each represent the state, behavior, and reward function of the Markov decision process associated with the safety status assessment of each underground cable. These represent the processed multi-source feature fusion data, namely, feature data related to cable temperature, feature data related to cable vibration, feature data related to cable deformation, and feature data related to cable ground images; The construction of M cable safety state assessment agents based on a multi-agent bi-adversarial deep Q-network to solve the Markov decision process specifically includes: The constructed M cable safety status assessment agents include value function networks with identical structures. and objective value function network The structure consists of two layers: the value function network and the target value function network. Both consist of one input layer, multiple hidden layers, and two output layers: the behavioral value function output layer and the target value function output layer. and the output layer of the advantage function The output value of the value function network Output layer by behavioral value function and the output layer of the advantage function Joint decision, namely: Among them: U and These represent the number of discrete action dimensions that the agent can choose and the evaluation levels of other security states besides the current evaluation output 'a', respectively. The method involves training an intelligent agent model for assessing the safety status of underground cables through extensive interactions between multiple agents and the data environment. The training steps include: Initialize the data environment and repeat the iteration steps until the preset number of training segments K is reached. The length of each training segment is set to N, and the update frequency of the agent is set to K. update ; The iterative steps include: The processed multi-source feature fusion data is input into... The decision made by the intelligent agent i for assessing the cable safety status at each point is the current cable safety status e. i,t ; The reward function r of the agent receiving feedback i,t Multi-source fusion characteristic data for the next cable operation Evaluation output a i,t and current multi-source fusion feature data Forming a quadruple (s i,t ,a i,t ,s i,t+1 ,r i,t ) Stored in the shared experience pool of the intelligent agent; If the agent's update condition is met, that is, the number of segments K that satisfy the update frequency is reached. i %K update =0, where: K i Given the current number of segments, a priority experience replay mechanism combined with an agent update mechanism is proposed. Each agent draws a small batch of quadruplets (s, a, s, r) from the shared experience pool as training samples. The agents are then trained using a multi-agent bi-adversarial deep Q-network algorithm combined with a joint loss function based on the priority experience replay mechanism. The value function network is then updated accordingly. Update; The specific principle of the priority experience replay mechanism implemented during the agent update process is as follows: Calculate the sampling weight of each empirical tuple in the empirical pool, and perform mini-batch quadruples sampling based on the sampling weight. The formula for calculating the sampling weight is as follows: Where: κ represents the sampling weight of the priority experience replay mechanism; This represents the total number of experience tuples stored for each agent in the shared experience pool. This indicates the priority of a particular empirical tuple among all currently stored empirical tuples; p represents the absolute value of the TD error between a particular empirical tuple and all other empirical tuples. Let sum(·) represent the set of absolute values ​​of the TD errors of a certain empirical tuple and all other empirical tuples, where sum(·) is a summation function; and φ represents a sampling weight coefficient. and Let s' and a' represent the value function and target value function of agent i, respectively, and let s' and a' represent the cable operation characteristic data and target action value of a certain empirical tuple in the next time slot. This represents the target value of agent i; The joint loss function utilizing the multi-agent dual-adversarial deep Q-network algorithm combined with the priority experience replay mechanism is expressed as: In each update round, the agent is trained by minimizing the joint loss function constructed above.

7. The underground cable safety assessment system based on cloud-edge collaboration and multi-agent deep reinforcement learning as described in claim 6, characterized in that, The system also includes: The system deployment module is used to deploy the multi-agent dual-adversarial deep Q network cable safety status assessment model obtained after training. The specific deployment steps are as follows: save the network parameters of the dual-adversarial deep Q network in the corresponding agent of each monitoring node after training in the cloud computing service center, read the multi-source data fusion features transmitted by each edge service node in real time, and evaluate the safety status of each cable operation monitoring point.

8. A computer-readable storage medium, characterized in that, It stores computer instructions, which, when executed, perform the underground cable safety assessment method based on cloud-edge collaboration and multi-agent deep reinforcement learning as described in any one of claims 1-5.

Images