Cable-stayed bridge structure failure mode searching method based on deep reinforcement learning and self-game strategy

Through deep reinforcement learning and self-game strategies, it is transformed into deep neural networks to search for failure modes of large bridge structures, solving the problem of ‘combination explosion’, and achieving efficient and high-precision failure mode search, suitable for structural health monitoring.

CN119066920APending Publication Date: 2024-12-03HARBIN INST OF TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411134557.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art is susceptible to the ‘combination explosion’ problem when searching for failure mode of large bridge structures, resulting in huge calculations and complex search processes, making it difficult to perform.

Method used

Using methods based on deep reinforcement learning and self-gaming strategies, the search process of structural system failure mode is transformed into a sequential decision-making process of deep reinforcement learning. By designing self-gaming processes and scoring functions, the selection strategy of deep neural network learning components is reduced. Computational complexity and dependence on training process.

Benefits of technology

It realizes efficient and high-precision search of the structural failure mode of large cable-stayed bridges, reduces the impact of the 'combination explosion' problem, improves training efficiency and stability, reduces the dependence on prior knowledge, and is suitable for structural health monitoring technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066920A_ABST
    Figure CN119066920A_ABST
Patent Text Reader

Abstract

The invention relates to a cable-stayed bridge structure failure mode searching method based on deep reinforcement learning and a self-game strategy. The invention relates to the technical field of civil engineering structure health monitoring, and converts a component selection process based on a failure probability in a structural system failure mode search process into a sequential decision process based on deep reinforcement learning, so that the search process does not need complex failure probability calculation and state conversion calculation any more, and the search efficiency is improved. And the influence of the'combined explosion 'problem on the search process is reduced. According to the self-gaming strategy, dependence on prior knowledge in the training process can be effectively reduced, the effect of a result obtained through finite element analysis in each time in the scoring function calculation process is fully played, the number of times of finite element analysis is reduced, and the training process is more intelligent.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of civil engineering structure health monitoring, and is a cable-stayed bridge structure failure mode search method based on deep reinforcement learning and self-game strategy. Background Art

[0002] Bridge structures play a vital role in modern transportation. They can significantly enhance connectivity between regions, ease traffic congestion, and improve the overall operational efficiency of transportation infrastructure. Large-scale construction of bridge structures occurred between the 1990s and the early 21st century. At present, many bridges around the world have been in service for more than 30 years. During the service life of bridge structures, various factors such as environmental conditions (wind, snow, temperature changes), human factors (vehicle loads, collisions), and extreme events (earthquakes, traffic congestion) will affect the bridge structure, resulting in various forms and degrees of damage to the structure such as damage, cracks, fractures and corrosion, which will have an adverse effect on the bearing capacity, durability and reliability of the bridge.

[0003] In order to ensure the safety and reliability of bridge structures during their service life and to improve their service life, safety assessment, reliability analysis and life prediction of bridge structures have become important research contents in bridge engineering. Due to the huge volume of bridge structures, complex and changeable structural forms, and the fact that the bearing capacity is affected by many factors, bridge structures are often converted into structural systems composed of basic components and analyzed during reliability analysis. In the field of structural system reliability research, the reliability analysis of structural systems is divided into two processes: failure mode search and multi-mode failure probability calculation. Among them, the failure mode of the structural system can reflect important information such as the failure mechanism, weak links and bearing capacity of the structure, and plays an important role in the safety assessment, inspection and maintenance of the structure.

[0004] According to the search method and judgment basis of failed components, the existing structural system failure mode search methods can be divided into two categories: failure criterion method and failure probability method. In the search process of the failure criterion method, the most unfavorable component is selected by defining and based on the risk degree index of the components in the structural system under load. This type of method has gradually developed in engineering design and practical applications, and uses engineering experience as the design basis for component failure criteria. Common failure criteria include component bearing ratio, effective bearing ratio, strength ratio, system strength, minimum load increment criterion, etc. The advantages of this type of method are mainly reflected in the small amount of calculation required and simple application, and it has clear physical meaning in the search process. However, as an early research result of structural system reliability, this type of method lacks rigorous theoretical derivation and practical verification, making it difficult to ensure the accuracy of failure mode search.

[0005] With the large number of failure probability solution methods proposed in the field of component reliability research, the failure mode search method of structural system has also been improved and enhanced. The selection basis of failed components in the search process has gradually changed to the failure probability of components. Representative methods include cutoff enumeration method, branch-and-bound method, elastic modulus reduction method, geometric method, β-bound method, etc. This type of method takes component reliability as the theoretical support, and the search process is the process of gradual failure of components in the structural system. It is more in line with the failure mechanism of the structural system and the essence of the reliability of the structural system. Therefore, the search results are more reliable and have a wider range of applications. However, when applied to structural systems with a large number of complex components, the search process will be limited by the "combinatorial explosion" problem, and the required calculation amount is huge, making the search difficult. If the search scale is controlled by branch-and-bound technology, it may lead to the loss or redundancy of failure modes.

[0006] In recent years, deep learning theory has developed rapidly and has been successfully applied to various research fields. Due to its powerful ability to solve high-dimensional nonlinear problems, deep learning has also been widely used in the field of structural reliability research. The present invention uses the high-dimensional nonlinear expression ability of deep learning and the decision-making ability of reinforcement learning to carry out research on the failure mode search method of cable-stayed bridge structure based on deep reinforcement learning and self-game strategy, to deal with the "combinatorial explosion" problem in the search process, and to achieve efficient and high-precision search of failure modes of large cable-stayed bridge structures. Summary of the invention

[0007] In view of the shortcomings of the prior art, the failure mode search of structural systems is an important research content of structural system reliability, and plays an irreplaceable role in the fields of structural reliability analysis, safety assessment and life prediction. However, the existing failure mode search methods are often affected by the "combinatorial explosion" problem when applied to large bridge engineering structures, making the search process very complicated and requiring a lot of calculations, resulting in difficulty in searching. In view of this problem, the purpose of the present invention is to propose a structural system failure mode search technology based on deep reinforcement learning and self-game strategy to achieve efficient and high-precision search for failure modes of cable-stayed bridge structures.

[0008] The present invention provides a method for searching failure modes of cable-stayed bridge structures based on deep reinforcement learning and self-game strategy. The present invention provides the following technical solutions:

[0009] A failure mode search method for a cable-stayed bridge structure based on deep reinforcement learning and self-game strategy comprises the following steps:

[0010] Step 1: Divide the cable-stayed bridge structure into a structural system composed of basic components, establish a finite element model based on the structural system design data, and establish a sample space for failure mode search based on the influencing factors and uncertainties of the structural system;

[0011] Step 2: Convert the failure mode search process of the structural system into a sequential decision-making process based on deep reinforcement learning, and convert the failure state of the structural system and the corresponding failed components into the state and action of deep reinforcement learning;

[0012] Step 3: According to the failure criteria of the structural system, the load effect and resistance of the components, a self-game process based on deep reinforcement learning and a corresponding scoring function are designed. Samples are extracted from the sample space to establish a training set. The scores of each component in each sample in the training set at different failure stages are calculated using a finite element model.

[0013] Step 4: Establish an asynchronous multi-process training framework based on deep reinforcement learning and self-game strategy, and enable the deep neural network to learn the selection strategy of failed components under the guidance of the scoring function;

[0014] Step 5: Use the trained deep neural network to search the sample space of the structural system to obtain the failure mode of the cable-stayed bridge structural system.

[0015] Preferably, the step 1 is specifically:

[0016] According to the type and function of each component, the structure is divided into a structural system composed of components, and a finite element model is established based on the design data, including all the load-bearing components that make up the structural system;

[0017] By monitoring the influencing factors of the structural system reflected in the data and considering the uncertainty of material properties and bearing capacity, a sample space of the structural system is established.

[0018] Preferably, the step 2 is specifically:

[0019] According to the influencing factors of the structural system, the considered randomness and the current failure state of each component, the failure state of the structural system in the current failure stage is established;

[0020] The failure state of the structural system is converted into a state based on deep reinforcement learning, the most unfavorable component selected according to the failure state is converted into an action based on deep reinforcement learning, and a failure mode search framework of the structural system based on deep reinforcement learning is established.

[0021] Preferably, the step 3 is specifically:

[0022] According to the failure component selection process in the failure mode search of the structural system, a self-game strategy is designed. In the self-game process, the deep neural network makes two choices according to the failure state of the structural system, corresponding to two components respectively; according to the corresponding scores of the components, the component with the higher score wins; during the training process, the deep neural network gradually learns how to select the component with the highest corresponding score according to the current failure state, that is, the most unfavorable component through the self-game process;

[0023] The deep neural network gradually learns the selection strategy of the most unfavorable component through the wins and losses in the self-game process. The score of the component needs to reflect its risk level in the current failure stage, so that the most unfavorable component corresponds to the highest score. The scoring function designed based on the above needs is calculated according to the following formula:

[0024] .Score(t)={Score(c 1 |t),SCOre(c 2 |t),…,SCOreT(c i |t),…,Score(c n |t)}.

[0025]

[0026] Among them, Score(c i |t) is the component c in the tth failure stage i The corresponding score, and are respectively the component c in the tth failure stage i The load effect and resistance of the components are determined by directly extracting samples in the sample space to establish a training set. The load effects of the components required in the scoring function at different failure stages of each sample are calculated through finite element analysis, and then the score of each component is calculated.

[0027] Preferably, the step 4 is specifically:

[0028] Step 4.1: In the training process based on deep reinforcement learning and self-game strategy, the deep neural network selects the failed component according to the failure state of the sample and transfers to the next failure stage of the sample. The above process is repeated until the structural system fails. The failure state and failed component correspond to the state and action in deep reinforcement learning respectively.

[0029] Step 4.2: The self-game process in training is based on the Monte Carlo tree search design. The search tree consists of multiple nodes, corresponding to the components in the structural system. Each child node is connected to its parent node through an edge, and the following statistical information of the node is stored in the edge:

[0030] {N(s i ,a i),W(s i ,a i ),Q(s i ,a i ),P(s i ,a i )}

[0031] Among them, N(s i ,a i ) is the number of node visits, W(s i ,a i ) is the total action value of the node, Q(s i ,a i ) is the average action value of the node, P(s i ,a i ) is the node prior probability. In the self-game process, before selecting an action, the deep neural network needs to perform multiple Monte Carlo tree search simulations based on the current failure state to determine the choice of action. Each simulation includes three steps: selection, expansion and evaluation, and backtracking.

[0032] The search probability obtained by the Monte Carlo tree search in the self-game process reflects the selection strategy of the most unfavorable component. The deep neural network updates its parameters so that the prior probability of its output gradually approaches the search probability. The network parameter update is performed according to the following formula:

[0033] l=-π T logp+c||θ|| 2

[0034] Among them, π is the search probability obtained in the self-game process; p is the prior probability of the component being selected, which is output by the deep neural network; c is the L2 weight decay coefficient;

[0035] Step 4.3: The asynchronous multi-process framework used in the training process includes a global network and multiple local networks. Before the training starts, each local network obtains a random sample from the sample set and starts its own training process based on deep reinforcement learning. The failure state and corresponding search probability data of the sample are collected through the self-game process and passed to the global network. The global network updates the parameters according to the data provided by the local network and shares the updated parameters with each local network again. During the training process, the local network is used for training and collecting data, and the global network is used to store, update and share network parameters.

[0036] Preferably, the selection, expansion, evaluation and backtracking are specifically:

[0037] In the selection step, starting from the root node, child nodes are selected step by step according to the following formula:

[0038]

[0039] in, This is the game stage. i . The node corresponding to the maximum value in a i The game stage i The node selected in U(s i ,a i ) is the game stage s i Middle node a i The profit signal of puct is a constant defining the degree of exploration of the Monte Carlo tree search; b is the node a i When the parent node of the search tree reaches the leaf node, the unexpanded nodes are expanded and evaluated according to the following formula:

[0040]

[0041] p=P(s L ,A(s L ))

[0042] Among them, score(c=a L |t) is node a L The score in the tth failure stage; P(s L ,A(s L )) is the game stage s L The prior probability of all legal nodes in aL For node a L The prior probability is output by the deep neural network according to the failure state. After the leaf node is expanded, the search tree is backtracked according to the following formula:

[0043] N(s i ,a i )=N(s i ,a i )+1

[0044] W(s i ,a i )=W(s i ,a i )+v

[0045]

[0046] After multiple simulations, the search probability of each node is calculated according to the following formula:

[0047]

[0048] Among them, π(a|s 1) is the search probability of node ai in the game stage s1; τ is a parameter used to adjust the balance between exploration and exploitation, and a choice is made in the self-game process based on the calculated search probability.

[0049] Preferably, the step 5 is specifically:

[0050] Use the trained deep neural network to search a large number of random samples in the sample space, establish a failure mode search process based on each random sample, and select the most unfavorable component failure mode through the deep neural network;

[0051] The failure mode search results of random samples in the sample space are statistically analyzed to obtain the main failure modes of the structural system.

[0052] A cable-stayed bridge structure failure mode search system based on deep reinforcement learning and self-game strategy, the system comprising:

[0053] A model building module, wherein the model building module divides the cable-stayed bridge structure into a structural system composed of basic components, builds a finite element model according to the structural system design data, and builds a sample space for failure mode search according to the influencing factors and uncertainties considered for the cable-stayed bridge structural system;

[0054] A conversion module, wherein the conversion module converts the failure mode search process of the structural system into a sequential decision-making process based on deep reinforcement learning, and converts the failure state of the structural system and the corresponding failure component into the state and action of deep reinforcement learning;

[0055] A training module, wherein the training module designs a self-game process based on deep reinforcement learning and a corresponding scoring function according to the failure criteria of the structural system, the load effect and resistance of the components, extracts samples from the sample space to establish a training set, and calculates the scores of each component of each sample in the training set at different failure stages through a finite element model;

[0056] A selection strategy module, wherein the selection strategy module establishes an asynchronous multi-process training framework based on deep reinforcement learning and self-game strategy, and enables the deep neural network to learn a selection strategy for failed components under the guidance of a scoring function;

[0057] A failure search module uses a trained deep neural network to search the sample space of the structural system to obtain the failure mode of the cable-stayed bridge structural system.

[0058] A computer-readable storage medium stores a computer program, which is executed by a processor to implement a failure mode search method for a cable-stayed bridge structure based on deep reinforcement learning and self-game strategy.

[0059] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a method for searching failure modes of a cable-stayed bridge structure based on deep reinforcement learning and self-game strategy is implemented.

[0060] The present invention has the following beneficial effects:

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] The present invention proposes a method for searching failure modes of cable-stayed bridge structures based on deep reinforcement learning and self-game strategy. The present invention transforms the component selection process based on failure probability in the failure mode search process of the structural system into a sequential decision-making process based on deep reinforcement learning, so that the search process no longer requires complex failure probability calculations and state transformation calculations, and reduces the impact of the "combinatorial explosion" problem on the search process;

[0063] The self-game strategy of the present invention can effectively reduce the reliance on prior knowledge during the training process, give full play to the effect of the results obtained from each finite element analysis during the calculation of the scoring function, reduce the number of finite element analyses, and make the training process more intelligent;

[0064] The scoring function of the present invention can reflect the risk difference of components in the structural system in the current failure state, reflect the relationship between the load effect and resistance of the components, and guide the deep neural network to learn the most unfavorable component selection strategy;

[0065] The asynchronous multi-process training method of the present invention can improve the training efficiency and stability of deep neural networks and avoid local optimal problems during training;

[0066] The training and search process of the method of the present invention is established based on the search process of failure modes, and its application is not limited by the type of structure and can be combined with the structural health monitoring technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0068] Figure 1 A flow chart of the self-game strategy designed for the failure mode search process of the structural system;

[0069] Figure 2 This is a flowchart of asynchronous multi-process training based on deep reinforcement learning and self-game strategy;

[0070] Figure 3 The finite element model diagram and the cable-stayed bridge structural component division diagram established based on the cable-stayed bridge design data;

[0071] Figure 4 The schematic diagram of vehicle load;

[0072] Figure 5 It is a schematic diagram of two-dimensional transformation of failure state data of cable-stayed bridge structure system;

[0073] Figure 6 This is the architecture diagram of the convolutional neural network used;

[0074] Figure 7 This is a training flowchart based on deep reinforcement learning and self-game strategy;

[0075] Figure 8 Schematic diagram of the searched failure mode of the cable-stayed bridge structure. DETAILED DESCRIPTION

[0076] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0078] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0079] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0080] The present invention is described in detail below in conjunction with specific embodiments. Specific embodiment one:

[0082] according to Figure 1-Figure 8 As shown, the specific optimization technical solution adopted by the present invention to solve the above technical problems is: the present invention relates to a failure mode search method for a cable-stayed bridge structure based on deep reinforcement learning and self-game strategy.

[0083] A failure mode search method for a cable-stayed bridge structure based on deep reinforcement learning and self-game strategy comprises the following steps:

[0084] Step 1: Divide the cable-stayed bridge structure into a structural system composed of basic components, establish a finite element model based on the structural system design data, and establish a sample space for failure mode search based on the influencing factors and uncertainties of the structural system;

[0085] Step 2: Convert the failure mode search process of the structural system into a sequential decision-making process based on deep reinforcement learning, and convert the failure state of the structural system and the corresponding failed components into the state and action of deep reinforcement learning;

[0086] Step 3: According to the failure criteria of the structural system, the load effect and resistance of the components, a self-game process based on deep reinforcement learning and a corresponding scoring function are designed. Samples are extracted from the sample space to establish a training set. The scores of each component in each sample in the training set at different failure stages are calculated using a finite element model.

[0087] Step 4: Establish an asynchronous multi-process training framework based on deep reinforcement learning and self-game strategy, and enable the deep neural network to learn the selection strategy of failed components under the guidance of the scoring function;

[0088] Step 5: Use the trained deep neural network to search the sample space of the structural system to obtain the failure mode of the cable-stayed bridge structural system.

[0089] The present invention proposes a method for searching failure modes of cable-stayed bridge structures based on deep reinforcement learning and self-game strategy. The present invention transforms the component selection process based on failure probability in the failure mode search process of the structural system into a sequential decision-making process based on deep reinforcement learning, so that the search process no longer requires complex failure probability calculations and state transformation calculations, and reduces the impact of the "combinatorial explosion" problem on the search process;

[0090] The self-game strategy of the present invention can effectively reduce the reliance on prior knowledge during the training process, give full play to the results obtained from each finite element analysis during the scoring function calculation process, reduce the number of finite element analyses, and make the training process more intelligent. Specific embodiment 2:

[0092] The difference between the second embodiment of the present invention and the first embodiment is that:

[0093] The step 1 is specifically as follows:

[0094] According to the type and function of each component, the structure is divided into a structural system composed of components, and a finite element model is established based on the design data, including all the load-bearing components that make up the structural system;

[0095] By monitoring the influencing factors of the structural system reflected in the data and considering the uncertainty of material properties and bearing capacity, a sample space of the structural system is established. Specific embodiment three:

[0097] The difference between the third embodiment of the present invention and the second embodiment is that:

[0098] The step 2 is specifically as follows:

[0099] According to the influencing factors of the structural system, the considered randomness and the current failure state of each component, the failure state of the structural system in the current failure stage is established;

[0100] The failure state of the structural system is converted into a state based on deep reinforcement learning, the most unfavorable component selected according to the failure state is converted into an action based on deep reinforcement learning, and a failure mode search framework of the structural system based on deep reinforcement learning is established. Specific embodiment four:

[0102] The difference between the fourth embodiment of the present invention and the third embodiment is that:

[0103] The step 3 is specifically as follows:

[0104] According to the failure component selection process in the failure mode search of the structural system, a self-game strategy is designed. In the self-game process, the deep neural network makes two choices according to the failure state of the structural system, corresponding to two components respectively; according to the corresponding scores of the components, the component with the higher score wins; during the training process, the deep neural network gradually learns how to select the component with the highest corresponding score according to the current failure state, that is, the most unfavorable component through the self-game process;

[0105] The deep neural network gradually learns the selection strategy of the most unfavorable component through the wins and losses in the self-game process. The score of the component needs to reflect its risk level in the current failure stage, so that the most unfavorable component corresponds to the highest score. The scoring function designed based on the above needs is calculated according to the following formula:

[0106] .Score(t)={Score(c 1 |t),Score(c 2 |t),…,SCOre(c i|t),…,Score(c n |t)).

[0107]

[0108] Among them, Score(c i |t) is the component c in the tth failure stage i The corresponding score, and are respectively the component c in the tth failure stage i The load effect and resistance of the components are determined by directly extracting samples in the sample space to establish a training set. The load effects of the components required in the scoring function at different failure stages of each sample are calculated through finite element analysis, and then the score of each component is calculated. Specific embodiment five:

[0110] The difference between the fifth embodiment of the present invention and the fourth embodiment is that:

[0111] The step 4 is specifically as follows:

[0112] Step 4.1: In the training process based on deep reinforcement learning and self-game strategy, the deep neural network selects the failed component according to the failure state of the sample and transfers to the next failure stage of the sample. The above process is repeated until the structural system fails. The failure state and failed component correspond to the state and action in deep reinforcement learning respectively.

[0113] Step 4.2: The self-game process in training is based on the Monte Carlo tree search design. The search tree consists of multiple nodes, corresponding to the components in the structural system. Each child node is connected to its parent node through an edge, and the following statistical information of the node is stored in the edge:

[0114] {N(s i ,a i ),W(s i ,a i ),Q(s i ,a i ),P(s i ,a i )}

[0115] Among them, N(s i ,a i ) is the number of node visits, W(s i ,a i ) is the total action value of the node, Q(s i ,a i ) is the average action value of the node, P(s i ,a i) is the node prior probability. In the self-game process, before selecting an action, the deep neural network needs to perform multiple Monte Carlo tree search simulations based on the current failure state to determine the choice of action. Each simulation includes three steps: selection, expansion and evaluation, and backtracking.

[0116] The search probability obtained by the Monte Carlo tree search in the self-game process reflects the selection strategy of the most unfavorable component. The deep neural network updates its parameters so that the prior probability of its output gradually approaches the search probability. The network parameter update is performed according to the following formula:

[0117] l=-π T logp+c||θ|| 2

[0118] Among them, π is the search probability obtained in the self-game process; p is the prior probability of the component being selected, which is output by the deep neural network; c is the L2 weight decay coefficient;

[0119] Step 4.3: The asynchronous multi-process framework used in the training process includes a global network and multiple local networks. Before the training starts, each local network obtains a random sample from the sample set and starts its own training process based on deep reinforcement learning. The failure state and corresponding search probability data of the sample are collected through the self-game process and passed to the global network. The global network updates the parameters according to the data provided by the local network and shares the updated parameters with each local network again. During the training process, the local network is used for training and collecting data, and the global network is used to store, update and share network parameters. Specific embodiment six:

[0121] The difference between the sixth embodiment of the present invention and the fifth embodiment is that:

[0122] The selection, expansion, evaluation and backtracking are as follows:

[0123] In the selection step, starting from the root node, child nodes are selected step by step according to the following formula:

[0124]

[0125] in, This is the game stage. i . The node corresponding to the maximum value in a i The game stage i The node selected in U(s i ,a i ) is the game stage s i Middle node a i The profit signal of puct is a constant defining the degree of exploration of the Monte Carlo tree search; b is the node ai When the parent node of the search tree reaches the leaf node, the unexpanded nodes are expanded and evaluated according to the following formula:

[0126]

[0127] p=P(s L ,A(s L ))

[0128] Among them, score(c=a L |t) is node a L The score in the tth failure stage; P(s L ,A(s L )) is the game stage s L The prior probability of all legal nodes in aL For node a L The prior probability is output by the deep neural network according to the failure state. After the leaf node is expanded, the search tree is backtracked according to the following formula:

[0129] N(s i ,a i )=N(s i ,a i )+1

[0130] W(s i ,a i )=W(s i ,a i )+v

[0131]

[0132]

[0133] After multiple simulations, the search probability of each node is calculated according to the following formula:

[0134]

[0135] Among them, π(a|s 1 ) is the search probability of node ai in the game stage s1; τ is a parameter used to adjust the balance between exploration and exploitation, and a choice is made in the self-game process based on the calculated search probability. Specific embodiment seven:

[0137] The difference between the seventh embodiment of the present invention and the sixth embodiment is that:

[0138] The step 5 is specifically as follows:

[0139] Use the trained deep neural network to search a large number of random samples in the sample space, establish a failure mode search process based on each random sample, and select the most unfavorable component failure mode through the deep neural network;

[0140] The failure mode search results of random samples in the sample space are statistically analyzed to obtain the main failure modes of the structural system. Specific embodiment eight:

[0142] The difference between the eighth embodiment of the present invention and the seventh embodiment is that:

[0143] The present invention provides a cable-stayed bridge structure failure mode search system based on deep reinforcement learning and self-game strategy, the system comprising:

[0144] A model building module, wherein the model building module divides the cable-stayed bridge structure into a structural system composed of basic components, builds a finite element model according to the structural system design data, and builds a sample space for failure mode search according to the influencing factors and uncertainties considered for the cable-stayed bridge structural system;

[0145] A conversion module, wherein the conversion module converts the failure mode search process of the structural system into a sequential decision-making process based on deep reinforcement learning, and converts the failure state of the structural system and the corresponding failure component into the state and action of deep reinforcement learning;

[0146] A training module, wherein the training module designs a self-game process based on deep reinforcement learning and a corresponding scoring function according to the failure criteria of the structural system, the load effect and resistance of the components, extracts samples from the sample space to establish a training set, and calculates the scores of each component of each sample in the training set at different failure stages through a finite element model;

[0147] A selection strategy module, wherein the selection strategy module establishes an asynchronous multi-process training framework based on deep reinforcement learning and self-game strategy, and enables the deep neural network to learn a selection strategy for failed components under the guidance of a scoring function;

[0148] A failure search module uses a trained deep neural network to search the sample space of the structural system to obtain the failure mode of the cable-stayed bridge structural system. Specific embodiment nine:

[0150] The difference between the ninth embodiment of the present invention and the eighth embodiment is that:

[0151] The present invention provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement a failure mode search method for a cable-stayed bridge structure based on deep reinforcement learning and self-game strategy. Specific embodiment ten:

[0153] The difference between the tenth embodiment of the present invention and the ninth embodiment is that:

[0154] The present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a method for searching failure modes of a cable-stayed bridge structure based on deep reinforcement learning and self-game strategy is implemented. Specific embodiment eleven:

[0156] The difference between the eleventh embodiment of the present invention and the tenth embodiment is that:

[0157] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0158] The present invention is a cable-stayed bridge structure failure mode search technology based on deep reinforcement learning and self-game strategy, comprising:

[0159] (1) Divide the structure into a structural system composed of basic components, establish a finite element model based on the design data of the structural system, and establish a sample space for failure mode search based on the influencing factors and uncertainties of the structural system;

[0160] (2) The failure mode search process of the structural system is transformed into a sequential decision-making process based on deep reinforcement learning, and the failure state of the structural system and the corresponding failed components are transformed into the state and action of deep reinforcement learning;

[0161] (3) According to the failure criteria of the structural system, the load effects and resistance of the components, a self-game process based on deep reinforcement learning and the corresponding scoring function are designed. Samples are extracted from the sample space to establish a training set. The scores of each component in each sample in the training set at different failure stages are calculated using a finite element model.

[0162] (4) An asynchronous multi-process training framework is established based on deep reinforcement learning and self-game strategy, and the deep neural network learns the selection strategy of failed components under the guidance of the scoring function;

[0163] (5) Use the trained deep neural network to search the sample space of the structural system to obtain the failure mode of the structural system.

[0164] The purpose of this implementation is to propose a structural system failure mode search technology based on deep reinforcement learning and self-game strategy, which can be applied to the failure mode search of large cable-stayed bridge structures.

[0165] The core of the present invention is to propose a structural system failure mode search method based on deep reinforcement learning and self-game strategy. The method is mainly divided into two steps. The first step is to establish a structural system failure mode search process based on deep reinforcement learning, transform the failure component selection process based on failure probability into a sequential decision-making process based on deep reinforcement learning, and output the failure mode through a decision-making based on a deep neural network to avoid complex failure probability calculations and failure state conversion calculations. The second step is to design a self-game strategy based on the logical relationship in the failure mode search process, combine the load effect of the component and the resistance design scoring function, and guide the deep neural network to learn the failure component selection strategy through the victory or defeat in the self-game process, reduce the demand for prior knowledge in the training process, and improve the efficiency of finite element analysis.

[0166] Step (1) is specifically as follows:

[0167] (2a) According to the type and function of each component, the structure is divided into a structural system composed of components, and a finite element model is established based on the design data, including all load-bearing components that make up the structural system;

[0168] (2b) Establish a sample space for the structural system by monitoring the influencing factors of the structural system reflected in the monitoring data and the uncertainties in material properties, bearing capacity, etc.

[0169] Step (2) is specifically as follows:

[0170] (3a) Establish the failure state of the structural system in the current failure stage based on the influencing factors of the structural system, the considered randomness and the current failure state of each component;

[0171] (3b) The failure state of the structural system is converted into a state based on deep reinforcement learning, the most unfavorable component selected according to the failure state is converted into an action based on deep reinforcement learning, and a failure mode search framework of the structural system based on deep reinforcement learning is established.

[0172] Step (3) is specifically as follows:

[0173] (4a) According to the failure component selection process in the failure mode search of the structural system, a self-game strategy is designed. In the self-game process, the deep neural network makes two choices based on the failure state of the structural system, corresponding to two components respectively. According to the corresponding scores of the components, the component with the higher score wins. During the training process, the deep neural network gradually learns how to select the component with the highest corresponding score according to the current failure state, that is, the most unfavorable component through the self-game process;

[0174] (4b) In order for the deep neural network to gradually learn the selection strategy of the most unfavorable component through the wins and losses in the self-game process, the component score needs to reflect its risk level in the current failure stage, so that the most unfavorable component corresponds to the highest score. The scoring function designed based on the above needs is calculated according to the following formula:

[0175] .Score(t)={Score(c 1 |t),Score(c 2 |t),…,Score(c i |t),…,Score(c n |t)}.

[0176]

[0177] Among them, Score(c i |t) is the component c in the tth failure stage i The corresponding score, and are respectively the component c in the tth failure stage i The load effect and resistance of the components are directly extracted from the sample space to establish a training set. The load effect of the components required in the scoring function at different failure stages of each sample is calculated through finite element analysis, and then the score of each component is calculated.

[0178] Step (4) is specifically as follows:

[0179] (5a) In the training process based on deep reinforcement learning and self-game strategy, the deep neural network selects the failed component according to the failure state of the sample and transfers to the next failure stage of the sample. The above process is repeated until the structural system fails. Among them, the failure state and the failed component correspond to the state and action in deep reinforcement learning respectively;

[0180] (5b) The self-game process in training is based on the Monte Carlo tree search design. The search tree consists of multiple nodes, corresponding to the components in the structural system. Each child node is connected to its parent node through an edge, and the following statistical information of the node is stored in the edge:

[0181] {N(s i ,a i ),W(s i ,a i ),Q(s i ,a i ),P(s i ,a i )}

[0182] Among them, N(s i ,a i ) is the number of node visits, W(si ,a i ) is the total action value of the node, Q(s i ,a i ) is the average action value of the node, P(s i ,a i ) is the node prior probability. In the self-game process, before selecting an action, the deep neural network needs to perform multiple Monte Carlo tree search simulations based on the current failure state to determine the action selection. Each simulation includes three steps: selection, expansion and evaluation, and backtracking. In the selection step, starting from the root node, the child nodes are gradually selected according to the following formula:

[0183]

[0184] in, This is the game stage. i . The node corresponding to the maximum value in a i The game stage i The node selected in U(s i ,a i ) is the game stage s i Middle node a i The profit signal of puct is a constant defining the degree of exploration of the Monte Carlo tree search; b is the node a i When reaching the leaf node of the search tree, the unexpanded nodes are expanded and evaluated according to the following formula:

[0185]

[0186] p=P(s L ,A(s L ))

[0187] Among them, score(c=a L |t) is node a L The score in the tth failure stage; P(s L ,A(s L )) is the game stage s L The prior probability of all legal nodes in ; For node a L The prior probability is output by the deep neural network according to the failure state. After the leaf node is expanded, the search tree is backtracked according to the following formula: N(s i ,a i )=N(s i ,a i )+1

[0188] W(s i ,a i )=W(s i,a i )+v

[0189]

[0190] After multiple simulations, the search probability of each node is calculated according to the following formula:

[0191]

[0192] Among them, π(as 1 ) is the game stage s 1 The search probability of the middle node ai; τ is a parameter used to adjust the balance between exploration and exploitation. According to the calculated search probability, a choice is made in the self-game process.

[0193] The search probability obtained by Monte Carlo tree search in the self-game process reflects the selection strategy of the most unfavorable component. The deep neural network gradually approaches the search probability by updating its parameters. The network parameter update is performed according to the following formula.

[0194] l=-π T logp+c‖θ|| 2

[0195] Among them, π is the search probability obtained in the self-game process; p is the prior probability of the component being selected, which is output by the deep neural network; c is L 2 Weight decay coefficient.

[0196] (5c) The asynchronous multi-process framework used in the training process includes a global network and multiple local networks. Before the training starts, each local network obtains a random sample from the sample set, starts its own training process based on deep reinforcement learning, collects the failure state and corresponding search probability data of the sample through the self-game process, and passes it to the global network. The global network updates the parameters based on the data provided by the local network and shares the updated parameters with each local network again. During the training process, the local network is used for training and collecting data, and the global network is used to store, update and share network parameters.

[0197] Step (5) is specifically as follows:

[0198] (6a) Use the trained deep neural network to search a large number of random samples in the sample space, establish a failure mode search process based on each random sample, and select the most unfavorable component failure mode through the deep neural network;

[0199] (6b) Perform statistical analysis on the failure mode search results of random samples in the sample space to obtain the main failure modes of the structural system.

[0200] A computer program written in Python was completed and applied to the failure mode search of a large-span steel box girder cable-stayed bridge under the influence of vehicle load and cable corrosion. This application verifies the feasibility and practicality of the present invention. The effects of the present invention are described below through specific examples.

[0201] like Figure 3 As shown in Figure 1, a 1:1 scale structural finite element model was established based on the design data of the cable-stayed bridge, and its components were divided and converted into a structural system. The finite element model includes all the load-bearing components of the cable-stayed bridge structure. The total length of the cable-stayed bridge is 964m, of which the largest span is 460m. The basic information of the finite element model is shown in Table 1.

[0202] Table 1

[0203]

[0204] The limit state function of the cable-stayed bridge structure under the influence of vehicle load and cable corrosion is as follows:

[0205]

[0206] Where L is the maximum span of the cable-stayed bridge structure; Δ(VLA c ) is the vertical deflection of the cable-stayed bridge structure under the influence of vehicle load and cable corrosion.

[0207] The cable-stayed bridge has six lanes in both directions. The vehicle load on the bridge is established through a vehicle load model based on monitoring data. The axle weight, wheelbase, speed and headway of vehicles on each lane can be considered separately. Figure 4 The vehicle load model can be used to consider the vehicle loads on different lanes in each component.

[0208] The uniform cable corrosion model is used to consider the change in the corrosion degree of the cables in the cable-stayed bridge over time. The corrosion process of the steel wire in the cable gradually develops from the outer layer of the cable core to the inner layer of the cable core, and is calculated according to the following formula:

[0209]

[0210] F(σ)=0.9938+0.00268σ

[0211]

[0212] Among them, A w (t) is the effective bearing area of ​​the wire cross section; D 0 is the diameter of the uncorroded wire section; r corr is the corrosion rate of the steel wire; F(Cl - ) is the chloride ion concentration adjustment coefficient; F(σ) is the stress adjustment coefficient; t is the service life; Tcorr is the corrosion start time; d eq,i is the corrosion depth of the i-th layer of cable core; R c is the corrosion diffusion rate of the cable. The parameters used in the cable corrosion model are given in Table 2.

[0213] Table 2

[0214]

[0215]

[0216] In the process of failure mode search, the failure state of the cable-stayed bridge structure system includes the vehicle load borne by the structure system, the effective load-bearing cross-sectional area of ​​the cable and the failure state of each component, which is expressed by the following formula:

[0217]

[0218] Where VL is the vehicle load random variable; A(t) is the effective load-bearing cross-sectional area of ​​the cable random variable; c for the component i In the failure mode search process, the failure state data of the cable-stayed bridge structure system are sorted into Figure 5 Convert to a two-dimensional matrix using Figure 6 The convolutional neural network shown in Figure 2 observes the transformed failure state and selects the most unfavorable component and Figure 7 A training process based on deep reinforcement learning and self-game strategy is established. The parameters used in the training process are shown in Table 3.

[0219] Table 3

[0220]

[0221] During the training, random samples were collected from the vehicle load model and the cable corrosion model to form a training set and a validation set, which contained a total of 4000 samples. The k-fold cross-validation training method was used for training, and the training set and validation set were divided into 4:1. The performance of the trained convolutional neural network on the validation set is shown in Table 4.

[0222] Table 4

[0223]

[0224] The trained convolutional neural network was used to search for failure modes of cable-stayed bridge structures at 0, 3, 6, 9, 12, and 15 years after the cable corrosion. The search results are shown in Table 5. Figure 8 shown.

[0225] Table 5

[0226]

[0227]

[0228] The method of the present invention can perform efficient and high-precision search for failure modes of large cable-stayed bridge structures taking into account multiple influencing factors, and can effectively deal with the "combination explosion" problem encountered by large and complex engineering structures in the process of failure mode search. The accuracy of the training results of the convolutional neural network is above 90%. The search results of the failure mode can reflect the bearing capacity and weak links of the cable-stayed bridge structure, and can be used as a basis for research such as structural safety assessment, reliability analysis and life prediction, and provide technical support.

[0229] In the description of this specification, the description of reference terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction. In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. Any process or method description in the flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code including one or more executable instructions for implementing the steps of a custom logic function or process, and the scope of the preferred embodiment of the present invention includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the art to which the embodiments of the present invention belong. The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as a sequenced list of executable instructions for implementing the logic function, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or N wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM).In addition, the computer readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, then editing, interpreting or processing in other suitable ways as necessary, and then storing it in a computer memory. It should be understood that the various parts of the present invention can be implemented with hardware, software, firmware, or a combination thereof. In the above-mentioned embodiment, N steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented with hardware, as in another embodiment, any one of the following technologies known in the art or their combination can be used to implement: a discrete logic circuit with a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit with a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0230] The above is only a preferred implementation of a cable-stayed bridge structure failure mode search method based on deep reinforcement learning and self-game strategy. The protection scope of a cable-stayed bridge structure failure mode search method based on deep reinforcement learning and self-game strategy is not limited to the above embodiments. All technical solutions under this idea belong to the protection scope of the present invention. It should be pointed out that for those skilled in the art, several improvements and changes without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A failure mode search method for cable-stayed bridge structure based on deep reinforcement learning and self-game strategy, characterized by: The following steps are involved: Step 1: Divide the cable-stayed bridge structure into a structural system composed of basic components, establish a finite element model based on the structural system design data, and establish a sample space for failure mode search based on the influencing factors and uncertainties of the structural system; Step 2: Convert the failure mode search process of the structural system into a sequential decision-making process based on deep reinforcement learning, and convert the failure state of the structural system and the corresponding failed components into the state and action of deep reinforcement learning; Step 3: According to the failure criteria of the structural system, the load effect and resistance of the components, a self-game process based on deep reinforcement learning and a corresponding scoring function are designed. Samples are extracted from the sample space to establish a training set. The scores of each component in each sample in the training set at different failure stages are calculated using a finite element model. Step 4: Establish an asynchronous multi-process training framework based on deep reinforcement learning and self-game strategy, and enable the deep neural network to learn the selection strategy of failed components under the guidance of the scoring function; Step 5: Use the trained deep neural network to search the sample space of the structural system to obtain the failure mode of the cable-stayed bridge structural system.

2. The method according to claim 1, characterized in that: The step 1 is specifically as follows: According to the type and function of each component, the structure is divided into a structural system composed of components, and a finite element model is established based on the design data, including all the load-bearing components that make up the structural system; By monitoring the influencing factors of the structural system reflected in the data and considering the uncertainty of material properties and bearing capacity, a sample space of the structural system is established.

3. The method according to claim 2, characterized in that: The step 2 is specifically as follows: According to the influencing factors of the structural system, the considered randomness and the current failure state of each component, the failure state of the structural system in the current failure stage is established; The failure state of the structural system is converted into a state based on deep reinforcement learning, the most unfavorable component selected according to the failure state is converted into an action based on deep reinforcement learning, and a failure mode search framework of the structural system based on deep reinforcement learning is established.

4. The method according to claim 3, characterized in that: The step 3 is specifically as follows: According to the failure component selection process in the failure mode search of the structural system, a self-game strategy is designed. In the self-game process, the deep neural network makes two choices according to the failure state of the structural system, corresponding to two components respectively; according to the corresponding scores of the components, the component with the higher score wins; During the training process, the deep neural network gradually learns how to select the component with the highest corresponding score, that is, the most unfavorable component, according to the current failure state through a self-game process; The deep neural network gradually learns the selection strategy of the most unfavorable component through the wins and losses in the self-game process. The score of the component needs to reflect its risk level in the current failure stage, so that the most unfavorable component corresponds to the highest score. The scoring function designed based on the above needs is calculated according to the following formula: Among them, Score(c i |t) is the component c in the tth failure stage i The corresponding score, and are respectively the component c in the tth failure stage i The load effect and resistance of the components are determined by directly extracting samples in the sample space to establish a training set. The load effects of the components required in the scoring function at different failure stages of each sample are calculated through finite element analysis, and then the score of each component is calculated.

5. The method according to claim 4, characterized in that: The step 4 is specifically as follows: Step 4.1: In the training process based on deep reinforcement learning and self-game strategy, the deep neural network selects the failed component according to the failure state of the sample and transfers to the next failure stage of the sample. The above process is repeated until the structural system fails. The failure state and failed component correspond to the state and action in deep reinforcement learning respectively. Step 4.2: The self-game process in training is based on the Monte Carlo tree search design. The search tree consists of multiple nodes, corresponding to the components in the structural system. Each child node is connected to its parent node through an edge, and the following statistical information of the node is stored in the edge: {N(s i ,a i ),W(s i ,a i ),Q(s i ,a i ),P(s i ,a i )} Among them, N(s i ,a i ) is the number of node visits, W(s i ,a i ) is the total action value of the node, Q(s i ,a i ) is the average action value of the node, P(s i ,a i ) is the node prior probability. In the self-game process, before selecting an action, the deep neural network needs to perform multiple Monte Carlo tree search simulations based on the current failure state to determine the choice of action. Each simulation includes three steps: selection, expansion and evaluation, and backtracking. The search probability obtained by the Monte Carlo tree search in the self-game process reflects the selection strategy of the most unfavorable component. The deep neural network updates its parameters so that the prior probability of its output gradually approaches the search probability. The network parameter update is performed according to the following formula: l=-π T logp+c||θ|| 2 Among them, π is the search probability obtained in the self-game process; p is the prior probability of the component being selected, which is output by the deep neural network; c is the L2 weight decay coefficient; Step 4.3: The asynchronous multi-process framework used in the training process includes a global network and multiple local networks. Before the training starts, each local network obtains a random sample from the sample set and starts its own training process based on deep reinforcement learning. The failure state and corresponding search probability data of the sample are collected through the self-game process and passed to the global network. The global network updates the parameters according to the data provided by the local network and shares the updated parameters with each local network again. During the training process, the local network is used for training and collecting data, and the global network is used to store, update and share network parameters.

6. The method according to claim 5, characterized in that: The selection, expansion, evaluation and backtracking are as follows: In the selection step, starting from the root node, child nodes are selected step by step according to the following formula: in, This is the game stage. i . The node corresponding to the maximum value in a i is the node selected in the game stage si; U(s i ,a i ) is the game stage s i Middle node a i The profit signal of puct is a constant defining the degree of exploration of the Monte Carlo tree search; b is the node a i When the parent node of the search tree reaches the leaf node, the unexpanded nodes are expanded and evaluated according to the following formula: p=P(s L ,A(s L )) Among them, score(c=a L |t) is node a L The score in the tth failure stage; P(s L ,A(s L )) is the game stage s L The prior probability of all legal nodes in ; For node a L The prior probability is output by the deep neural network according to the failure state. After the leaf node is expanded, the search tree is backtracked according to the following formula: N(s i ,a i )=N(s i ,a i )+1 W(s i ,a i )=W(s i ,a i )+v After multiple simulations, the search probability of each node is calculated according to the following formula: Among them, π(a|s1) is the search probability of node ai in the game stage s1; τ is a parameter used to adjust the balance between exploration and exploitation, and a choice is made in the self-game process based on the calculated search probability.

7. The method according to claim 6, characterized in that: The step 5 is specifically as follows: Use the trained deep neural network to search a large number of random samples in the sample space, establish a failure mode search process based on each random sample, and select the most unfavorable component failure mode through the deep neural network; The failure mode search results of random samples in the sample space are statistically analyzed to obtain the main failure modes of the structural system.

8. A cable-stayed bridge structure failure mode search system based on deep reinforcement learning and self-game strategy, characterized by: The system comprises: A model building module, wherein the model building module divides the cable-stayed bridge structure into a structural system composed of basic components, builds a finite element model according to the structural system design data, and builds a sample space for failure mode search according to the influencing factors and uncertainties considered for the cable-stayed bridge structural system; A conversion module, wherein the conversion module converts the failure mode search process of the structural system into a sequential decision-making process based on deep reinforcement learning, and converts the failure state of the structural system and the corresponding failure component into the state and action of deep reinforcement learning; A training module, wherein the training module designs a self-game process based on deep reinforcement learning and a corresponding scoring function according to the failure criteria of the structural system, the load effect and resistance of the components, extracts samples from the sample space to establish a training set, and calculates the scores of each component of each sample in the training set at different failure stages through a finite element model; A selection strategy module, wherein the selection strategy module establishes an asynchronous multi-process training framework based on deep reinforcement learning and self-game strategy, and enables the deep neural network to learn a selection strategy for failed components under the guidance of a scoring function; A failure search module uses a trained deep neural network to search the sample space of the structural system to obtain the failure mode of the cable-stayed bridge structural system.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to claims 1-7.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method of claims 1-7 is implemented.

Citation Information

Cited By

  • MCTS and self-game-based confrontation effectiveness evaluation method, system and equipment and medium

    CN119849310A

  • A method for quickly constructing a component failure criterion model

    CN122886328A